SiteGround web hosting suffered a significant four day outage beginning Monday November 8, 2021. It wasn’t until November 12th that they tweeted that they had resolved the problem. Many customers lost rankings in Google and a significant amount of website traffic as the holiday shopping season fast approaches.
Many SiteGround publishers remain upset, largely because of the perceived slow pace of recovery from lost Google Search traffic.
What Caused the SiteGround Problem?
SiteGround provided a statement revealed that the problem was isolated to an issue between Amazon’s Global Accelerator and Google’s crawler.
According to SiteGround:
“On Friday we managed to isolate the Google bot crawling issue to a networking problem that was specific only to Amazon’s Global Accelerator and Google’s crawler bot subnet.
We implemented a fix that bypasses that problem and we’re happy to say that our clients’ sites now get properly crawled and most of them have already returned their rankings.
We are still working closely with both Amazon and Google on finding the cause of the problem.
Based on the latest updates, we suspect it’s a routing issue and Amazon is in contact with Google trying to narrow it down. From Amazon we know there are other clients of theirs that have been affected as well.”
What is Amazon Global Accelerator?
Amazon Global Accelerator is a service that helps solve network congestion on the Internet to help speed up websites.
This is how Amazon describes the Global Accelerator:
“AWS Global Accelerator is a networking service that improves the performance of your users’ traffic by up to 60% using Amazon Web Services’ global network infrastructure. When the internet is congested, AWS Global Accelerator optimizes the path to your application to keep packet loss, jitter, and latency consistently low.”
SiteGround Solves Issue
SiteGround tweeted on Friday November 12th that they had identified the issue and fixed it.
“Status Update: We are glad to inform you that we have implemented a fix for the Google bot crawling issue experienced by some sites. Websites are already being crawled successfully. Please allow a few hours for the DNS changes to take effect. Thank you for your patience!”
Nearly a week after the hosting outage began SiteGround publicly announced on Monday November 15th what the cause of the problem was.
“Status Update: On Friday we managed to isolate the Google bot crawling issue to a networking problem that was specific only to Amazon’s Global Accelerator and Google’s crawler bot subnet. We implemented a fix that bypasses that problem.”
SiteGround followed up with a tweet to express their happiness:
“We’re happy to say that the majority of our clients’ sites are being crawled now and most of them have already returned their rankings.”
That followed another tweet that reported the success of the fix:
“Аll websites hosted on our end were fully operational, and there were no DNS resolution issues with the requests submitted from any other service.”
SiteGround Customers Still Upset
SiteGround implemented a fix. But many customers remained upset over the weekend as their sites still appeared to be affected.
That may not have been a problem at SiteGround but rather caused by delays in the DNS system as that information propagated across Internet, which could take a few days.
Positive Reports from SiteGround Clients
Some customers reported that their sites have recovered:
So far every thing back to normal. In fact, I might have benefited indirectly from the spam update so actually better than normal
— Norman Schwarze (@pelerinages) November 15, 2021
I use the SG nameservers again. It works. Also I saw one site back in the top of google again. Phew. fingers crossed.
— Reinout te Brake (@reinouttebrake) November 14, 2021
Many Negative Tweets About SiteGround
Some publishers continued to tweet over the weekend about their ongoing problems, which might have been related to the DNS information or Google’s having to re-crawl the sites.
Nevertheless, customers were still tweeting about the slow pace of website traffic recovery.
I’ve still got indexing issues across a number of the sites that I manage.
— Jake Boyle Consulting (@JBCAUS) November 14, 2021
@SiteGround Your DNS issue has caused 50% of my sites to be dropped from the Google index and a loss of 60% of daily traffic. Until today, I didn’t receive any email for an explanation on the matters. Your CEO should apologize publicly for the inconvenience caused to the users.
— Ocean W (@OceanW62958303) November 15, 2021
WE’ve been losing money the last few days because of your “issues” you say “thanks for your patience” and are you gonna give us the money back or what?! this is awful and frustrating!!!
— Mapaula (@mapaulaguti) November 15, 2021
Still have several posts not on Google. Have spent hours going through, checking, requesting re-index. Any idea how much longer?
— AbFabTravels (@Abfabtravels) November 15, 2021
@SiteGround , I still have the same problem. The error is still there. https://t.co/62Wf7ryfmD
— Toon (@toondebacker) November 13, 2021
Are you sure the issue has been fixed? Still experiencing a drop of visits…
— Pietro L. Carotenuto (@pietrolcdotcom) November 14, 2021
I have just tried to index one of the websites I manage and it has been rejected. The issue is far from fixed. Updates should be published on our hosting dashboards!
— Black White and Grey (@bwglawuk) November 13, 2021
SiteGround responded to those tweets on Monday November 15th by reiterating that the problem should already be resolved:
“We’ve deployed a fix and the Google bot can crawl the sites we host.
The issue should be resolved across our platform by now, many clients have confirmed.
Please DM us with some URL examples and additional info so we can help further.”
Why do SiteGround Customers Continue to Suffer?
Although SiteGround announced that the problem has been fixed some customers continue to suffer loss of traffic.
That is not unexpected and perhaps SiteGround could have helped customers by making sure they understood what to expect next in terms of Google’s having to re-index the websites.
Basically, when a site goes missing for an extended period of time Google will begin removing the missing site from its index. That’s what SiteGround customers experienced over the weekend.
However Google never really goes away. Google’s crawler, Googlebot, continues to return to the missing website to check if it returned.
Once the site returns after an extended absence it can take a few days to up to as much as ten days to fully recover, depending on how many web pages need to be re-crawled.
My own experience a couple years ago with hundreds of thousands of web pages that temporarily disappeared was that it took about ten days to recover after a two week outage.
But for most publishers with smaller sites the recovery may be significantly faster.
Google Offers Insights Into Site Recovery
Google’s John Mueller tweeted and retweeted some helpful information about what SiteGround customers should expect in the coming days in terms of Google re-indexing their websites.
Google’s John Mueller tweeted insights about this process in October:
“If you’re curious about what happens in Google Search with an outage like Facebook recently had, it’s generally a 2-part reaction: when we can’t reach a site for network / DNS reasons, we see it like a 5xx HTTP server error. This means we reduce crawling:”
Then:
“The URLs remain indexed as they are, the site continues to rank as it used to. This is a temporary state though. If we becomes a persistent error (if it lasts more than 1-2 days), we will start dropping those URLs from indexing.”
Mueller tweeted that an outage will not cause Google to change the rankings of the site after it comes back.
“There’s no direct ranking change when this happens – we don’t see the site as being low-quality or similar, but if those URLs aren’t indexed, they can’t rank either.”
Once the outage is fixed, Google will re-crawl the websites. This is important to note, that the site cannot resume ranking where it used to rank until Google has finished re-crawling the website.
“When the site comes back, if we have dropped URLs from indexing, those will generally pop back in once we can successfully crawl them again. Crawling will also speed up again if we can tell that the server’s fine.”
John Mueller repeated this reassurance on November 12, 2021 in a series of tweets:
“Once it’s resolved, Googlebot crawling & indexing picks up automatically again. The crawl rate goes up over time as the errors disappear, the dropped URLs will get recrawled over time and make it back into the index. The visibility will stabilize again.”
He reaffirmed that there are no lasting effects from an outage.
John tweeted:
“There are generally no lasting effects from temporary outages like these. Technical issues come & go, we have to do our best to make sure users can find their way to your wonderful sites through search results.”
Mueller tweeted this tip for fast re-indexing:
“If you have important pages that you need reprocessed quickly, I’d use “Inspect URL” in Search Console to resubmit them. Within a website, using internal linking to highlight & link to what you really care about is also good.”
Most replies to Mueller’s tweets were positive but not enough for some to blunt their lingering feelings.
That’s fine that there may not be lasting effects but what about immediate effects? New content cannot be found and we are 2 weeks from Black Friday one of the biggest times for e-commerce, so that’s a lasting effect on revenue of new content can’t be crawled…
— Chad Wyatt (@ChadWyatt14) November 13, 2021
Websites Recover From Temporary Outages
What happened last week was literally an unprecedented event. SiteGround is widely viewed in the industry as a reliable web host, which is why it is so popular.
Websites should be back to ranking where they formerly did within days as Google continues to crawl and re-index the sites that temporarily dropped out.
