A Historical past of Google Cloud and Data Center Outages

Regardless of Google’s fame for reliability, outages are nothing new. Whether or not brought on by software program updates, networking points, or – much less often – knowledge middle fires, outages throughout Google Companies may cause severe disruption for customers.

Right here’s a timeline of serious Google outages, analyzing the incidents’ causes, affect, and Google’s response:

April 2023: Google’s Wet Day in Paris

In April 2023, Google Cloud confronted a difficult day of floods, a knowledge middle hearth, and different Google Cloud Networking issues, inflicting service disruptions throughout a number of service areas. Learn extra

August 2022: Google Data Center Hearth

On August 8, 2022, an electrical incident induced a fireplace at Google’s knowledge middle campus in Council Bluffs, Iowa, injuring three workers. The fireplace occurred on the identical day as outages of the corporate’s maps and search service, though Google stated the 2 incidents had been unrelated. Learn extra

1a-Main-Image-Google.jpg

July 2022: Google Data Centers Knocked Offline by London Warmth

In July 2022, temperatures in London reached 40 levels Celsius. Google and Oracle skilled points with their cooling programs within the warmth, inflicting outages and knocking some web sites offline. Learn extra

December 2020: Google Companies Together with Gmail, YouTube Undergo Main Outage

Google’s outage in December 2020 affected Gmail, YouTube, Pokémon GO, Google Dwelling, and different merchandise globally. Whereas outages are usually not unusual, this particular outage was notable for its affect throughout the Alphabet portfolio. The vast majority of affected companies had been returned to performance inside an hour. Learn extra

Associated:Data Center Catastrophe Restoration: Important Measures for Enterprise Continuity

July 2018: Google Cloud Disruption Brings Snapchat, Spotify Down

In July 2018, fashionable apps, together with Snapchat and Spotify, briefly went down after a failure of Google’s cloud-computing companies. Whereas it was initially unclear what had induced the outage, there have been experiences of an incident on the Google Cloud Standing Dashboard. Learn extra

November 2017: Data Center Failover Error Kicks Google Cloud Companies Offline

Memcache – a part of Google App Engine – went down in November 2017. With Memcache being unavailable, requests went to the Datastore service, making a surge of exercise resulting in errors and latency points. Learn extra

August 2016: Google Explains What Went Mistaken to Trigger PaaS Outage

On August 11, 2016, a two-hour outage on Google App Engine affected 37% of purposes hosted in its US Central area. Google blamed the incident on a site visitors router software program replace triggering a rolling restart throughout normal periodic upkeep, which concerned engineers shifting purposes between knowledge facilities. The decreased router capability finally led to overloading, and Google’s handbook site visitors redirection was not sufficient to resolve the issue till a configuration error inflicting a site visitors imbalance was recognized and stuck. Learn extra

Associated:A Historical past of AWS Cloud and Data Center Outages

Google-Cloud.jpg

April 2016: Google Reimburses Cloud Purchasers After Google Compute Engine Outage

An 18-minute outage in April 2016 affected Google Cloud Engine customers throughout a number of areas. After the incident, Google claimed its engineering groups could be engaged on “a broad array of prevention, detection and mitigation programs meant so as to add further protection,” and it reimbursed customers as much as 25% of their month-to-month costs. Learn extra

August 2015: Lightning in Belgium Disrupts Google Cloud Companies

In August 2015, a collection of lightning strikes in Belgium knocked some cloud storage programs offline. Experiences had initially acknowledged that lightning had struck electrical programs at one in all its knowledge facilities within the small city of St Ghislain, however a spokesperson later confirmed a neighborhood utility grid had been hit. Learn extra

March 2015: Google Traces Cloud Outage to Defective Patch

In March 2015, for the second time in a month, Google Compute Engine suffered an outage, with some customers experiencing disruptions for as much as 45 minutes. It was a partial outage, that means some customers weren’t impacted, some noticed a slowdown, whereas others skilled points contacting their cloud VMs. Google recognized a patch downside as having induced the problems. The configuration change was examined earlier than deployment however affected some VMs when dwell. Learn extra

Associated:Incident Response: Classes Discovered from a Data Center Hearth

February 2015: Google Compute Engine, AOL Undergo Early Morning Outages

February 19, 2015, noticed two outages on the identical day: Google Compute Engine was down throughout a number of areas for round an hour, and AOL skilled an prolonged outage that lasted a lot of the morning. Google blamed the Google Compute Engine incident on community points, which induced connectivity loss throughout many zones. AOL’s e mail service was resolved after a morning of points, with AOL gradual to reveal the reason for the issue – some claimed there had been a community situation. Learn extra

October 2014: A number of Google Cloud Companies Expertise Downtime

Google Cloud Companies customers skilled points with Gmail, Google Hangouts, Google Analytics, and Google e mail safety service Postini in October 2014. Whereas the incident affected the vast majority of customers, it was resolved comparatively shortly. Learn extra

January 2014:  Gmail Internet App Outage

On January 24, 2014, the broadly used app Gmail went down because of an inner bug producing an “incorrect configuration.” Learn extra

December 2012: Load Balancer Misbehavior Cited in Google Outage

In December 2012, an incident report confirmed the reason for a latest Gmail outage to be a software program replace that induced a networking situation, particularly in Google’s load balancers. Google defined {that a} “bug within the software program replace induced it to incorrectly interpret a portion of Google knowledge facilities as being unavailable.” The outage had induced points for customers accessing Gmail, and plenty of Chrome customers additionally skilled browser crashes. Learn extra

February 2010: When the Energy Goes Out at Google

After an influence outage in February 2010, Google shared a collection of steps it could be taking to handle the incident. Google dedicated to further scheduled drills for on-call employees, common audits of operations paperwork, a transparent coverage framework for emergencies, and a serious infrastructure change in App Engine. The outage had induced greater than two hours of downtime for Google App Engine. Learn extra

September 2009: Router Ripples Cited in Gmail Outage

On September 1, 2009, a Gmail outage meant customers had been unable to entry Gmail by way of the online interface. Google acknowledged that the trigger was its underestimating the load that routine upkeep on some Gmail servers would place on supporting routers. Google fastened the issue by bringing further routers on-line and stated it then elevated Gmail’s router capability and is taking additional steps to keep away from a repeat of the incident. Learn extra

July 2009: Google App Engine Hit By Outage

On July 2, 2009, Google App Engine skilled excessive latency and error charges, inflicting hours of efficiency points – all purposes accessing the Datastore had been affected. Learn extra

Could 2009: Rolling Outage for Google

On Could 14, 2009, an error in one in all Google’s programs induced site visitors to be directed by Asia, making a site visitors jam. The incident affected about 14% of customers, with points reported on Google Information, Gmail, and Google Calendar, amongst different companies. Learn extra

February 2009: Gmail Outage Centered on European Community

A Gmail outage on February 24, 2009, was brought on by disruptions in its European knowledge facilities. Sudden points with a software program replace resulted in over two hours of downtime for Gmail customers. Learn extra

August 2008: Gmail Service Outage

August 11, 2008, additionally noticed a Gmail service outage: many Gmail customers had been unable to entry their e mail because of a problem within the contacts system utilized by Google that prevented Gmail from loading correctly. Learn extra

June 2008: Google App Engine Outage

On June 17, 2008, Google App Engine, the utility computing platform for builders, skilled a number of prolonged outages throughout which a major proportion of requests resulted in errors. The errors had been associated to the Datastore. Learn extra