Cable Forum - Facebook Outage

Cable Forum (https://www.cableforum.uk/board/index.php)

- General IT Discussion (https://www.cableforum.uk/board/forumdisplay.php?f=19)

- - Facebook Outage (https://www.cableforum.uk/board/showthread.php?t=33710426)

Re: Facebook Outage

Quote:

Originally Posted by Hom3r (Post 36095844)

If you check this link, it appears that many other sites have suffered outages at the same time.

https://downdetector.co.uk/

From the Washington Post

Quote:

It’s also possible the outage was affecting other Internet services, Nash* said. When the services went down, so many users tried to load the sites that it caused a run on traffic on the Internet’s DNS infrastructure.

“The reason these failures are so crazy is because there’s so much interconnectiveness of the Internet we rely on,” Nash said.

* Courtney Nash, senior research analyst at security company Verica.

Re: Facebook Outage

someone is going to be called into a rather serious meeting with HR present in the morning.

Re: Facebook Outage

A meeting with no coffee…

Re: Facebook Outage

Quote:

Originally Posted by idi banashapan (Post 36095852)

Someone is going to be called into a rather serious meeting with HR present in the morning.

Well I know that these updated have an off-line test to check them, but the real world is different.

Re: Facebook Outage

I, for one, welcome our new cybernetic overlords…

https://www.cableforum.uk/board/atta...3&d=1633382245

Re: Facebook Outage

Quote:

Originally Posted by Jaymoss (Post 36095862)

And it is back

Not here it isn’t …

If it’s been a DNS fail it’ll take a while for everyone to reconnect.

Re: Facebook Outage

When I posted that I had got a whatsap message and then check FB and working. All seems to be up here now

Re: Facebook Outage

Quote:

Originally Posted by Chris (Post 36095863)

If it’s been a DNS fail it’ll take a while for everyone to reconnect.

In theory, it shouldnt - its actual DNS changes that take a while to propergate, due to TTL.
But if DNS simply fails due to the source servers being down, then it shoud all start working again as soon as the source servers are up.

Of course, they may get overloaded to start with.

That said, some of the reports suggest BGP changes were the problem.
In that case it can take time for more changes (ie fixes) to find their way to all routers.

Re: Facebook Outage

hmmm . . I'm wondering if I can block facebook through the router (tplink) settings . . her indoors would just think it was still down :D

Re: Facebook Outage

Quote:

Originally Posted by nffc (Post 36095842)

So where is everyone going to post fuel tanker panic updates now?

Also, it really does look to me like someone's hosed the DNS records from inside, whether that's a misconfiguration by accident or if someone's done it to take it offline, will remain to be seen.

There is nothing wrong with teh DNS records. They still exist but systems can't access them due to a config update on the BGP routing.

As Hugh posted,

Quote:

someone at Facebook caused an update to be made to the company’s Border Gateway Protocol (BGP) records. BGP is a mechanism by which Internet service providers of the world share information about which providers are responsible for routing Internet traffic to which specific groups of Internet addresses.

Re: Facebook Outage

https://www.irishtimes.com/business/...ange-1.4691536

[EXTRACT]

Facebook, Instagram and WhatsApp are now back up and running after being shuttered by a global outage on Monday.

The widespread disruption was blamed on a “faulty configuration change”, with Facebook saying in a statement:

“Our engineering teams have learned that configuration changes on the backbone routers that coordinate network traffic between our data centres caused issues that interrupted this communication.

“This disruption to network traffic had a cascading effect on the way our data centres communicate, bringing our services to a halt.”

The company in a late Monday blog post did not specify who executed the configuration change and whether it was planned.

Several Facebook employees who declined to be named had told Reuters earlier that they believed that the outage was caused by an internal mistake in how internet traffic is routed to its systems.

The failures of internal communication tools and other resources that depend on that same network in order to work compounded the error, the employees said. Security experts have said an inadvertent mistake or sabotage by an insider were both plausible.

“We want to make clear at this time we believe the root cause of this outage was a faulty configuration change,” Facebook said in the blog.

Re: Facebook Outage

Quite a few sites were very slow yesterday.

Things seem ok this morning though.

Re: Facebook Outage

Didn’t even notice/miss it. Don’t have WhatsApp, Instagram. Might log in to Facebook once a week.

Re: Facebook Outage

Quote:

Originally Posted by Mick (Post 36095920)

Didn’t even notice/miss it. Don’t have WhatsApp, Instagram. Might log in to Facebook once a week.

I dont use instagram at all, the only time I use whatsapp is when my kids want to send me pictures from their phone. I probably look at facebook twice a week.

The only reason I knew was my daughter was having trouble connecting to whatsapp, so asked me to take a look.
Having determined it was a WA issue and not our network, I went on to google and came across the outage reports.