Touring an IS&T Datacenter, and How My Dad Crashed the Internet by Danny B.D. '15
I promise they're related.
A non-negligible number of people reading this were probably pretty panicked at the submission of their Early Action applications during the last few days. That’s over now, so breathe and allow me to distract you with mindless procrastination something cool from MIT.
Also, I’ll be telling the story of how my father became one of the few people in the world who has managed to crash the Internet. I don’t want to lead with that story though, so I’ll save it for later :D
First, let’s talk about computing. MIT’s flagship computer club is SIPB, or the Student Information Processing Board. Their origins date back 45 years, but they still serve the MIT community with open-source projects, general computer help, and resources galore. Last Wednesday, I attended the SIPB Computer Tours and Story Time. It was, to say the least, extremely cool.
Our evening began with a walk westward, wandering unwaveringly towards Westgate’s weirdly numbered neck of the woods. (As an aside, MIT’s building-numbering system is confusing enough already, but seriously: What the heck happened on far West Campus? It’s ridiculous.) We reached W91 and W92, which house data centers and processing power, for our tour inside. Mind you, on the outside W91 has an awfully ominous edifice:
But Back In The Day™, W91 served as a wind tunnel for the Institvte, which explains its high ceilings and open spaces within its floor plans. W91 later saw the installation and later removal of the Console Space, fondly known as “the Bridge”:
It’s seen some upgrades since then, and is now a really pretty building of servers and cables galore. Here’s what we found:
See those clear walls between us and the servers in this photo? They’re there for cooling: cold air comes up through floor vents inside of those cordoned-off areas, and chills the racks. Hot air exits the other side, and rises up into the AC intake. By separating the data center into “cold aisles” and “hot aisles”, it takes a lot less AC to cool the servers, and the whole place becomes a lot more energy efficient.
I did theater tech in high school, and immediately assumed that these were power cables. Carefully, I got a closer look:
Nope, not power cables. It’s hard to read, but each of those cables holds 576 single-mode fibers. A single-mode fiber can beam something like 10 gigabits per second on its own, so yeah, you could say that we at MIT have…connections.
We also walked by this massive…thing:
This is a giant box which holds…tape. You know, tape?
Turns out that tapes have a very low failure rate and a very high storage density, so they’re really useful for backups. That giant grey box contains five petabytes (5000 terabytes, or one-twentieth of Data’s storage capacity; thanks WolframAlpha).
To access all of those tapes, there are these robot arms which run along tracks, pull out tapes, and run them back to the tape reader: they let MIT access anything in those five petabytes in about thirty seconds. Plus, they’re mesmerizing to watch:
So that was cool.
After touring W91 and W92, we headed back east toward the rest of campus and made our way to Story Time with Jeff Schiller. Jeff’s been at MIT for a long time, and been network manager for MIT’s campus since 1984. The only reason why he wasn’t network manager before 1984 is because the network started in 1984, under his guidance. In the decades hence, he’s built up a huge collection of stories about MIT policies, IT debacles, and a bunch of other stuff. If you’re in the area for this event next year, I highly recommend it.
Alright, you’ve stuck with this so far, so I should probably give you my dad’s story now. It connects back, I promise.
How My Dad Crashed the Internet
In February of 1995, the Internet was a lot smaller, and in general a more trusting place. At the time, my father was working as a sales engineer for Ascend Communications, which built communications equipment in the 80s and 90s. Their latest and greatest product was an access switch known as the Ascend MAX TNT—a name which, by the way, is not something you should stamp all over a package to be sent through an airport. (This was tested empirically. It didn’t make it to Interop Atlanta.) Ascend got the contract to install one of these things at an Internet Service Provider, FastNet, in Bethlehem, PA. My dad drove down, and went to go plug the switch into the network.
Since Ascend did both sales and manufacturing, they needed a lab to test and troubleshoot their products. In the case of an access switch, it’s important to attach it to a network in a controlled environment and make sure it can talk to the other devices on that network. This normally would involve hooking the switch into the network, and then re-configuring all of the devices one at a time to use the modem as their gateway through the network. It’s pretty slow though, so the switch’s default settings were configured with a kind of test mode: when attached to the test network, it would send messages to everything on that network and tell them to use it as their gateway. The devices would listen, and testing took a lot less setup time. It was certainly helpful.
You’re probably starting to see where this is going. The box my dad brought to install had been accidentally left in its default configuration.
*click*
In my dad’s words, “The box turned on, then went all winky-blinky, then went solid lights, then their customer service reps came running in and asked if we had done anything. We yanked the plug and said no.”
True to its configuration, the MAX TNT sent out a broadcast to all devices which could hear it, telling them to use it as their gateway to the network. Except this time, “the network” wasn’t a test network back at Ascend. It was the Internet as a whole.
Remember how I said that the Internet was a more trusting place back then? All of these devices within reach of the MAX TNT my dad just plugged in received its message, and decided to comply by rerouting all of their Internet traffic through this little box. The switch couldn’t handle it by a mile. Down went the east coast’s Internet. The west coast fared a bit better, but things hit a snail’s pace worldwide for a few minutes. After my dad ripped the box out of the wall, devices across the Internet, unable to reach their new trusted gateway, reverted back to their old gateways, and access resumed.
I told this story to Jeff Schiller after his story-telling session. His response? “I remember this.” Turns out, he used it as an example to advocate for network security upgrades for MIT: if this could happen accidentally, why couldn’t it happen maliciously? Security’s a much larger part of the modern web, so this wouldn’t really work anymore, but it was still awesome learning that my dad’s adventures had had some effect on MIT.