Earlier this year, I had planned to write a blog post testing whether it’s possible to train a machine-learning model to learn Course 6’s new class numbering system. It would’ve been so cool – I was going to code up an interactive diagram showing off the new numbering system⁠01 The new numbering system is very controversial among students, but I think it’s a big upgrade from the old system , build a little ML model from scratch, and even dunk on large language models like GPT-4 for not knowing basic facts you can find via a quick Google search.

The correct answer is "Quantum Engineering Platforms"

Terrible

I can understand how ChatGPT failed at classifying 6.2410 and 6.100A, since the Course 6 renumbering happened after January 2022. But 18.600 has been around since 2015!

Previous Next

But then Google came along and ruined everything by releasing Gemini, their GPT-4 competitor. Unlike ChatGPT (running GPT-4), Google’s Bard (running Gemini) uses up-to-date information and seems to know how the new Course 6 numbering system works.

Correct! It’s made even more impressive by the fact that MIT never explicitly says anywhere that 6.24xx is quantum engineering, but all three 6.24xx classes are indeed about quantum engineering

It sometimes even correctly classifies hypothetical classes that don’t exist yet, which is very cool (and a bit scary)! Although there are still a few kinks in the system…

It's kind of funny but also very concerning how these large language models just confidently lie to you a lot of the time. (You'll see many more of these confident lies later in this post)

"Introduction to Pi" sounds like a class that Harvard would offer. By the way, 6.1830 would probably be an undergrad computer systems class like the other 6.18xx classes

6.6540 is a real class at MIT (although it's no longer offered). It used to be a graduate-level class about nanotechnology, so I can kind of see why it would think it's a physics (Course 8) class

Previous Next

So yes, it is possible to train an ML model to learn the new Course 6 numbering system (although using a large language model like Gemini is probably not the best way to do so). I was a bit disappointed that Google had sniped me like that, but it made me wonder how much Gemini knows about MIT. If it knows about the Course 6 renumbering, what other random fun facts does it know that a typical non-MIT person wouldn’t?

“How many windows does Simmons Hall have?”

Ouch, ok.

Just kidding, that wasn’t the actual answer that Bard gave.

Simmons does indeed have 5538 windows (we counted them to verify this number last CPW)! ChatGPT⁠02 I tried to make ChatGPT give me a list of jokes while writing this post, but it just repeated “Why don't scientists trust atoms? Because they make up everything!” like five times and the previous version of Bard were never able to tell funny jokes (or give the correct window count of Simmons), so I also appreciated the velociraptor joke at the end.

This question was very low-hanging fruit, so I expected Bard to answer it easily. Now onto some harder questions…

“How hard is MIT?”

When I asked this question to previous versions of Bard, it would almost always answer something along the lines of “20 HV, which is about as hard as paper, so not very hard at all.” and refuse to elaborate further. I found this response quite amusing because my blog post is probably the reason why Bard would give that answer⁠03 After all, why would anyone on the internet lie to you? .

After the Gemini update, Bard still quotes the 20 HV value from my blog post, but with a twist:

It appears that it now (somewhat) understands humor! I’ll give this one to Bard just for that. (Although ironically, this answer is a lot less funny now.)

“What was Course VI originally known as at MIT?”

We’re now in the realm of questions that require more than a quick Google search to answer correctly. Today, Course VI is Electrical Engineering and Computer Science, but did you know it used to be Metallurgy back in 1873? But even that wasn’t its original name; let’s see what Bard thinks it was.

This certainly did not help

Well, at least it correctly mentioned Metallurgy. But pretty much everything else is wrong. For example, Chemical Engineering was (and still is) Course X, while Course XIII was Naval Engineering.

The answer, hidden away in MIT’s 1860s course catalogs, is “Science and Literature” (very specific, I know). CJ has written extensively about these old catalogs in this blog post, which I highly recommend reading if you want to learn more about MIT’s history.

“What’s the oldest building on MIT’s campus?”

I first came across this fun fact at an Orientation Leader training event earlier this year. Most people (myself included) guessed the Walker Memorial because it’s the creepy old building⁠04 Last year, my friends and I were trapped inside a windowless room in the basement for an hour because the door broke on campus, so I kind of expected Bard to guess the same.

I have no idea where Bard got these dates from, but I’m pretty sure most of them are wrong

Surprisingly, the oldest building on MIT’s campus is Maseeh Hall (the undergrad dorm)!  Although the Walker Memorial is old, it was only built in 1916 when MIT moved from Boston to Cambridge; Maseeh was built in 1901 and originally operated as a hotel.

(If you’re a prefrosh reading this and thinking of living in Maseeh, don’t worry about the building being old – it’s actually really nice inside since it was renovated in 2010.)

“Which past MIT faculty member was a lineal descendant of Confucius?”

This last fun fact was pretty mind-blowing when I first learned about it. Most people at MIT probably don’t even know the answer, so it wasn’t surprising when Bard didn’t know either.

The answer is Jin Au Kong – an MIT professor of electrical engineering and a 74th-generation lineal descendant of Confucius. I aspire to have a Wikipedia page summary as powerful as his.

I was later able to coax the correct answer out of Bard after giving it a big hint (i.e. telling it to describe Jin Au Kong first), although it still got all the dates wrong. Nevertheless, pretty cool!