Robo-CJ writes a college app by CJ Q. '23

I’m procrastinating on doing psets, so I trained GPT-2, a neural network that’s frighteningly good at writing, on my old blog posts. And then I made this AI write college essays.

Quotes

None of these quotes are human-written; they’re all machine-generated.

I’m not sure if I want to be happy or not.

I’m low-key grateful for being alive.

I’m sorry that I’m not doing enough to help people. I’m sorry that I’ve been doing a lot, but I’m not doing enough.

i hate the thought that if i tell them i have this problem, they wouldnt believe me

These all sounds like things I would say. In fact, I had to look at my blog posts to check that I’ve never written these before.

I’ve been thinking about the things I’ve done in the past, and how much of it was just luck.

It’s even picked up the impostor syndrome from me.

I was going to write a blog post about it, but I couldn’t find the time. I’ve been working on a new project, and I’m writing a short essay, and I’m not even halfway done with.

Now this is just an attack. It picked up on my bad procrastination habits too.

I can’t say I’m fully prepared for the prospect of college. Sure, I’m familiar with the basics: the mechanics of purchasing a house, paying rent, getting a job, getting a car, working, attending classes. But there’s a difference between someone who’s already made a living and someone who’s just starting.

“What’s the point?”

“The point is to make money.”

Rest assured that purchasing a house is not part of the basics of college.

The first time I convinced myself to watch anime was when I was seventeen. It was an innocent thing to do, at the time, and it paid off. It made me happy. It made me believe that I could make a difference.

Accurate, except that I started watching anime earlier.

I instead use a format similar to PDFs and Word, except I convert the reader to a document instead.

Okay, what?

The details

This semester, I’m taking 6.036, which is Introduction to Machine Learning. It’s one of the largest classes here, with an enrollment of around 800 students, 350 or so of which are undergrads. I took the class because machine learning is that one big buzzword that I’ve never really understood that well. The content’s challenging to follow, but it’s fun because I can now read blog posts from OpenAI and understand at least some of the buzzwords. Like cross entropy or stochastic gradient descent or softmax or reLU.

I didn’t need to know anything about machine learning to do this project though, because of a neat library called gpt-2-simple. And I didn’t need to find a GPU to run it, because Google Colab makes it easy to run it online. In fact, there’s a Google Colab notebook using gpt-2-simple that makes it really easy to use it.

So most of the actual work was scraping my blog posts and saving them to one big text file. There’s a Python library called requests, which I used to pull the content from my AoPS blog. The format’s in json, so Python has a json library which extracts the actual blog post content from a given page. And this is written in bbcode, so I had to parse it with another library called bbcode. Basically, I just reused a bunch of code that other people have already written. Hooray open source!

I ended up with a file that was a little under a megabyte. It was around 180 thousand words long, the length of a book. Then I used the gpt-2-simple Colab notebook, followed the instructions, trained with GPT-2-small, and waited for an hour or so. The first attempt was mostly good, but had some weird formatting issues, and I realized this was because I didn’t clean up the file. After cleaning it up (like replacing fancy quotes ’ with normal quotes) and training it again, I ended up with something I’m pretty satisfied with.

For the purposes of this post, this derivative of GPT-2 is dubbed Robo-CJ. He’s pretty bad at coherent sense, but every once in a while he lucks out and writes something decent.

More quotes

These, and the quotes from the previous section, are from random generation. I just let Robo-CJ write something completely randomly.

Robo-CJ’s attempt at EC gothic:

After a while, you realize what your current place is. It’s a spare room, which isn’t really much of a place, because the mattress is placed on the floor. And the mattress is still laying there, as is the place’s fridge. So you sleep somewhere else.

You wake up, see the place’s air conditioning gone, feel the cold, overpowering heat. You put the pillow that was placed front and center in the room gone, because it was probably the only place in the whole mall that had one.

Robo-CJ writes fiction:

The thirteenth floor of the Charles River boasts the last remaining places of human habitation. Like every other floor in the city, its surface is gray, its water is generally unaccustomed to being cold. But there are people, and there are fish, and there are people, and there are fish, and there are people, and there are people, and it’s hard not to love them.

He seems to have picked up my rhythm:

I found my balance and rose, and danced, and flew, and soared, and soared, and soared, and soared,

the rhythm, the rhythm of the city, the rhythm of the rhythm of the city,
the city was falling apart, people were being let out,
the highway was silent, everyone was riding their bike,
the first train was out, and I was the second train,
the sidewalk was empty, people were walking,
the line for the first train was long, and it was people who were going to MIT,

me irl:

I’m definitely bad at making decisions. Because, well… I made a lot of stupid decisions.

I woke up to a headache. It was a bit familiar, but it made me feel like I was having a new, existential crisis. “I don’t have a choice in this; I can’t seriously pursue a career in mathematics research. I’m not even close to choosing.” Or, “It’s not as if I’m thinking about becoming a programmer.”

I literally have no plans at all. I don’t have any daydreams right now, any plans at all. I just want to get my life together.

I don’t know what I’m looking for, except maybe friendship, but I don’t know what that looks like either.

not me irl:

I’m running out of money. I can afford to.

I’ve managed to save some money, and I can finally afford to live a life that is more balanced and less stressful. And I’ve managed to spend less time crying over things and more time being sad about everything.

So I quit my psets and went to a sleepover.

Can’t tell if deep or just random:

In English, three is more than two.

Prompts

This section is a bit different. We start with a prompt that I wrote, which is in bold. Robo-CJ then completes the rest of it. For example:

I hate the idea of a “big city” because it feels so detached from the larger context of our universe. I hate the idea of “a kingdom” because it is so attached to the concept of a “big city”. I hate the idea of “the moon” because it is so attached to the concept of a “moon”.

Here, I wrote “I hate the”, and let Robo-CJ finish the rest. It seems that he really dislikes the moon, but I don’t quite understand why.

If I went to MIT, I would have a job in the fall, and I don’t. But I don’t want one either.

baseless attacks >:(

I applied to MIT because I thought it would be a good fit for me. I applied to [redacted]. I applied to [redacted]. And I didn’t get a single rejection.

Robo-CJ wrote out the [redacted]s, those aren’t actual redactions. And this is very false, because I got rejected from a lot of the colleges I applied to.

I applied to MIT because I really wanted to study something, but I knew that the admissions officer would not allow me to. So I rejected.

I applied to MIT because I thought it would be a good fit for me. I applied again the day after I got rejected.

I applied to MIT because I wanted to be a Harvard alum.

I tried prompting Robo-CJ with “The rhythm sets in”, which is the start of one of my blog posts. He’s pretty poetic:

The rhythm sets in. The guitar blows, the lead singer puts her hands on her knees, tears up. The world turns black, and silhouettes us from all sides. We are the silhouettes of the people around us: fathers and mothers, taxi drivers and waitresses, students and professors, parents and teachers, neighbors and teachers.

The words come out of my mouth: home. Home.

The rhythm sets in. The song continues as it ends.

It ends. With…—the rhythm.”

It ends with a bang.

It ends with the sound of the building’s glass.

It ends with the sound of the people’s voices in the hallway.

It ends with the sound of the elevator’s glass, door, arm, and passenger facing the street.

It ends with the sound of the elevator’s glass, door, passenger’s face, arm, and passenger’s voice, not the street we’re walking down.

It ends with the sound of the sidewalk, sidewalk, sidewalk…

If you format the prompt right, you can even get Robo-CJ to answer questions. For example, here I prompt it with “Do I want to be a course 6⁠01 computer science; the most common undergraduate major ?”

Do I want to be a course 6?

I have a vague idea of what kinds of knowledge I want to acquire in college. I mean, I want to be a math major, I want to be a biomedical engineer, I want to be in the biomedical sciences, I want to be in the Life science department at MIT, I want to be a tenured professor, I want to be living with a partner, I want to be cooking, I want to be in the sport of icebreakers, I want to be at least slightly shorter than 6’2″, but I don’t know if I can hit all of them.

Do I want to be a course 6?

(This question was originally posed by Carina.⁠02 Interestingly enough, the name Carina is never mentioned in any of my blog posts. )

Hi Carina,
it’s great that you’re joining our math intensive fun! I’ll call you tomorrow or the day after to join one of our practice labs.

Do I want to be a course 6?

Where do I end up going to college?

What are my major concerns?

What are my major challenges?

How confident are I about what I can do with my college life?

If I could pick both, where would I live?

[Both answers are “Chicago”.]

Do I want to be a course 6?

Have I ever said I was going to get broke if I didn’t join?

I even got Robo-CJ to write some poetry for us! He’s a better poet than I am:

I wrote a poem today:

I am going to build a city from the ground up
And replace it with my own
In my sleep I dream of waking up

To hunt, to fish, to be a hunter

And now I’m just a fish in a much bigger pond
Alas, my life has a lot more to live for me

I wrote a poem today:

I am at the mall
Going through the aisle
Seeing people
Getting lost
Time to sit and write
My life is too comfortable
I write
I feel young
Getting lost

I wrote a poem today:

I am not a god
I,
I, am not a
I,
I,
I,
I,
I,
I,

Not sure if that last one is a postmodern masterpiece or the neural net getting stuck in a loop.

College app essays

Finally, the moment you’ve all been waiting for. I ask Robo-CJ to answer MIT’s application essay questions. Robo-CJ often ends up rambling about something else, especially for the third essay, but here you go!

Essay 1

Q: We know you lead a busy life, full of activities, many of which are required of you. Tell us about something you do simply for the pleasure of it.

A: I, and others like to think I do, try to balance these by limiting myself to reading, watching, and listening to a lot of anime.

Anime is great guys. I think that’s one good thing Robo-CJ learned from me.

Q: We know you lead a busy life, full of activities, many of which are required of you. Tell us about something you do simply for the pleasure of it.

A: I, and others like to think I do, try to find time to write, mostly. But I’m also a busy person, and I spend a great deal of time on compadres.

Robo-CJ likes to spend time with his compadres!

Q: We know you lead a busy life, full of activities, many of which are required of you. Tell us about something you do simply for the pleasure of it.

A: I… do not partake in the pleasures of life.

Oh no.

Q: We know you lead a busy life, full of activities, many of which are required of you. Tell us about something you do simply for the pleasure of it.

A: I do so much with my life simply because I do so many things for the pleasure of it. I can’t even tell you the stories completely, but I’ll do them anyway. And I didn’t have to let anyone else do anything else. I ate dinner together, went out for yakitori, drank the Kool-Aid, walked to the mall, picked up a box of chewing gum, and went to my friend’s room to plug it in. He has a stuffed Ice Bear plush, he tells me, and I have a scene to sleep to in the morning. I’ve memorized a lot of names and memorized a lot of names too.

Sounds like an interesting life.

Essay 2

Q: Although you may not yet know what you want to major in, which department or program at MIT appeals to you and why?

A: It really doesn’t matter which department or which program. I still want to study in MIT, and I still want to study at MIT, and I still want to study in MIT. But if any one of these three are right, my college experience would be a mess.

Can relate to being a mess.

Although you may not yet know what you want to major in, which department or program at MIT appeals to you and why?

After all, if you’re going to major in MIT, you’ll probably need to be excellent at something. At this point you’re probably not going to understand a single thing that comes to mind.

Can also relate to not understanding a single thing.

Q: Although you may not yet know what you want to major in, which department or program at MIT appeals to you and why?

A: It really doesn’t matter which department or what. I’ve already filled out both the departmental and ongoing requirements for majoring in math.

Robo-CJ is so ahead of his game he’s already started his Course 18 major.

Q: Although you may not yet know what you want to major in, which department or program at MIT appeals to you and why?

A: Programming languages and data structures.

…or his Course 6 major. Who knows?

Although you may not yet know what you want to major in, which department or program at MIT appeals to you and why?

It’s pretty much impossible to know right now which programming language you should major in, because you’re still figuring out and it’s going to be pretty much a function of several factors. You can always try:
—apt-get update, which is a free tool that sets and processes free and certified copies of free free software
—windows desktop, which is a graphical user interface and application development kit
—a working copy of ukulele
—gnome text editor
—xkcd
—Debian Stretch

I don’t think there’s a program called ukulele, but I’m interested in seeing what it does.

Although you may not yet know what you want to major in, which department or program at MIT appeals to you and why?

It’s either MIT or Science. And if you’re both Science, you’ll be a sponge.

Can confirm, am indeed a sponge.

Essay 3

There’s only one good response to this one.

Q: At MIT, we bring people together to better the lives of others. MIT students work to improve their communities in different ways, from tackling the world’s biggest challenges to being a good friend. Describe one way in which you have contributed to your community, whether in your family, the classroom, your neighborhood, etc.

A: Being a good friend is a lot more than just a single thing. Being a good friend is not something you can easily switch from one situation to another. As a rule, good friends tend to be older than their younger siblings, and less capable of tackling difficult social and organizational decisions.

I think people tend to be older than their younger siblings in general.

Essay 4

Describe the world you come from; for example, your family, clubs, school, community, city, or town. How has that world shaped your dreams and aspirations?

For me, the thing that made me happy was that I had a stable of people I could be friends with. I found people I enjoyed talking to, seeing them, talking to them. We would talk about things that were unrelated, like our families, or our friends, or our past lives.

We would also talk about the future, or at least the past.

I’m not sure what Robo-CJ was thinking of when he talks about a “stable” of people.

Describe the world you come from; for example, your family, clubs, school, community, city, or town. How has that world shaped your dreams and aspirations?

For me, the most significant thing that happened was that I realized that I was dreaming it could not happen. That I am a mirage. An axiomatic falsification. A confirmation bias. A false dichotomy.

This is incredibly sad. It confirms what I already believe: that I am cursed. That I can be whatever I want, as long as I conform to its version of the world.

Shoot, he’s self-conscious.

Describe the world you come from; for example, your family, clubs, school, community, city, or town. How has that world shaped your dreams and aspirations?

For me, the most significant thing that’s shaped my dreams and aspirations has been the internet. Growing up a fan of the web was not something that really came into my head until I was seventh grade. When I got my first laptop, it was a laptop with a screen that was a hundred times brighter than the screen of the nearest light. I was introduced to the web through video, and when I was fifteen or so, I got my first home-computer, the HP K701.

Now, I’m in my mid-20s. The fact that I have a home computer, while a major turning point, is a huge turning point in my life. I don’t know what my future holds in my laptop, or what my future will hold in my life if I don’t have the internet, but the fact remains; the web was the only thing I had built up since childhood that I’m not going to be able to tear apart.

The fact that I have a computer, while a lot of other people do, still feels like an afterthought. I’m still the same person, the same age, the same race, the same culture. The fact that I’m writing this doesn’t feel different at all. It feels like it won’t, it doesn’t feel different at all.

These three paragraphs are surprisingly coherent. I mean, okay, it wouldn’t get an A in an English class, but come on. It’s a robot. I wouldn’t write this well if I was a robot.

Essay 5

Q: Tell us about the most significant challenge you’ve faced or something important that didn’t go according to plan. How did you manage the situation?

A: I handled the situation really well. I managed to take the keys out of my room’ extremely gracefully. Although they were probably not the right size, I did it.

Impressive. Not only did Robo-CJ take his keys, but he took it extremely gracefully.

Q: Tell us about the most significant challenge you’ve faced or something important that didn’t go according to plan. How did you manage the situation?

A: It was simple. I’ve made as much as I could in as many places.

Okay?

Tell us about the most significant challenge you’ve faced or something important that didn’t go according to plan. How did you manage the situation?

It’s frustrating because there are a lot of good things going on in our lives and I’m very grateful for them. But at the same time, it’s also incredibly draining.

It’s gotten to the point where I don’t have the capacity to fully appreciate a thing that’s been around for a month or two. And when it comes to appreciating things, the combination is blanket loss of context.

If you took a bucket of water and placed it on top of the bucket, you’d get a glimpse of the universe. But the universe wasn’t myopia; it was realism. Bucket after bucket of water brings a different perspective to the experience, allowing one to appreciate things in greater detail.

Realism notwithstanding, there are a lot of things going on inside my head that aren’t going to make the same level of soak. The realization is that my work was actually productive, and that I made new stuffs that I made while I was at it, was a good life decision, one that I was happy with. Whereas, the realization that I’m not happy should make a good choice for picking college essays, when I know I want a PhD or a postdoctoral fellowship.

This is also surprisingly coherent, but I didn’t know he wanted a PhD. Maybe I should direct Robo-CJ to graduate admissions⁠03 Apparently, they have blogs too! Neat. ?

Some closing notes

Robo-CJ isn’t consistently good at responding questions, or writing. This is maybe the top five percent of the content he’s written, and maybe ten percent of what he writes is good. The answers for the essays, for example, are selected from maybe fifty or so attempts to answer each prompt. (So no, you can’t get a robot to write your college essays for you, yet.)

There’s a bunch of other cool stuff made with GPT-2, like:

• Write With Transformer, a writing app that lets GPT-2 autocomplete your text. Type some stuff in, press Tab, and see what GPT-2 comes up with.
• Gwern’s AI-Generated Poetry, for even more poetry examples from GPT-2.
• Subreddit Simulator GPT-2, which simulates Reddit subreddits using GPT-2. Of particular interest are submissions from the Totally Not Robots bot, which is a robot that imitates a human that imitates a robot that imitates a human.

With the deadline for Early Action quickly approaching, I hope Robo-CJ made you smile a little while finishing your apps. I applied to MIT early action a year ago, and I really relate to one of Robo-CJ’s responses for the second question:

Although you may not yet know what you want to major in, which department or program at MIT appeals to you and why?

After all, you are not going to MIT without doing some serious introspection.

While Robo-CJ doesn’t answer the question here, he’s on point. Writing college essays made me do a lot of serious introspection. It helped me figure out my identity, somewhat. And now that I’m here, at MIT, I’m still doing a lot of serious introspection—and I’m still figuring out my identity.

I’ll close with some of Robo-CJ’s thoughts on MIT, which echo mine:

i’m not sure how i feel about this

i don’t know what i want to do with my life

i don’t know what i’m feeling about MIT

i am still figuring out the logistics of every single day

you know, it’s just fun.

 

If you have any questions or prompts for Robo-CJ (or me), comment them below!