uzay avatar

2024 in Review by Uzay G. '26

the things and stuff of 2024

This was one of the years of all time. I learned a lot, less about science than usual but more about relationships and who I am. It was a bit painful at times, but there were also many great things.

I caught glimpses of some beautiful ideas, I settled into my life at MIT and now have a circle of people who inspire me and illuminate my life, and I feel more confident to face the world, knowing my strengths and weaknesses a bit better. I also did a lot of collaborative work and really appreciate that kind of thing much more now!

I tried to do research really hard. I grinded, and developed, but I struggled. I am trying to finish the projects I’ve been doing and put them out this month mostly. Quality research is challenging.

In 2024 I developed strength and self reliance. Obviously it’s normal to depend on others, but this year I saw myself confronted with the fallbacks of depending too much. I learned to be happier on my own, and develop a slightly healthier, more understanding mindset towards the way I end up living my life. This year I have had deep, beautiful friendships that I am beyond grateful for. But in some of these lows, I grappled with a sense that things were off, that there was some true connection right around the corner but always out of grasp. I was bound by expectation, waiting to see and be seen by somene. This expectation weighs on others and destroys your lucidity. Instead I have been trying to, mostly successfully after a while, choose to take in the things that bring color to my life, and nurture them from a place of strength. When you are no longer dependent on someone or something in an existential, unhealthy way, you can more clearly see things as you are. You can recognize both the pain and the beauty of the things you are graced with in your friendships, relationships, etc… and then choose what deserves a place in your life.

I think my sense of beauty is growing more and more. I watched some movies that I really liked this year, and I wrote more poetry. Some images really stuck with me deeply, and I want to keep seeking out these kinds of insane artistic experiences.

I have a much better model of what it means to be a researcher thinking about the questions that I’m interested in. Although it may not seem like it from my outputs, I have a better sense of a real agenda I can pursue that satisfies me intellectually/aesthetically and genuinely matters.

I know that I will keep learning, seeking out art, people and new experiences. I know that I will create the conditions for my goals wherever I am, and I know that the world will change faster and faster. I feel okay about all of this, and am excited by everything that lies ahead.

I am in a bit of a corny mood tonight, so I want to leave one message to that person looking back, and it is this: go find the magic. Drink it down without greed, but don’t forget its slow passage. If your mindset goes sour remember there is so much beauty to grasp always. In the famous words of Baudelaire, re-enacted here by Serge Reggiani:

Il faut être toujours ivre, tout est là ; c’est l’unique question. Pour ne pas sentir l’horrible fardeau du temps qui brise vos épaules et vous penche vers la terre, il faut vous enivrer sans trêve.

Mais de quoi? De vin, de poésie, ou de vertu à votre guise, mais enivrez-vous!

Et si quelquefois, sur les marches d’un palais, sur l’herbe verte d’un fossé, vous vous réveillez, l’ivresse déjà diminuée ou disparue, demandez au vent, à la vague, à l’étoile, à l’oiseau, à l’horloge; à tout ce qui fuit, à tout ce qui gémit, à tout ce qui roule, à tout ce qui chante, à tout ce qui parle, demandez quelle heure il est. Et le vent, la vague, l’étoile, l’oiseau, l’horloge, vous répondront, il est l’heure de s’enivrer ; pour ne pas être les esclaves martyrisés du temps, enivrez-vous, enivrez-vous sans cesse de vin, de poésie, de vertu, à votre guise.

How is it that I get to live every day and see all this magic? that I can learn more about the structure of this world? That I get to think about the unbelievable beauty day after day crystallized in the slow motion of particles, in shapes mapping in and out of each other or the practice of poetry, and this web of rich inner lives illuminating each other? that I love these people and their personable, similar and yet alien outlooks bouncing off against mine? that I get to feel understood and seen, that you understand a view on who I am and choose to walk it through day after day? how is that I get to experience these lives singing against mine for as long as we choose to? and that even after that, I will get to collect these sweet postcards on the road that have made me this single person, lost in immensity, writing it all down?

And so, read this as a catalogue of magic; of passion blooming out and in; of dreams being workshopped; I learned a cool phrase just now, kangai muryō – I am overwhelmed by the beauty of things, and I will stay overwhelmed. Will you come along with me?

~~

Goals

2024 goals

  • write a lot (2/10)
  • try theoretical research and decide if it’s a fit (eg theory of DL) (9/10)
    • did a bunch, not sure if it’s so simple as a “fit” or not but I understand my perspective a bit better
  • be the main person on a research project I was excited about (10/10)
    • lots of research this year
  • write a poetry collection (8/10)
    • took a poetry workshop at harvard, working on revising now
  • one substantial coding project (2/10)
  • learn more math/physics (9/10)
  • one climbing V7, consistently get V5s (8/10)
    • consistent V5s, one V6
  • get a good anki setup/figure out uzay x spaced repetition (0/10)
  • get better at technical writing (8/10)
  • get a better perspective on my position and goals relevant to AI (7/10)

2025 goals #

  • wrap up current research (kaivu project, tanishq project, isola project)
  • converge on a key research problem to focus on and make progress on it (focus on is like multi year style hopefully)
    • am prioritizing some ideas right now but don’t want to say too much yet
  • write a lot, in particular about my current model of the world / AI. related to reflecting on what I wanna do.
  • in general with respect to my media consumption, shift the create:consume ratio a bit upwards, reflect more and create things because I want to devote more to each individal idea
  • climbing: consistent v7s? one v8
  • gain in confidence in my actions, trust myself and the future
  • get out of my comfort zone

Research

[… this section is to be improved, probably as a separate post because of how much attention it deserves]

I don’t want to get into the weeds here just yet, but this is probably where most of my learning has gone.

I’ve been working on a few projects:

  • In the Isola lab I’ve been trying to investigate certain claims, for example by Von Oswald et al that inner optimization and gradient descent style dynamics underpin a lot of in context learning in transformers
    • in particular how do they hold up on real data?
    • this work has been super interesting conceptually and I’ve also gotten my hands dirty with a bunch of empirical analysis of real models
    • it’s crispened a bunch of intuitions on how in context happens, but I need to push through a bit to get to the finish line
    • If I do this well I think it might inform how people think about these sorts of in context optimization claims in the real world, along with potential new ways of analyzing how in-context information is used by a model
  • with Kaivu we’ve been investigating using interpretability methods to understand/create a phenomenology of adversarial attacks, for example extending Aleksander Madry’s work
    • we struggled for a while to get solid vision SAEs
    • we then moved on to language, and get solid results on the mechanistic emergence of features from optimized, OOD inputs
      • submitted this to NeurIPS workshops, got in and went together :P!
  • now Kaivu, Atticus and I are thinking a bit about explicit world modeling and human-model collaboration for science
    • more on this coming soon!
  • with Tanishq and others I’ve been working on a project related to strange effects of data diversity and task interference
    • we submitted some work to ICLR which we are now reworking, more coming

I don’t want to post too much about the future but there are a lot of ideas I’m excited about here. I will also post something about the papers that have stood to me so far. Excited to keep hopping along this research journey.

I’m converging a bit on the type of work/setting I will want to work in long term. I am strongly considering applying to graduate school, and I want a lot of independence. I believe at some point I will pivot to some sort of research startup, where a set of friends and I with a few takes and ideas try to make something happen, in ML. Already currently starting to do things along this vein.

Learning

What I wanted to learn

  • software (2/10)
    • concurrent stuff
    • computer architecture
    • OS design
  • info theory (10/10)
  • flexibility (2/10)
  • piano :/ (0/10)
  • quantum mech (10/10)
  • stat mech (7/10)
  • topology (8/10)
  • RL (3/10)

What I learned

Classes

The memorable ones!

  • Quantum II – 8.05
    • I learned quantum mechanics in 2024!
    • It’s really interesting, and the content was very well presented in this class
    • intense though
    • I was inspired by the scientific ingenuity in the development of quantum mechanics. It was also pretty to see the algebra I know show up so neatly in QM
    • I have loved and continue to love the physicist approach; the quick intuitive arguments; the fact you can formally ground arguments but also just see them right away in your mind’s eye by thinking about physical reality
  • Harvard Poetry Workshop
    • harvard has really good creative writing workshops
    • this was a great experience
    • harvard students are much more serious about the humanities, it’s their thing for the people in these classes!
    • I met some cool writers, and developed my own writers
    • might try to put out some of my poetry
  • An Algorithmist’s Toolkit – 18.408
    • covered
      • spectral graph theory
      • convex bodies, high dimensional geometry, concentration of measure
      • optimization
    • Prof. Kelner, who is also my academic advisor, is super knowledgeable and quite pedagogical
    • super interesting class
    • this was still probably one of the most hard to follow classes I’ve taken though? it was very advanced and moved quickly

  • Inference and information – 6.7800

    • great class

    • concretely learned:

      • what is the formal and statistical setup people have found to describe the problem of learning & inferring things given data, when evaluated with respect to the optimal solutions (and what are those solutions)

      • what are the main challenges to these kinds of learning problems, and what are desirable properties or assumptions we can make to simplify our analysis

      • given large amounts of data on a process we want to model, under certain assumptions, what can we say about the techniques that estimate that parameter asymptotically

      • how can we quantify the notion of “information” and estimating randomness

    • Meta skills

      • the mathematical knowhow to take a bunch of scary symbols and boil it down to intuitive understanding and solve problems/prove properties of your algorithms

      • statistical background to understand most not too niche ML papers
      • a “physicist’s mindset” to inference problems: learning how to take problems and kind of guess their solution/play with the question to solve it

  • Secure Hardware Design – 6.5950

    • very fun
    • I like the hacker mindset because breaking things, especially secure things, makes you focus on really understanding something through and through. You need to explore every angle and get a deep understanding of the way the components interact. Now, in many fields you need to learn something deeply, but often especially in school you don’t have real feedback loops on your learning, or at least feedback loops that aren’t artificial like exams. This makes your understanding more brittle, whereas when you try to break something you explore every facet, and breaking it adequately becomes the feedback loop for your learning.
    • This mindset of turning things into a game extends to many different things. Even in math, you often read about how great mathematicians have the skill of playing with mathematical objects and examples, breaking them down, seeing where things fail and where they work, to develop a good perspective of the field. I think it’s pretty key, but in cybersecurity it’s baked in.

  • 21W.735 Reading and Writing the essay

    • read a bunch of really good essays and got precise detailed feedback. This class has shown me how writing can really help me clarify my thoughts on something, and I’ve gotten better at conveying ideas in a pretty, argumentative way.

    • I think my writing interests sits in this zone between conveying aesthetic emotion/a sense of shared empathy and understanding and actually bringing out philosophical ideas and arguments.

    • What I wrote about:

      • existentialism and committing to one’s values

      • cultural elites and the values they reflect, contrasting French intellectualism and the American tech scene

      • judgement as a form of love and deep empathy

      • political polarization and the rise of discourse focused on making institutions feel safe rather than advancing particular values in a normative way
      • I want to unpack some of these more and want to bring them to the blogs for 2025.

  • topology 18.901

  • 18.821 Project lab in mathematics

    • learned some stuff about mathematical communication and technical writing
    • worked with some fun people, pretty cool stuff but quite a bit of work for a communication class

I also did two reading programs with grad students for the MIT directed reading programs, where we read through technical content and then I presented at the end to my fellow students!

  • One on differential topology, following Guillemin & Pollack
    • book is really good, content was fun, enjoyed
  • One on statistical mechanics, following David Tong’s notes
    • also fun, stat mech is nice
    • ended by summarizing a Lecun paper on spin glass models of neural networks

What I want to learn #

  • basics of neuroscience
  • basics of biophysics, evolution dynamics, selection mechanisms in biology
  • reinforcement learning
  • flexibility
  • robotics?
  • explore AI for Math
  • read more philosophy than I did this year

Writing

What I wrote

Not much! Lots of journaling/independent stuff but not much I wanted to post.

I did a lot of writing for my classes, as I mentioned before, in poetry and essays, which I want to catch up to and put out there. My 3 public posts are here, mostly life updates and some thoughts.

I wrote papers! LOL

And some technical writing. I’ve gotten less and less public, for better or worse, partially also as I’ve felt that the standards I want to have increase, and my slack decreases.

What I wanted to write

  • a primer on abstract algebra
    • Nope
  • philosophy poasting
    • Nope
  • more poetry
    • !
  • technical posts that link ideas across subjects
    • AI and TCS?
      • Nope
  • thoughts on rationality

What I want to write

I want to publish and finish some of the thoughts from this year, notably:

  • essay on cultural differences between the France-US, what heroes say about culture, and the process vs the outcome
    • and where I fit in with all of this
  • list impactful papers; research ideas
  • distill my research aesthetic
  • write down some research lessons from the past year
  • write down some pieces on powerful media I’ve been consuming
    • eg taste of cherry, Dostoevsky

Content I liked

Note: shifting style of this post from enumerative to still enumerative, but for the things that actually stood out, not complete.

Books

Did not read much this year! Want to read more next year. These are just the books that I find worth mentioning, the rest are listed here, along with a bunch that I didn’t really find worth mentioning afterwards.

Movies

  • Taste of Cherry
    • just wow, maybe one of my favorite movies ever now
  • In the Mood for Love
    • some insane scenes
  • Whiplash
  • Beau Travail
  • Shutter Island
  • Mishima biopic

Articles

https://www.nplusonemag.com/issue-35/fiction-drama/the-feminist/

https://www.gleech.org/love

https://www.dwarkeshpatel.com/p/gwern-branwen

https://www.commonreader.co.uk/p/learning-to-love-how-the-poet-dana

https://barnacles.substack.com/p/understanding-as-an-art

https://www.oldtimestrongman.com/articles/the-iron-by-henry-rollins/

https://dynomight.substack.com/p/automated

https://x.com/RichardMCNgo/status/1835381944722563284

https://www.alexirpan.com/2016/01/03/grad-school.html

https://goldenblue.substack.com/p/siddhartha

https://www.theatlantic.com/magazine/archive/2001/12/all-you-need-is-love/302351/

https://paulgraham.com/worked.html

https://goldenblue.substack.com/p/only-the-body-speaks

https://leonardtang.me/posts/Life-Gradient-Descent/

https://paulgraham.com/genius.html

https://danwang.co/college-girardian-terror/

https://www.approachwithalacrity.com/101-things-for-my-past-self/

https://www.bitsofwonder.co/p/give-your-friends-a-chance-to-abandon

https://twitter.com/NPCollapse/status/1763960083858198573

https://nicholas.carlini.com/writing/2024/my-research-logfile.html

https://amalianegreponti.com/why-did-you-come-here/

https://vitalik.eth.limo/general/2024/01/31/end.html

https://www.fortressofdoors.com/i-lost-my-son/

https://usefulfictions.substack.com/p/1154dba1-49f6-4feb-b091-6d4a7eefa94d

https://space.ong.ac/escaping-flatland