cognitive augmentation by Vincent H. '23

last semester, aside from doing work for the classes i was enrolled in, i also read through all the course materials for a few other classes i was interested in but didn’t have time to take. one of those was cognitive augmentation (mas.s65), a class on using technology to augment human capabilities in creative ways that i’d for the most part never considered before. it turned out to be a very inspiring class that truly expanded the limits of what i thought was possible with present-day technology. in this post i’ll briefly summarize a few of my favorite readings from the class; if you’re curious, you can find the other readings on the course site linked above

before we go on, i should mention that the replication crisis in psychology and cognitive science is extremely prevalent and important. as in, researchers have been unable to consistently replicate famous results such as the dunning-kruger effect and the stanford prisoner experiment. so while i typically trust mit professors to fact-check course content before teaching it, please take all of the following findings with healthy amount of skepticism; i don’t have time to do a detailed literature review on everything i’ll be writing about here, so if you want more details on experiment setup and validity you’ll have to look into it yourself

i. identity illusions

• Mel Slater: becoming a better person through VR embodiment. this article focuses on targeting racial and sexual biases through VR (virtual reality) embodiment, ie. VR experiences where your virtual body belongs to a different race or gender than your physical body. for instance, they put white people in black avatars and found that simulating positive experiences resulted in the study participants having decreased implicit racial bias (whereas simulating negative experiences increased bias). meanwhile, putting men in simulations of women undergoing sexual harassment resulted in less measured tendency to harass women afterwards
• Conversations between self and self as Sigmund Freud—A virtual body ownership paradigm for self counselling. in this study, participants underwent a VR simulation of a therapy session that took place as an extended conversation with themselves. there were two figures in the simulation – the participant, as well as a figure that looked like freud, and each participant would take turns embodying either their own avatar or the freud avatar while talking to themselves. it’s difficult to get a meaningful measure of therapy session quality or long-term improvement, but the authors did find that the participants’ moods after this conversation were significantly more improved than when freud was replaced with a generic avatar
• Virtually Being Einstein Results in an Improvement in Cognitive Task Performance and a Decrease in Age Bias. one more paper on VR identity illusions. this time, the study participants worked through a series of cognitive tasks and surveys, except some of the participants’ VR bodies resembled einstein. as you can infer from the paper title, the people with einstein avatars performed better on the cognitive tasks and displayed reduced ageism. (the most plausible explanation for why they did better on the tasks is that the altered appearance increased their confidence, not that it made them smarter)

ii. sleep

• Sleep for cognitive enhancement. this article discusses applications of modifying sleep to help people learn better. we’ve found that there are specific brain oscillations which occur during sleep that seem to help consolidate new memories (which may partially explain why you don’t have a good memory of things that happen when you’re sleep-deprived). the idea in this paper is that we can trigger oscillations to target and reactivate specific memories. the researchers expose people to certain audio cues and scents while playing a memory concentration game, and then re-expose people to the same signals while they’re sleeping; re-exposure to these signals causes the neurons related to the game to become active again during sleep, and the researchers found that this results in better game performance on the following day. they also try triggering the memory consolidation oscillations manually by running electric currents through the brain; this seems to work too
• Following a Script to Escape a Nightmare. this paper isn’t really about cognitive augmentation as much as a general tactic for reducing fear. they take people with recurring nightmares and perform rehearsal therapy on them. the people imagine a version of the nightmare where some of the scary details are replaced with harmless ones; then they imagine a room where the scary version of the nightmare is on the far side of the room and the tame version is on the near side. this scripting procedure apparently helped the patients stop having nightmares, and i believe similar tactics have been used on other phobias
• Quit Smoking in Your Sleep. in this study, researchers expose smokers to the scent of cigarette smoke, followed by a bad smell like that of rotten eggs. when they perform this procedure during restful sleep, a significant portion of the smokers quit smoking in subsequent weeks; when they try the same procedure while the participants are awake, they notice no significant change in quit rate. nobody really understands why the procedure has to be done during sleep to be effective, but that is what the data currently suggests

iii. new senses

• Can You See With Your Tongue? this is the standard neuroplasticity experiment which helped motivate the design of artificial neural networks, meaning i’d already seen this result before from taking machine learning classes, but i thought it’d be worth sharing here anyway. the idea behind neuroplasticity is that the brain can process signals consistently regardless of their source (and this helped ml researchers realize they could build artificial neural networks where each neuron had essentially the same structure, whereas previously ai researchers would hand-craft large decision trees where each node had hard-coded structure and features). so if you blindfold someone but feed in electric signals encoding visual data through an array of electrodes implanted onto their tongue, their brains will still be able to process the signals and they’ll see through their tongues instead of their eyes. this has clear applications in eg. helping restore sight in people with damaged optic nerves. a similar method can be used for other senses, eg. peoples’ sense of balance comes from the orientation of tiny hairs in their inner ears, so if someone’s hairs are damaged we can feed in the relevant electric signals through their tongues and they’ll still be able to feel balance and walk normally. (neuroplasticity studies tend to feed in signals through the tongue because it has a high concentration of nerve endings)
• An attempt to extend human sensory capabilities by means of implant technology. this study attempts to extend the concept of neuroplasticity one step further – instead of rewiring senses the brain already processes, like vision and balance, they try to teach the brain an entirely new sense, namely that of detecting ultrasonic waves (which human ears can’t hear). they take an ultrasonic sensor and implant it in a human arm nerve; whenever the sensor goes off, it stimulates the nerve at a frequency that it usually doesn’t get stimulated at, thereby producing a unique signal in the brain that typically doesn’t exist. then they teach users to understand when this signal is on vs off by repeatedly giving them examples where ultrasound is present or absent, and after performing this calibration for a while users can navigate a room blindfolded while primarily relying on ultrasound feedback for obstacle detection. this paper is relatively short (only 6 pages) but was probably the most impressive reading from the class in my opinion, because they’re able to achieve a novel result through very simple methods and first principles (ie. no invasive brain surgery or black boxes)

iv. miscellaneous

• ​​Project Us: A Wearable for Enhancing Empathy. this is a cute project which doesn’t really perform cognitive augmentation as much as it provides a new experience for people. they take couples and measure their emotional arousal during a conversation (this can be done reliably by generating tiny currents and measuring how well a person’s skin conducts electricity; tldr more arousal = more sweat = more conduction). when one person was emotionally agitated, their partner was alerted in real time; this allowed the couples to have a better sense of each others’ emotional states, since explicit communication of feelings is often difficult, and for the most part the couples said they enjoyed the experience and would want to incorporate it into their regular interactions
• Living bits: Opportunities and challenges for integrating living microorganisms in human-computer interaction. this paper discusses utilizing microbes to essentially act as bio-computers. for instance, they develop microbial yogurts which measure various health indicators as they pass through a person’s digestive system; the microbes then produce different pigments depending on the level of indicator that was detected. the end result is that the feces containing the microbes have different colors depending on the levels of health indicators, so consuming this yogurt provides people with a noninvasive way of understanding their own health at a rudimentary level. they also do similar experiments where microbes which respond to certain pollutants are embedded into clothing, resulting in clothes that change color depending on air quality
• NeverMind: Using Augmented Reality for Memorization. this paper focuses on using AR (augmented reality) to increase the strength of mnemonic devices. the researchers focus on a commonly used mnemonic called the memory palace, where users take facts they are trying to remember, create a symbolic representation of the facts, and then imagine positioning that symbol at a specific location within an imaginary house. then, to recall facts they previously memorized, the users picture their imaginary house and walk through it until they come across the relevant symbol; i’ve never used this method myself but it apparently helps people remember things. anyway, the insight behind this paper is that instead of having a purely imaginary house, users can be given a vr headset with an actual house inside, and while they perform the memory palace mnemonic they can also perform the same spatial indexing process using the house in the vr headset; this process takes the already-effective memory palace mnemonic and apparently improves retention even further
• BoostMeUp: Improving Cognitive Performance in the Moment by Unobtrusively Regulating Emotions with a Smartwatch. this is a simple application of technology to emotional regulation. the problem here is that when people are nervous or anxious, they often perform worse (eg. auditions, tests, sports events), and the knowledge that they’re under pressure often causes them to continue feeling pressure. the researchers’ proposed solution is creating a smartwatch that notices when the user’s heart rate is elevated and then produces slow pulses when this is the case; the user’s brain then instinctively associates these slow pulses with their own pulse, thereby causing them to think their heart rate is slower than it actually is. in some cases this can break the cycle of self-perpetuating pressure, and the researchers found that it gave the users a higher chance of remaining calm during high-stress environments