AI resources

Here are a few AI-related resources I've generated or collected. (This list is updated periodically; I think I've been adding something new at least once every few months or so. I'll also eventually organize it more systematically, once a good scheme for doing so becomes clearer.)

• For an in-depth treatment of the shockingly wild possible short- and medium-term future of AI (within the next few years), I highly recommend Leopold Aschenbrenner's Situational Awareness essay series. (In short: it's plausible that we might have AGI by 2027 and ASI anywhere from 10 months down to 10 hours thereafter, and the race to AGI absolutely must become a government project along the lines of the Manhattan Project.) The entire sequence is pretty long (165 pages, albeit with quite wide margins), but the link contains a one-page introduction as well as a one-paragraph summary of each essay. It's worth mentioning that these views are not universally held (cf. e.g. this podcast episode), but they do seem to be fairly dominant in SF.
• I highly recommend the 3Blue1Brown video series on deep learning, which explains the mathematics of neural networks (including GPTs) in a clear and compelling way.
• Slides from a short expository talk called "A mathematical introduction to the transformer architecture" are here.
• I've been consuming Andrej Karpathy's beautiful Neural Networks: Zero to Hero lecture series, which starts essentially "from scratch" and arrives at (a baby version of) GPT-2 in lecture 7. It's really great: he goes through both the math and the code, and crucially he also explains the journey of how we (as a species) arrived at these various design decisions -- a sort of potted history of AI in the modern era. To solidify my own understanding, I've been writing notes on both the math and the journey (omitting the code), adding a bit of mathematical background and rigor along the way. You can see an incomplete draft here. If you'd like to be notified when the notes are complete, please let me know via the Gmail address with username "the.etale.site".
• For an introduction to alignment research, I highly recommend the ARENA curriculum. Its explanations are clear, and its exercises are both precise and adequately closed-ended (as opposed to open-ended); solutions are also provided. (This is to a decent extent based on the curriculum from Redwood Research's Machine Learning Alignment Bootcamp (MLAB), access to which can be requested here.)
• As a complement to the ARENA curriculum, I also highly recommend Jacob Hilton's Deep Learning Curriculum. By contrast, it offers a number of readings and associated open-ended implementation challenges, grouped by topic. Others' solutions are linked in the GitHub repository.
• For getting a handle on the actual process of training neural networks, I recommend Andrej Karpathy's recipe.
• In response to a New York Times guest essay, I wrote a simple Python script that generates AI-generated "beach read" stories; see here for further explanation.