top of page



Minimum Faith
Within the study of machine learning, you'll often hear that the objective is to find the solution that maximizes likelihood . We have a...
Ethan Smith
Mar 148 min read


Softmax Attention is a Fluke
Calibrated Attention Calibrated Attention NanoGPT Attention is the magic ingredient of modern neural networks. It is the core of what has...
Ethan Smith
Mar 1310 min read


Discrete Diffusion Sudoku and Diffusion Lore
A short attempt at a small portion of the diffusion Family Tree https://www.canva.com/design/DAGgnVB3x2s/b52Y3Kg-frWdRlPzI3_5pA/edit?utm_...
Ethan Smith
Mar 35 min read


Kolmogorov Complexity
Mandelbrot function: Zn+1 = (Zn)^2 + C | Location: -1.4732524061369524549 + -0.0058138265122775765014 i , Radius:...
Ethan Smith
Mar 114 min read


To create something new, you need to make some noise.
One of the most interesting things about the development of AI was the order of achieved milestones. Relatively small models can create convincingly masterful lifelike art while a much larger language model may fail common sense problems that your average 10 year old may have no problem with . But then at the same time, they are incredibly capable. The same language models failing at some trivial tasks may also explain difficult math concepts in a very intuitive manner and
Ethan Smith
Feb 127 min read


How I like to think about diffusion
It's a bit hard to see in the diagram but in addition to being convolved with a gaussian, these points are also drifting towards zero....
Ethan Smith
Jan 264 min read


Classifier free guidance and reinforcement learning
https://sweet-hall-e72.notion.site/Classifier-Free-Guidance-to-Approximate-RL-9f78c02801c6434da61f37c8d843c5bf
Ethan Smith
Jan 261 min read


The Tough Case for Free Will
"Would someone without free will do this?" is one of the most common responses I've heard to the proposition that free will might not...
Ethan Smith
Jan 1311 min read


How absurd it is, that anything is at all.
Art from "In the Aeroplane over the Sea" by Neutral Milk Hotel There are several dimensions by which we may feel small. You may feel...
Ethan Smith
Jan 1210 min read


AI should help humans understand each other.
Empathy through technology
Ethan Smith
Oct 18, 202410 min read


Why are Modern Neural Nets the way they are? And Hidden Hypernetworks.
https://sweet-hall-e72.notion.site/Why-are-Modern-Neural-Nets-the-way-they-are-And-Hidden-Hypernetworks-6c7195709e7b4abbada921875a951c54
Ethan Smith
Oct 6, 20241 min read


Do Diffusion Transformers Deserve The Hype?
https://sweet-hall-e72.notion.site/Do-Diffusion-Transformers-Deserve-The-Hype-9b9ca7bead374b47aac96558714c203b
Ethan Smith
Jul 28, 20241 min read


Automated LoRA Discovery and Teaching Neural Networks to make Neural Networks
https://sweet-hall-e72.notion.site/Automated-LoRA-Discovery-and-Teaching-Neural-Networks-to-make-Neural-Networks-22aa3b5ad66e4bc985ff2c93...
Ethan Smith
May 26, 20241 min read


Diffusion and Autoregressive Models for Learning to Solve Mazes
https://sweet-hall-e72.notion.site/Diffusion-and-Autoregressive-Models-for-Learning-to-Solve-Mazes-c3bc4bcdfa304ecd9531ee5445a4da66
Ethan Smith
May 21, 20241 min read


I Learn to Diffuse, or Data Alchemy 101: a Mnemonic Manifesto
https://arxiv.org/pdf/2208.03998
Ethan Smith
May 12, 20241 min read


Traversing through CLIP Space, PCA and Latent Directions
https://sweet-hall-e72.notion.site/Traversing-through-CLIP-Space-PCA-and-Latent-Directions-b898932e13684d58957405b4a2747a79
Ethan Smith
May 6, 20241 min read


LoraNorm
https://sweet-hall-e72.notion.site/LoraNorm-91f5927c7c3a48f9beeec7c2910c82ad?pvs=4
Ethan Smith
Apr 14, 20241 min read


2 UNet Classifier Free Guidance
https://sweet-hall-e72.notion.site/2-UNet-Classifier-Free-Guidance-e2eb8833d01e4e77869a2eff4631ebbc?pvs=4
Ethan Smith
Apr 14, 20241 min read


SGDImagePrompt and QuickEmbedding
https://sweet-hall-e72.notion.site/SGDImagePrompt-and-QuickEmbedding-502cf95e48ff4a4db44823e117b7f779?pvs=4
Ethan Smith
Apr 14, 20241 min read


Learning Space Filling Curves with Autoencoders
https://sweet-hall-e72.notion.site/Learning-Space-Filling-Curves-with-Autoencoders-e39e41ce75894c3a8fecfee0f3bbfb23?pvs=4
Ethan Smith
Apr 14, 20241 min read
bottom of page