In our pursuit to perfect neural networks, we often look to how humans learn for reference, which has had varying degrees of success....

In another post, I wrote about parameterless (boneless) attention as a means of mixing information across datapoints weighted by their...

I felt compelled to make this post after seeing yet another reinforcement learning paper for diffusion models that does spectacularly in...

Something I've been thinking about a lot lately is how humans handle noisy recognition. Maybe you recognize the image above, if not you...

I’ve seen a lot of convoluted tutorials on attention but nothing really made it click for me more as understanding as mixing a projected...

Presently, most optimizers used in deep learning do not explicitly accommodate their updates with respect to the expected range of...

Calibrated Attention Calibrated Attention NanoGPT Attention is the magic ingredient of modern neural networks. It is the core of what has...

A short attempt at a small portion of the diffusion Family Tree https://www.canva.com/design/DAGgnVB3x2s/b52Y3Kg-frWdRlPzI3_5pA/edit?utm_...

It's a bit hard to see in the diagram but in addition to being convolved with a gaussian, these points are also drifting towards zero....

https://sweet-hall-e72.notion.site/Classifier-Free-Guidance-to-Approximate-RL-9f78c02801c6434da61f37c8d843c5bf

https://sweet-hall-e72.notion.site/Why-are-Modern-Neural-Nets-the-way-they-are-And-Hidden-Hypernetworks-6c7195709e7b4abbada921875a951c54

https://sweet-hall-e72.notion.site/Do-Diffusion-Transformers-Deserve-The-Hype-9b9ca7bead374b47aac96558714c203b

https://sweet-hall-e72.notion.site/Automated-LoRA-Discovery-and-Teaching-Neural-Networks-to-make-Neural-Networks-22aa3b5ad66e4bc985ff2c93...

https://sweet-hall-e72.notion.site/Diffusion-and-Autoregressive-Models-for-Learning-to-Solve-Mazes-c3bc4bcdfa304ecd9531ee5445a4da66

https://sweet-hall-e72.notion.site/Traversing-through-CLIP-Space-PCA-and-Latent-Directions-b898932e13684d58957405b4a2747a79

https://sweet-hall-e72.notion.site/Learning-Space-Filling-Curves-with-Autoencoders-e39e41ce75894c3a8fecfee0f3bbfb23?pvs=4

https://sweet-hall-e72.notion.site/Mimicking-Diffusion-Models-by-Sequencing-Frequency-Coefficients-8e5a60e876d640c390369627d55330b1

https://sweet-hall-e72.notion.site/ContrastiveDPO-for-Diffusion-Generalizing-DPO-to-multiple-items-PART1-226b3746aa4d4ff9995d1e26b38a9674

Image from: https://twitter.com/toshi2fly/status/911306344376012800 Post: https://sweet-hall-e72.notion.site/Dipole-Attention-Opposites-M...

https://sweet-hall-e72.notion.site/Speeding-up-Diffusion-Reviewing-DiffusionGANs-Consistency-Models-and-Flow-Models-80b985120b8f472094cdc...