In our pursuit to perfect neural networks, we often look to how humans learn for reference, which has had varying degrees of success....

Many facets of our universe seem to gravitate towards unstable, chaotic equilibria. One such example is criticality, a threshold where a...

In another post, I wrote about parameterless (boneless) attention as a means of mixing information across datapoints weighted by their...

I felt compelled to make this post after seeing yet another reinforcement learning paper for diffusion models that does spectacularly in...

Lately, I've felt that a lot of human culture is becoming the same. Groups have become very hive-minded and homogenous on an...

We are racing toward an uncertain future that draws nearer every day. We have a rough idea of what the future could look like. I'm...

Something I've been thinking about a lot lately is how humans handle noisy recognition. Maybe you recognize the image above, if not you...

Once upon a time, I was set on becoming a psychiatrist. I had always been deeply interested in psychology. Throughout life, I've spent...

The Distillery - a look under the hood of vibe coding Introduction Vibe coding may be one of the best and worst things 2025 has had to...

I’ve seen a lot of convoluted tutorials on attention but nothing really made it click for me more as understanding as mixing a projected...

One conviction I hold very strongly is that "special" people are possibly much more common than we may be lead to believe. While sure, we...

Presently, most optimizers used in deep learning do not explicitly accommodate their updates with respect to the expected range of...

Within the study of machine learning, you'll often hear that the objective is to find the solution that maximizes likelihood . We have a...

Calibrated Attention Calibrated Attention NanoGPT Attention is the magic ingredient of modern neural networks. It is the core of what has...

A short attempt at a small portion of the diffusion Family Tree https://www.canva.com/design/DAGgnVB3x2s/b52Y3Kg-frWdRlPzI3_5pA/edit?utm_...

Mandelbrot function: Zn+1 = (Zn)^2 + C | Location: -1.4732524061369524549 + -0.0058138265122775765014 i , Radius:...

One of the most interesting things about the development of AI was the order of achieved milestones. Relatively small models can create...

It's a bit hard to see in the diagram but in addition to being convolved with a gaussian, these points are also drifting towards zero....

https://sweet-hall-e72.notion.site/Classifier-Free-Guidance-to-Approximate-RL-9f78c02801c6434da61f37c8d843c5bf

"Would someone without free will do this?" is one of the most common responses I've heard to the proposition that free will might not...