This 4000-Parameter GPT Fits in Your Browser and Teaches You Nothing About Real AI
Are we really learning about GPTs when we shrink them down to toy size? Meet Microgpt, the latest Show HN darling that lets you visualize a 4000-parameter "language model" right in your browser at microgpt.boratto.ca.
The project, posted by user "b44" in February 2026, earned a respectable 98 points on Hacker News by doing something genuinely useful: making AI less mysterious. Built on Andrej Karpathy's microgpt - a 243-line Python implementation that dropped just days earlier - this browser version ports the concept to WebAssembly and WebGPU.
Here's what you get: a neural network that learns to generate names using a simple 26-token alphabet. One token per English letter. Click around, watch activations flow through attention heads, see the KV cache in action. It's interactive AI education without the usual Python dependency hell.
<> "The 26-token simplicity versus larger models' tokenization challenges" makes this approachable, as HN user mips_avatar noted. No subword tokens, no vocabulary nightmares./>
The pedagogy is solid. Real GPTs use complex tokenization schemes that obscure what's actually happening. This character-level approach lets you reason about every decision. You can literally see why the model thinks 'A' should follow 'MAR' when generating "MARY."
But here's my problem with these educational toys: they're lying by omission.
Production GPTs don't work like this. At all. The challenges that make or break real language models - subword efficiency, scaling laws, distributed training, alignment - disappear entirely at 4000 parameters. We're teaching people to understand horse-drawn carriages when they need to build rockets.
The technical execution is admirable:
- Browser-based training with WebGPU acceleration
- Real-time visualization of attention maps
- Interactive forward pass exploration
- Clean port of Karpathy's Adam optimizer and backprop implementation
The gap between toy and reality is a chasm. This microgpt trains on name datasets with hyperparameters like learning_rate=0.01 and beta1=0.85. GPT-4 trained on trillions of tokens across thousands of GPUs with techniques this toy will never need: gradient checkpointing, model parallelism, reinforcement learning from human feedback.
HN commenters get this partially. User kfsone praised how it "avoids complex subword tokens to focus on spelling names" - but that's exactly the problem. Subword tokens aren't complexity for complexity's sake. They're fundamental to how modern language models actually work.
<> "No formal expert endorsements from Karpathy or others noted," though his original microgpt serves as the foundation./>
The market opportunity is narrow. This sits firmly in the educational category with zero commercial viability. The 4000-parameter limit makes it useless for anything beyond demonstration. It might inspire edtech startups or boost the creator's portfolio for AI visualization gigs, but it won't scale to production.
Hot Take: Educational AI tools that oversimplify do more harm than good.
Yes, this makes transformers less intimidating. Yes, beginners can finally "see" attention mechanisms. But we're creating a generation of developers who understand toy models while production AI races ahead at incomprehensible scale.
The real value isn't in the 4000 parameters - it's in Karpathy's original insight that you can build a complete GPT implementation in under 250 lines. That's the lesson worth preserving.
Microgpt succeeds as a visualization tool and fails as preparation for real AI work. It's the neural network equivalent of learning to code with LOGO turtles: charming, memorable, and ultimately inadequate for the complexity ahead.
Still, click around. Watch those attention heads light up. Just remember you're playing with shadows on the cave wall.
