Developer releases mini-AGI, a continual learning model trainable on 8GB consumer GPU
A new open-source language model lets users train and continuously improve their own AI on modest hardware without forgetting previous knowledge.
What to know
- mini-AGI enables training and continuous learning of language models on consumer GPUs (8GB VRAM) without catastrophic forgetting—a capability absent from existing commercial models.
- The model stores weights on disk and pages them to GPU as needed, making parameter count limited by disk space rather than VRAM, bringing full training within reach of individual users.
- Creator describes it as a toy-level proof-of-concept, not frontier capability, demonstrating that continual learning without catastrophic forgetting is feasible on modest hardware with accessible tooling.
volotat Developer, mini-AGI creator
How it unfolded 1 development · click the chart to see its coverage articlesposts
-
1
Creator positions mini-AGI as alternative to frozen commercial models
volotat articulates the core problem mini-AGI addresses: existing language models are frozen after training, cannot be trained from scratch on personal hardware, and forget previously learned knowledge when fine-tuned on new data. Mini-AGI's architecture enables continuous learning from a stream of data without catastrophic forgetting on modest hardware.
“Every language model you can actually own today is a model somebody else trained and then froze… The moment you try, it forgets what it knew before.”
— volotat -
first by HN Best, 5d ago · also HN Frontpage
1 more headline
- Mini-AGI – dynamic continual learning model trained from scratch on 8GB VRAM HN Frontpage · 5d ago
-
I am not sure about eventually, but it learns on the steady paste so far. The main thing to keep in mind is that it learns on the single STREAM of data. Not randomized batched samples. Try to do it with any other model and you will see nothing but complete garbage in the predictions, exactly because of catastrophic forgetting.And here are the…
2 more of the top 3 · 13 posts in this stretch
-
I built a smaller scale replication. Most of your claims prove out but the lever is essentially Biderman et als "learn less forget less". My first pass actually made this mistake initially and the findings didn't replicate. After controlling for step density I replicated your findings. Here is a write…
-
I have not looked carefully but it seems like this is over-promising on avoiding catastrophic forgetting.The "trunk learning rate" is set at 0.1x the learning rate for the experts, so learning on different subjects disproportionately happens in the experts, and the trunk portion is comparatively more stable. But the population of experts can grow…
-
-
background
Developer volotat releases mini-AGI on GitHub — volotat publishes mini-AGI, a continual learning byte-level language model designed to train from scratch on a single 8GB VRAM GPU. The model stores weights on disk, pages them onto the card as needed, and dynamically grows or prunes capacity during training. It is currently in early stages with weights not yet published while the initial training run continues through its first pass over the corpus.
What people are saying 9 voices from 1 site · best of 13 · verbatim
- Sep 22
-
I read your github page and there's mention of continual learning but no argument how catastrophic forgetting is avoided. I think you should explain that better.
-
I'd love to explore a question. If I have a standard elementary school math textbook, along with all the results of every correct and incorrect answer my child got on the textbook exercises and class tests, is it possible to train a model that can help me figure out what concepts my child is struggling with and have that model provide help?
- Sep 21
-
Mini-AGI is a totally inappropriate name - it seems what this project is shooting for, but not delivering on, is being a language model with "continual learning".Where it seems to fail, by design, on this goal is in delivering continual learning that is more than just "memorization with LRU catastrophic forgetting".That said, props to the author…
-
Interesting. I wonder how much could be gained from using tokenization, which makes the model work at a semantic level rather than a syntactic level? I think it’s a force multiplier, but idk if it works here.
-
If you actually scroll through the transcript he links to, you will see that something that looks like it could be training is happening, but no coherent responses are coming out at any point. At least not that I saw skimming through.That might explain why there are no benchmarks of any kind.
-
This is the first thing I see in my life that really looks like proto-AGI, it deserves its name.
-
Is this architecture actually able to generalize or is it mostly based on memorization? Have you tried some basic tasks that require generalization? e.g. number addition etc?
-
Seems interesting, I've been messing with a lot of continuous learning approaches lately and it's cool to see something that's built from the ground up for avoiding catastrophic forgetting. Worth a clone for sure
-
Nobody will throw rocks, I think most people are curious/suspicious about the big players and wants more hands-on since we suspect that this all will come down in cost soon enough.