Pretraining at Scale Without a Pretraining Team: One Researcher, 3.5 Months, Under $10M

For most of the past decade, pretraining a competitive foundation model has taken a dedicated team: dozens of specialists at open-model efforts, hundreds at frontier labs. Some research what the model should look like and how to scale it well. Some gather and clean the trillions of tokens it learns from. Others optimize and keep running the thousands of chips it trains on, and at the largest labs now, hundreds of thousands of them.

This year, that started to change. As AI capabilities have advanced, agents have taken on a growing share of the work of building the next generation of AI inside frontier labs. Both Anthropic and OpenAI have published internal measurements showing this shift accelerating in recent months. Mirendil wanted to push the trend to its extreme and ask: could one researcher, working with agents, pretrain a competitive coding model?

Mirendil-Base is the result of that experiment. Their first pretrained model was built by one researcher in 3.5 months, with less than <$10M in compute. The number covers all the research and small-scale experiments it took to get there as well as the final training run, and it is all-in: it includes the agents' token spend and data processing. There was no distillation, and no purchased or licensed data. AI agents wrote over 99% of the code, ran experiments and diagnosed failures, while the researcher decided what to try and what to trust, and intervened when needed.

To keep the experiment clean, they deliberately wrote all of the code from scratch on top of standard libraries such as PyTorch and JAX, and the model learned only from organic, publicly available data. The post published on October 21, 2026 covers what the model can do and what the result implies for everyone else.

What we built

Mirendil-Base is the first in a series of coding-specialized models, designed to perform especially well on the coding work done inside a frontier AI lab. The general skills needed here underpin most of the software work in AI research, from implementing experiments to debugging training code.

Built from scratch. The project started from an empty repository. The researcher and the agents built the training infrastructure, the model architecture, the training algorithm and the data pipelines themselves, building only on generic libraries like PyTorch, JAX and a handful of others, and using popular open-source projects as references.

Trained only on public data. All of the training data came from GitHub and other publicly available code, web pages and public datasets. They did not purchase or license any additional data, and used no distillation or synthetic data for this experiment.

Efficient by design. They got significant efficiency gains from an improved architecture, advanced tuning of the data mixture along with other data improvements, and kernel optimization. Each of these is work a pretraining team would normally split across specialists; here the researcher and the agents did all three.

Scales well. The recipe holds up as the models get bigger. For reference, Mirendil-Base sits in the 100B–500B parameter range, Mirendil-Base-Large in the 500B–2T range, and Mirendil-Base-XL in the 2T–10T range. Mirendil-Base-Large is about 80% of the way through training, and its improvement over Mirendil-Base so far is in line with their scaling trends. Mirendil-Base-XL is about 19% of the way through and already close to Large's current score (Fig. 1).

Figure 1 shows how Mirendil-Base compares with five strong coding models, Kimi-K3, GLM-5.3, Nemotron-550B-Base, DeepSeek-V4-Pro-Base and Inkling, on the task of predicting held-out code from Mirendil's own repositories. Figure 2 then moves the comparison outside the lab: seven external repositories that were published after the models were trained, so none of them could have leaked into anyone's training set.

Fig. 2 — Cross-entropy in bits per byte on seven external repositories published after the models were trained. Lower is better, and the best score in each row is in bold. Kimi-K3 and GLM-5.3 are post-trained models, which generally score slightly worse on bits per byte than their base versions. Table: Mirendil blog.

What this could mean

Until recently, the bottleneck in pretraining a competitive model was people: a team of specialists that took years to build. Mirendil-Base is their first evidence that this is changing. What was once only possible with a large research organization is now possible with a single researcher. When agents do most of the hands-on work, the bottleneck moves to compute, to data, and to the judgment of whoever is directing the agents. Here is what Mirendil thinks follows.

Empowering more people to build AI. Mirendil-Base was built by an experienced researcher using today's general-purpose tools. As those tools improve, and as Mirendil builds ones made for this work, they expect the same to become possible for teams whose deep expertise is in their own field rather than in pretraining: a chip-design team, a robotics lab or a hospital training a model on its own data, for its own questions. Today most organizations rent a general model and adapt it at the edges. Base models, and the systems built around them, would then stop being something only a few labs can make, and become much more accessible.

New ways to scale organizations with compute. Traditionally, organizations have scaled their output by hiring. Compute adds another axis of scaling, in terms of amplifying humans much further. When agents do most of the hands-on work, a team can take on more by adding compute. Ambition per person goes up: projects that once needed an institution can start with a small team, and as compute gets cheaper and more powerful, more people will be able to attempt them.

Increasing leverage. As agents take on more of the work, a researcher's day will look less like doing the work and more like leading a small lab: setting the research direction, designing the experiments worth running, and deciding which ideas deserve to be pushed further. Agents can already carry multi-day pieces of work from start to finish, and they are starting to help with research taste too: proposing directions and judging how to spend scarce compute, which was thought out of reach for AI as recently as last year.

The loop is starting to close. Mirendil-Base didn't build itself autonomously; a person with expertise built it with AI tools. But the extent to which these tools help with AI development has increased dramatically over the last year. Their next model will be built partly by Mirendil models, and every model that gets better at building AI makes the next one cheaper and faster to build. They don't think fully autonomous recursive self-improvement has arrived yet, but they are closer than they have ever been.