A better research system is not the same thing as a research system guaranteed to succeed. That distinction matters enormously.
I recently changed my mind about AGI.
My starting position was straightforward: large language models are extraordinarily useful, but making a language model bigger does not automatically turn it into a complete, autonomous intelligence.
I still think “just scale it until AGI appears” leaves rather a lot unexplained. But I had been looking too narrowly at the model itself.
The more interesting possibility is that AGI emerges from the whole system around the model: persistent memory, specialist models, planning, tools, feedback, an executive layer and enough connections into the world to do something useful.
AGI (Artificial General Intelligence) refers to an AI system capable of performing a broad range of cognitive tasks, adapting across domains, learning, planning and acting with a level of general competence comparable to or beyond humans.
ASI (Artificial Superintelligence) goes much further: a generally intelligent system whose cognitive abilities substantially exceed the best humans across most or all important domains.
Scaffolding.
That changed my perspective. What has not changed is my scepticism about the next leap:
AGI arrives → improves itself → rapidly becomes ASI.
There is still a rather large ??? in that sequence.
I was looking at the engine, not the system
I use AI extensively for coding, planning and operating Amazon businesses. It lets me run multiple streams of work that would otherwise require far more people or far more time.
It also gives me a daily demonstration of the difference between impressive reasoning and reliable execution.
A model can explain a sophisticated campaign strategy, then apply it backwards. It can understand an entire software plan, then complete half of the first task and ask whether it should continue. A new session can spend ages reconstructing something another session understood perfectly yesterday.
Those experiences originally reinforced my view that we were missing something fundamental.
But many of those weaknesses are also systems problems.
Imagine a persistent coordinator that knows the project, delegates work to specialist sessions, keeps track of decisions and dependencies, and commissions independent reviews. Workers receive relevant context rather than rediscovering everything. Important constraints are enforced by software rather than merely mentioned in a prompt.
That is what I mean by scaffolding: not another elaborate system prompt, but actual infrastructure around the models. The International AI Safety Report uses the term similarly, describing software that enables models to plan, remember, pursue goals and interact with their environment.
International AI Safety Report 2026
My original test for meaningful initiative was deliberately personal: could a system notice that my birthday was approaching, remember that I like Alien, and decide to write me a novel as a surprise?
Initially, I dismissed that as something you could only fake with automation.
Then I stepped back.
Suppose a scheduled process simply asks the system to review what matters. It notices the birthday, considers possible gifts, chooses the novel, plans it, delegates the writing, reviews the result and delivers it.
Nobody explicitly programmed “write Christian an Alien novel.” The scheduler only provided an opportunity to act.
At that point, objecting that “a cron job started it” seems less interesting than asking what the system actually accomplished.
One birthday novel would not prove AGI. But it illustrates the architectural point: the properties I was demanding need not all live inside one model.
I started calling this synthetic AGI. That is my shorthand, not a formal technical category. Eventually, if the system demonstrates sufficiently broad competence, adaptability and initiative, the adjective may become irrelevant.
Self-improvement is real. An intelligence explosion is a stronger claim.
For this discussion, AGI means broad, adaptable competence across cognitive tasks. ASI means something substantially stronger: general capabilities far beyond those of the best humans, not merely a system that completes more tasks simultaneously.
Could an LLM-based system improve itself?
Yes. There are already limited research demonstrations.
Sakana AI’s Darwin Gödel Machine modifies its own agent code and evaluates the results on coding tasks. Its reported improvements include better editing tools, patch validation and workflows. That is genuine system-level self-improvement, not evidence of general superintelligence.
Sakana AI: Darwin Gödel Machine
Google DeepMind’s AlphaEvolve provides another example. It combines language models, evolutionary search and automated evaluation to discover algorithms, including improvements to parts of Google’s computing and AI-training infrastructure.
These examples matter because the argument cannot sensibly be “LLMs never invent anything.” Nor can it be “an AI system cannot contribute to improving AI.”
The more defensible question is:
How much improvement can that process produce, how reliably, at what cost, and over what timescale?
An agent discovering a better tool is not the same achievement as a system repeatedly creating successors that are dramatically better at discovering further improvements.
The first gives us evidence for a useful feedback loop. It does not, by itself, establish where that loop ends.
The missing step is successful research
The simplified intelligence-explosion story looks like this:
AGI designs a better AGI → that AGI designs an even better one → repeat, faster each time.
The word doing the most work is better.
The system does not initially produce a better intelligence. It produces a candidate improvement.
That candidate must survive implementation, evaluation and comparison with what already exists. Perhaps it works in a small experiment but fails at scale. Perhaps it improves one capability while damaging another. Perhaps the gain disappears when tested outside the conditions used to develop it.
Or perhaps i
t is an excellent idea that simply does not work.
Consider a hypothetical system that proposes 100 promising architectures. It might identify a breakthrough. It might also produce 100 expensive disappointments.
Both outcomes are compatible with a system that is exceptionally good at generating plausible research directions.
This is not a peculiar limitation of AI. It is the distinction between doing research and obtaining the result you hoped for.
A researcher can eliminate a promising approach without discovering a successful alternative. That is useful knowledge. It is not the solution.
The International AI Safety Report explicitly identifies uncertainty here: evidence on AI-assisted research automation is mixed, and evaluating whether an algorithmic change genuinely improves model performance can itself take substantial time.
International AI Safety Report 2026
That is the central issue for me:
Accelerating research does not guarantee a particular discovery on a particular schedule.
It may mean finding successful approaches faster. It may also mean discovering, much faster, that a long list of apparently promising approaches fails.
The result can be scientifically valuable without producing ASI.
The experiments still have an electricity bill
We also need to distinguish generating ideas from validating them.
A system might produce candidate improvements much faster than we can afford to test them. In that case, the immediate result is an experimental backlog, not an intelligence explosion.
The cost pressure is not imaginary. A study by Epoch AI and Stanford researchers, using models released through 2023, estimated that amortised training costs for its frontier-model sample had grown roughly 2.4 times annually since 2016. That is a historical estimate, not a rule that every new model must cost more than its predecessor.
The rising costs of training frontier AI models
But it illustrates why “the AI will train a better AI” cannot be treated as a costless operation.
There is compute to allocate, data to prepare, experiments to run, results to evaluate and failed approaches to discard. More research agents also consume inference resources. They may share infrastructure efficiently, but they are not free parallel universes.
Parallelism can shorten a queue by spending more resources. It does not abolish the resource requirement.
Nor does a modest improvement automatically create a runaway cycle. A 2% gain on a well-defined task tells us little about whether the next gain will be 2%, whether it will cost twice as much, or whether it will require ten failed attempts first.
And intelligence is not a single percentage anyway.
In my own coding workflows, I have sometimes found newer models less useful than older ones: stronger planning, perhaps, but worse completion, restraint or scope discipline. That is an observation about my work, not proof that model progress has stalled.
It does, however, make me wary of diagrams where every generation is represented as an uncomplicated upward arrow.
Better at what? Measured how? With which regressions? At what cost?
Those questions belong inside the self-improvement loop.
The real bottleneck may not be intelligence at all, but verification. An AGI could generate candidate improvements at extraordinary speed, but it still has to establish that they actually work. Verification can be cheap in closed domains with a clear reward signal – games, formal proofs, code with deterministic tests. But the improvements that matter most for recursive intelligence are open-ended: Is this genuinely a better research architecture, does it generalise, and what did it make worse? Those questions require runs, benchmarks and novel tasks – feedback from reality that cannot be shortcut. Recursive self-improvement is only a free lunch if verification is cheap – and the most consequential improvements may be precisely the hardest ones to verify.
A thousand Einsteins is not a discovery guarantee
There is an understandable temptation to replace the difficult part with scale.
Cannot solve it? Launch more agents.
That could help enormously. More agents can explore more approaches, divide work, challenge assumptions and test alternatives.
But “more research capacity” and “the required breakthrough will occur” are different propositions.
Some work can be divided. Other work depends on results that have not arrived yet. Some hypotheses need better evidence. Others turn out to be wrong.
A thousand brilliant researchers may achieve things none could accomplish alone. They still do not come with a contractual guarantee that the particular problem we care about will be solved.
The same applies to an AI research organisation.
It might generate extraordinary scientific progress. It might discover that a promising route has serious limitations. It might produce a succession of useful but increasingly expensive improvements.
None of those outcomes requires the system to be useless or unintelligent.
They simply do not amount to automatic research success.
Where my scepticism needs its own guardrails
There is an important counterargument, and I cannot dismiss it without undermining my own reasoning about AGI.
If intelligence can be a property of the whole system, then superintelligence could also be a property of the whole system.
I cannot accept that scaffolding might produce AGI and then declare that ASI must, by definition, require one individually superior model.
Perhaps better search, memory, verification, specialist components and coordination could collectively produce capabilities far beyond any constituent model. Fixed underlying weights do not prove a fixed ceiling on the larger system.
Also, not every improvement requires a vast new training run. AlphaEvolve’s reported work includes algorithmic changes that improved existing computing operations and reduced training time. Some useful feedback loops can operate through software changes and relatively fast evaluation.
So the argument is not that hardware guarantees a comfortable timetable, or that scaffolding can never produce ASI.
It is that neither outcome follows automatically.
We need to understand how much useful improvement remains available, how well it transfers between tasks, how reliably we can evaluate it, and whether the next round becomes easier or harder.
That is a research question. Naming the process “recursive self-improvement” does not answer it.
The two bridges should not be confused
I now see a plausible architectural route towards AGI:
Capable models + persistent memory + coordination + tools + feedback + learning + permissioned autonomy.
That is not a solved recipe. Listing components is easier than making them function reliably together. But it is a direction I can take seriously without requiring one language model to become an entire mind by itself.
The second bridge remains more uncertain:
AGI → sustained improvements in AI research capability → sufficiently large, broad and rapid gains → ASI.
To become convinced of rapid recursive progress, I would want to see repeated, independently validated improvements; not just promising proposals or a single successful optimisation. I would want the cost, elapsed time, failures and regressions included in the account.
That does not mean waiting for ASI before taking safety seriously. Uncertainty is a reason to test carefully and constrain authority, not permission to deploy recklessly.
But it is also not permission to turn a possible trajectory into an inevitable one.
I changed my mind about AGI because looking at the whole system answered an objection I had been placing entirely on the model.
I remain sceptical about an automatic, rapid transition to ASI because the next objection has not been answered merely by adding more agents.
Scaffolding may help us build a vastly better researcher.
It does not guarantee what that researcher will discover.
