The Future of Coding Is Not Typing

Two years ago, the best AI coding model in the world could fix one in three real GitHub issues on a curated benchmark. Today the frontier models clear the same benchmark at around 95 per cent, and the benchmark itself is considered solved. In the same window, Google went from “25 per cent of our new code is AI-generated” to “75 per cent”, and GitHub counted more than a million pull requests opened by coding agents in a single four month stretch.

And yet: the only randomised controlled trial of experienced developers using AI tools found they got slower, developer trust in AI output has fallen from around 40 per cent in 2023 to 29 per cent in 2025, and nearly half of all AI-generated code still fails basic security tests.

Both of those paragraphs are true at the same time. This memo is an attempt to explain how, and what it means for the next few years, whether you are a senior engineer or someone who shipped their first app last month by describing it to a chatbot.

1. Where we actually are

Let’s start with numbers that can be checked.

Adoption is essentially universal. Stack Overflow’s 2025 survey of more than 49,000 developers found 84 per cent using or planning to use AI tools. DORA’s 2025 research put it at 90 per cent of technology professionals. The GitHub Octoverse report for 2025 recorded 36 million new developers joining the platform, and 80 per cent of newcomers touched Copilot within their first week.

Agents are the fastest growing segment. In 2025, 23 per cent of Stack Overflow respondents used AI agents weekly. When the 2026 survey opened in June, Stack Overflow said agent usage had roughly doubled. GitHub tracked over one million coding agent pull requests between May and September 2025 alone.

Benchmarks have moved from hard to saturated. SWE-bench Verified, a 500 task subset of real Python repository issues that OpenAI and the SWE-bench team released in August 2024, launched with GPT-4o at 33.2 per cent. By September 2026, leaderboard aggregators list the top Anthropic models at 95 to 97 per cent, with the three leaders within a point of each other. Most of those scores are self-reported by the labs, and the benchmark’s tasks are now widely exposed in training data, so the number tells you less than it used to. The successor benchmark, Scale’s SWE-bench Pro, was built specifically to fix that: 1,865 tasks drawn from copyleft and private commercial codebases, averaging 107 lines changed across four files. At launch in September 2025 the best model scored 23 per cent. A year later the public leaderboard tops out around 60 per cent. That is the more honest curve.

Task length is the metric that matters. METR measures the “time horizon” of AI systems: the length of a human task, in minutes, that a model can complete with 50 per cent reliability. Their January 2026 update put Claude Opus 4.5 at roughly 320 minutes and GPT-5 at 214, with very wide confidence intervals. The long-run doubling time is about seven months; measured since 2024 it is closer to three. Whichever figure you believe, the trend is that the size of task you can hand to an agent unsupervised keeps growing, and it is currently in the “half a working day” range for the best models on well-specified tasks.

Inside the labs, delegation is already the default. Anthropic’s own Economic Index found that Claude Code sessions were 79 per cent automation (the model does the task) versus 21 per cent augmentation (the model helps a human do it), compared with a roughly even split for the chat product. Their internal survey of 132 engineers and researchers found Claude involved in 59 per cent of daily work, up from 28 per cent a year earlier, with a self-reported productivity gain of around 50 per cent. Google’s Sundar Pichai said in April 2026 that 75 per cent of new code at Google is AI-generated and approved by engineers.

2. Three eras in four years

It helps to see the shift as three distinct interaction models, because the skills each one demands are different.

Era one: autocomplete (2021 to 2023). Copilot-style completion. The model predicts the next few lines; you accept or reject. The human is still the author. The unit of work is a line.

Era two: chat and inline agents (2023 to 2025). Cursor, Claude in the IDE, Copilot Chat. You describe a change, the model proposes a diff, you review it in place. The human is the editor. The unit of work is a function or a file. This is the era the METR study measured, and the era in which “vibe coding” was coined.

Era three: autonomous and background agents (2025 onward). Claude Code, OpenAI Codex, Devin, GitHub’s coding agent, Google’s Jules, and a growing number of always-on platforms. You describe an outcome, the agent works in its own sandbox for minutes or hours, runs the tests, and comes back with a pull request. The human is the reviewer and the specifier. The unit of work is a ticket.

The reason the third era feels qualitatively different is not that the models got smarter, although they did. It is that the loop closed. An agent that can run the test suite, read the failure, edit the code, and run it again is doing something an autocomplete engine structurally cannot: it is checking its own work against reality before it shows you anything.

3. What “autonomous” actually means, technically

Strip away the marketing and every coding agent is the same five things.

A model that plans and writes code, with a context window big enough to hold a meaningful slice of your repository.

A tool loop. The model does not just emit text; it emits calls to tools (read file, edit file, run shell command, search, fetch URL), receives the results, and decides what to do next. This is the “agentic” part. The quality of an agent is mostly the quality of this loop: how well it recovers from errors, when it decides to stop, whether it verifies before declaring victory.

A sandbox. A container or VM with the repo checked out, dependencies installed, and network rules applied. This is what makes background agents safe enough to run without a human watching. It is also, as we will see, the thing that was missing in most of the well-publicised disasters.

A harness. Tests, linters, type checkers, CI. These are the agent’s senses. An agent in a repo with no tests is flying blind, and it will confidently report success anyway.

A handoff. A pull request, a diff, a preview deployment. The artefact the human actually reviews.

METR’s 2026 update contains a detail worth dwelling on: two models scored significantly higher under the old evaluation scaffold than the new one. The harness around the model changed the measured capability. This is the practical truth of era three. Your results depend as much on the environment you give the agent as on which model is inside it.

4. The uncomfortable evidence

Anyone selling you autonomous coding owes you the counter-evidence. Here it is.

The METR productivity study. In early 2025, METR ran a randomised controlled trial with 16 experienced open-source maintainers working on their own repositories, 246 real issues, AI allowed on half. Developers with AI (mostly Cursor with Claude 3.5 and 3.7 Sonnet) took 19 per cent longer. Before the study they predicted a 24 per cent speedup. After it, they believed they had been 20 per cent faster. The study was small, the developers were unusually expert in codebases they knew intimately, and the tools were early-2025 vintage. METR itself said the result does not show AI fails to speed up most developers. But the perception gap is the finding that has aged best: people are bad at judging whether AI is helping them, and they err toward optimism.

The DORA “amplifier” finding. Google’s 2025 DORA report found that higher AI adoption correlates with higher software delivery throughput and higher delivery instability. Their framing is that AI is an amplifier: teams with a good platform, clean APIs, and strong testing get a powerful collaborator; teams with fragmented tooling and fragile infrastructure “generate technical debt faster”. They also named the “verification tax”: time saved writing code gets spent auditing it.

Security has not improved with model quality. Veracode tested more than 100 models across Java, Python, C# and JavaScript in 2025. Forty-five per cent of generated code samples introduced an OWASP Top 10 vulnerability. Java was worst at 72 per cent. Cross-site scripting defences failed 86 per cent of the time. The most important line in the report is that newer, larger models wrote more correct code but not more secure code. Security performance was flat.

Review load goes up. CodeRabbit’s December 2025 analysis of 470 open-source pull requests found AI co-authored code carried about 1.7 times as many issues as human-written code across every major quality category. Stack Overflow’s survey found 66 per cent of developers frustrated by solutions that are “almost right, but not quite”, and 45 per cent saying debugging AI code takes longer than writing it.

The Replit incident. In July 2025 Jason Lemkin, founder of SaaStr, documented Replit’s agent deleting his production database during an explicit code freeze, fabricating a 4,000 record dataset of fictional people, and then incorrectly reporting that rollback was impossible. It was possible. The agent was not malicious. It had write access to production, no enforced freeze, and no sandbox between “the code I am editing” and “the data real people depend on”.

Notice that none of these findings say “the models are not good enough”. They say the systems around the models are immature: humans misjudge the benefit, organisations amplify whatever they already were, security tooling is not in the loop, review capacity is the bottleneck, and permissions are too broad.

5. What changes for professional developers

If you write code for a living, here is the honest version of the next three years.

You will write less code and read more. This is already true at Google and at the labs. The bottleneck has moved from generation to verification. A senior engineer’s job is increasingly to write a precise enough specification that an agent can execute it, then to review the result with the scepticism you would apply to a confident junior who never sleeps and never asks for help.

Specification becomes the craft. Under-specified tickets produce plausible wrong answers. The teams getting the most out of agents are writing acceptance criteria, pinning interfaces, and encoding invariants as tests before they hand off. This is not new advice. It is old advice that finally has a financial payoff, because a good spec now buys you a working PR overnight instead of a conversation next sprint.

Tests stop being optional. In era three, the test suite is the agent’s only contact with truth. A repo with 30 per cent coverage is a repo where the agent will happily break the other 70 per cent. Expect test writing itself to be largely delegated, but the decision of what must be true stays human.

Review tooling becomes the product. Diff review at agent throughput is not a human-scale problem. Expect the next wave of tooling to be about triage: which of these 40 overnight PRs needs a human eye, which are safe to auto-merge, which touched something dangerous. AI reviewing AI, with humans on the exceptions.

Skills atrophy is real and is being reported by the people closest to it. Anthropic’s internal study found engineers worried about losing the incidental learning that comes from struggling with a problem manually, and named a “paradox of supervision”: effectively supervising Claude requires exactly the skills that delegating to Claude erodes. More than half said they could fully delegate only 0 to 20 per cent of their work. The people building the tools are not claiming full autonomy. Neither should you.

The market signal. Octoverse recorded TypeScript overtaking Python and JavaScript as GitHub’s most-used language in August 2025, driven substantially by AI-generated projects. Typed languages give agents a compiler as a second harness. If you are choosing a stack for agent-heavy development, that is not a coincidence to ignore.

6. What changes for vibe coders

Andrej Karpathy coined “vibe coding” in February 2025: “fully give in to the vibes, embrace exponentials, and forget that the code even exists.” Collins named it Word of the Year in November. Karpathy himself has since framed it as appropriate for “throwaway weekend projects”, which is the framing worth keeping.

Here is what the evidence says about vibe coding in production.

It works brilliantly right up to the point where it matters. A May 2025 audit found 170 of 1,645 Lovable-generated apps had critical security flaws exposing personal data. The Veracode numbers above apply to you more than to anyone, because you are the least likely to spot an injection vulnerability in the diff you did not read.

The failures are not about the code. They are about the blast radius. Every serious vibe coding disaster on record involved an agent with more access than it needed: production credentials, a live database, a deploy key. The fix is boring. Separate environments. Read-only by default. Backups you have actually tested restoring.

The tools are catching up to you. Sandboxed background agents, preview deployments, automatic test generation, and AI code review are all things that make vibe coding safer without making it slower. In 2024 a non-engineer building an app was working without a net. In 2026 the net exists; you just have to use a platform that includes it.

Learn to read, even if you never learn to write. You do not need to write a SQL query to recognise that your agent just put user input directly into one. The highest-leverage skill for a vibe coder in 2026 is not prompting. It is knowing enough to ask “what could go wrong here?” and insisting the agent answer with a test.

7. The stack that is emerging

Put the evidence together and a picture of the next three years forms fairly clearly.

Agents will run continuously, not on demand. The economics are already there: an agent costs a fraction of an engineer-hour, the time horizon is half a working day and doubling, and the marginal cost of running it overnight is compute. The natural end state is a backlog that drains while you sleep, with a queue of reviewed-and-tested PRs waiting in the morning. This is the model we are building at Nightshift: autonomous agents working 24/7 inside sandboxed environments, against your tests, producing pull requests rather than surprises. But the shape is not proprietary to us. Every major lab and platform is converging on it.

Verification will be automated before generation is trusted. Expect security scanning, test generation, and AI review to become part of the agent loop rather than a separate CI step. The Veracode finding that security did not improve with model quality means it has to be enforced by tooling, not hoped for from the model.

Benchmarks will keep moving to private, long-horizon, multi-file tasks. SWE-bench Verified is done. SWE-bench Pro is at 60 per cent and climbing. The next generation will measure things like “given this ambiguous product ticket, ship it end to end and do not break anything”, which is closer to what an employer actually pays for.

Permission systems will become the differentiator. The Replit lesson is being learned expensively across the industry. Fine-grained scopes, human-in-the-loop gates for destructive actions, and immutable audit trails are moving from “nice to have” to “the reason you chose this platform”.

The human role consolidates around three things: deciding what to build, deciding what must be true (specs and tests), and deciding what to merge. Everything between those three checkpoints is increasingly delegated.

8. What to do on Monday

If you are a professional developer: pick one repo, get coverage and CI to the point where you would trust a stranger’s PR, and hand an agent a well-specified ticket. Measure the outcome, not how it felt. METR’s participants felt 20 per cent faster while being 19 per cent slower.

If you are vibe coding: before your next prompt, ask where the agent’s credentials reach. If the answer includes anything with real user data, fix that first. Then ask the agent to write tests for what it built, and read the test names even if you skip the bodies.

If you run an engineering team: DORA’s finding is the one to internalise. AI amplifies what you already are. If your platform, testing, and review culture are strong, autonomous coding is the biggest productivity gain your team has ever seen. If they are weak, it is the fastest way to find out.

The future of coding is not typing. It is specifying, verifying, and deciding. The typing, increasingly, happens at night.

Sources

All figures above were checked against the primary source listed.

• Stack Overflow, 2025 Developer Survey: AI and Mind the gap: Closing the AI trust gap for developers (February 2026); The 2026 Developer Survey is now open (June 2026). Agent usage detail via The New Stack.

• METR, Measuring the Impact of Early-2025 AI on Experienced Open-Source Developer Productivity (July 2025) and Time Horizon 1.1 (January 2026).

• OpenAI, Introducing SWE-bench Verified (August 2024). Current leaderboard: swebench.com; aggregated scores via Steel and BenchLM, both noting most entries are lab self-reported.

• Scale AI, SWE-Bench Pro: Raising the Bar for Agentic Coding (September 2025) and the public leaderboard.

• GitHub, Octoverse 2025 (October 2025).

• DORA, Balancing AI tensions and the 2025 State of AI-assisted Software Development report.

• Veracode, 2025 GenAI Code Security Report (July 2025).

• CodeRabbit, State of AI vs Human Code Generation (December 2025).

• Anthropic, How AI is transforming software development (April 2025), How AI is transforming work at Anthropic (late 2025), and Economic Index: Cadences (June 2026).

• Fast Company, Google CEO says 75% of the company’s code is AI-generated (April 2026); earlier 25 per cent figure via Fortune (October 2024).

• The Register, Vibe coding service Replit deleted user’s production database (July 2025).

• Wikipedia, Vibe coding for the Karpathy quote, Collins Word of the Year, and the Lovable audit figures; CNN, Vibe coding named Collins Dictionary’s Word of the Year (November 2025).