Designing an AI-Native Delivery Process
Seven posts in, the pieces are: code generation getting cheaper shifts the bottleneck to validation. AI fails in a specific, confident-looking way that doesn't match what most review processes were built to catch — and it's not just the code, developers themselves are bad at judging whether AI is actually speeding them up. A test suite alone isn't a validation system. CI/CD ends up carrying more of the trust-building work that used to happen in a person's head. Observability picks up the failures that were never anticipated enough to test for. A repo that's legible to an agent produces fewer wrong guesses in the first place. And AI can build most of its own guardrails, as long as something deterministic still enforces them.
Put together, that's not a checklist. It's closer to an operating model — what a delivery process actually looks like when a meaningful fraction of the changes flowing through it were AI-generated.
A Rough Shape
I don't think there's one correct version of this, but a reasonable shape looks something like:
- A human defines the desired outcome, the constraints, and what "done" actually means — acceptance criteria, not vibes.
- AI proposes an implementation plan against that.
- AI implements a small change. Small, specifically — see the CI/CD post for why size matters more than it used to.
- Automated systems validate syntax, behavior, compatibility, security, and performance, without a human in the loop for any of it.
- A human reviews architectural intent and risk — not line-by-line correctness, which the automated layer already covers, but whether this is the right thing to be building at all.
- The change lands in a preview or ephemeral environment.
- Automated and human acceptance checks run against it there.
- The feature ships progressively, behind a flag.
- Production telemetry validates the outcome against what step 1 actually said "done" meant.
- The rollout stops or reverses automatically when the signals from step 9 say it should, without waiting for someone to notice.
The details of any one step are what the rest of this series was about. The point of laying it out end to end is that it's a loop, not a pipeline with a start and an end — step 9 feeds back into step 1 for the next change, and a lot of teams are currently missing that feedback path entirely. They've adopted the AI at steps 2 and 3 and left the rest of the loop exactly as it was.
Measuring the Loop
Step 9 is where this whole thing either works or turns into theater, and it depends entirely on what you're actually measuring. Lines of code generated, suggestions accepted, PRs opened — those are activity metrics. They tell you the loop is spinning. They don't tell you it's working.
DORA's own framework grew a fifth metric in 2025 for basically this reason: deployment rework rate, alongside the familiar four — deployment frequency, lead time, change failure rate, recovery time. It exists because change failure rate alone wasn't catching everything; a team could ship fast and "successfully" while quietly generating rework nobody was counting. Their 2025 report on AI-assisted teams found close to what you'd predict from everything earlier in this series: AI boosted individual output, and at the same time, change failure rate and rework rate got measurably worse at the team level — for teams whose delivery system wasn't ready to absorb the extra volume. Same finding as the very first post in this series, from a completely different source. The difference was never the AI. It's what's underneath it.
One metric worth tracking that DORA doesn't: the percentage of AI-generated code that gets substantially rewritten before or after it ships. Not "accepted" — actually still there, unchanged, a month later. If that number's high, the loop isn't validating anything. It's just relabeling human rewrite work as AI-assisted.
The Actual Bet
I think the teams that end up ahead here won't be the ones with the best coding agents. Coding agents are converging fast enough that the model you're using this quarter is a temporary advantage at best. The teams that end up ahead will be the ones with the best change-validation and delivery systems wrapped around whatever agent they're using, because that's the part that compounds, and it's the part that doesn't get automatically obsoleted by the next model release.
Which is really just the thesis this whole series has been circling: AI can give a team extraordinary feature velocity. Whether that velocity turns into shipped, reliable software instead of an expensive mess depends entirely on whether DevOps, QA, observability, and automation can keep up with the volume of change AI makes possible. The AI was never the hard part. It was always going to be everything downstream of it.
Seven posts in, the pieces are: code generation getting cheaper shifts the bottleneck to validation. AI fails in a specific, confident-looking way that doesn't match what most review processes were built to catch — and it's not just the code, developers themselves are bad at judging whether AI is actually speeding them up. A test suite alone isn't a validation system. CI/CD ends up carrying more of the trust-building work that used to happen in a person's head. Observability picks up the failures that were never anticipated enough to test for. A repo that's legible to an agent produces fewer wrong guesses in the first place. And AI can build most of its own guardrails, as long as something deterministic still enforces them.
Put together, that's not a checklist. It's closer to an operating model — what a delivery process actually looks like when a meaningful fraction of the changes flowing through it were AI-generated.
A Rough Shape
I don't think there's one correct version of this, but a reasonable shape looks something like:
- A human defines the desired outcome, the constraints, and what "done" actually means — acceptance criteria, not vibes.
- AI proposes an implementation plan against that.
- AI implements a small change. Small, specifically — see the CI/CD post for why size matters more than it used to.
- Automated systems validate syntax, behavior, compatibility, security, and performance, without a human in the loop for any of it.
- A human reviews architectural intent and risk — not line-by-line correctness, which the automated layer already covers, but whether this is the right thing to be building at all.
- The change lands in a preview or ephemeral environment.
- Automated and human acceptance checks run against it there.
- The feature ships progressively, behind a flag.
- Production telemetry validates the outcome against what step 1 actually said "done" meant.
- The rollout stops or reverses automatically when the signals from step 9 say it should, without waiting for someone to notice.
The details of any one step are what the rest of this series was about. The point of laying it out end to end is that it's a loop, not a pipeline with a start and an end — step 9 feeds back into step 1 for the next change, and a lot of teams are currently missing that feedback path entirely. They've adopted the AI at steps 2 and 3 and left the rest of the loop exactly as it was.
Measuring the Loop
Step 9 is where this whole thing either works or turns into theater, and it depends entirely on what you're actually measuring. Lines of code generated, suggestions accepted, PRs opened — those are activity metrics. They tell you the loop is spinning. They don't tell you it's working.
DORA's own framework grew a fifth metric in 2025 for basically this reason: deployment rework rate, alongside the familiar four — deployment frequency, lead time, change failure rate, recovery time. It exists because change failure rate alone wasn't catching everything; a team could ship fast and "successfully" while quietly generating rework nobody was counting. Their 2025 report on AI-assisted teams found close to what you'd predict from everything earlier in this series: AI boosted individual output, and at the same time, change failure rate and rework rate got measurably worse at the team level — for teams whose delivery system wasn't ready to absorb the extra volume. Same finding as the very first post in this series, from a completely different source. The difference was never the AI. It's what's underneath it.
One metric worth tracking that DORA doesn't: the percentage of AI-generated code that gets substantially rewritten before or after it ships. Not "accepted" — actually still there, unchanged, a month later. If that number's high, the loop isn't validating anything. It's just relabeling human rewrite work as AI-assisted.
The Actual Bet
I think the teams that end up ahead here won't be the ones with the best coding agents. Coding agents are converging fast enough that the model you're using this quarter is a temporary advantage at best. The teams that end up ahead will be the ones with the best change-validation and delivery systems wrapped around whatever agent they're using, because that's the part that compounds, and it's the part that doesn't get automatically obsoleted by the next model release.
Which is really just the thesis this whole series has been circling: AI can give a team extraordinary feature velocity. Whether that velocity turns into shipped, reliable software instead of an expensive mess depends entirely on whether DevOps, QA, observability, and automation can keep up with the volume of change AI makes possible. The AI was never the hard part. It was always going to be everything downstream of it.
Six posts in, the pieces are: code generation getting cheaper shifts the bottleneck to validation. AI fails in a specific, confident-looking way that doesn't match what most review processes were built to catch. A test suite alone isn't a validation system. CI/CD ends up carrying more of the trust-building work that used to happen in a person's head. Observability picks up the failures that were never anticipated enough to test for. And AI can build most of its own guardrails, as long as something deterministic still enforces them.
Put together, that's not a checklist. It's closer to an operating model — what a delivery process actually looks like when a meaningful fraction of the changes flowing through it were AI-generated.
A Rough Shape
I don't think there's one correct version of this, but a reasonable shape looks something like:
- A human defines the desired outcome, the constraints, and what "done" actually means — acceptance criteria, not vibes.
- AI proposes an implementation plan against that.
- AI implements a small change. Small, specifically — see the CI/CD post for why size matters more than it used to.
- Automated systems validate syntax, behavior, compatibility, security, and performance, without a human in the loop for any of it.
- A human reviews architectural intent and risk — not line-by-line correctness, which the automated layer already covers, but whether this is the right thing to be building at all.
- The change lands in a preview or ephemeral environment.
- Automated and human acceptance checks run against it there.
- The feature ships progressively, behind a flag.
- Production telemetry validates the outcome against what step 1 actually said "done" meant.
- The rollout stops or reverses automatically when the signals from step 9 say it should, without waiting for someone to notice.
The details of any one step are what the rest of this series was about. The point of laying it out end to end is that it's a loop, not a pipeline with a start and an end — step 9 feeds back into step 1 for the next change, and a lot of teams are currently missing that feedback path entirely. They've adopted the AI at steps 2 and 3 and left the rest of the loop exactly as it was.
The Actual Bet
I think the teams that end up ahead here won't be the ones with the best coding agents. Coding agents are converging fast enough that the model you're using this quarter is a temporary advantage at best. The teams that end up ahead will be the ones with the best change-validation and delivery systems wrapped around whatever agent they're using, because that's the part that compounds, and it's the part that doesn't get automatically obsoleted by the next model release.
Which is really just the thesis this whole series has been circling: AI can give a team extraordinary feature velocity. Whether that velocity turns into shipped, reliable software instead of an expensive mess depends entirely on whether DevOps, QA, observability, and automation can keep up with the volume of change AI makes possible. The AI was never the hard part. It was always going to be everything downstream of it.
Seven posts in, the pieces are: code generation getting cheaper shifts the bottleneck to validation. AI fails in a specific, confident-looking way that doesn’t match what most review processes were built to catch — and it’s not just the code, developers themselves are bad at judging whether AI is actually speeding them up. A test suite alone isn’t a validation system. CI/CD ends up carrying more of the trust-building work that used to happen in a person’s head. Observability picks up the failures that were never anticipated enough to test for. A repo that’s legible to an agent produces fewer wrong guesses in the first place. And AI can build most of its own guardrails, as long as something deterministic still enforces them.
Put together, that’s not a checklist. It’s closer to an operating model — what a delivery process actually looks like when a meaningful fraction of the changes flowing through it were AI-generated.
A Rough Shape
I don’t think there’s one correct version of this, but a reasonable shape looks something like:
- A human defines the desired outcome, the constraints, and what “done” actually means — acceptance criteria, not vibes.
- AI proposes an implementation plan against that.
- AI implements a small change. Small, specifically — see the CI/CD post for why size matters more than it used to.
- Automated systems validate syntax, behavior, compatibility, security, and performance, without a human in the loop for any of it.
- A human reviews architectural intent and risk — not line-by-line correctness, which the automated layer already covers, but whether this is the right thing to be building at all.
- The change lands in a preview or ephemeral environment.
- Automated and human acceptance checks run against it there.
- The feature ships progressively, behind a flag.
- Production telemetry validates the outcome against what step 1 actually said “done” meant.
- The rollout stops or reverses automatically when the signals from step 9 say it should, without waiting for someone to notice.
The details of any one step are what the rest of this series was about. The point of laying it out end to end is that it’s a loop, not a pipeline with a start and an end — step 9 feeds back into step 1 for the next change, and a lot of teams are currently missing that feedback path entirely. They’ve adopted the AI at steps 2 and 3 and left the rest of the loop exactly as it was.
Measuring the Loop
Step 9 is where this whole thing either works or turns into theater, and it depends entirely on what you’re actually measuring. Lines of code generated, suggestions accepted, PRs opened — those are activity metrics. They tell you the loop is spinning. They don’t tell you it’s working.
DORA’s own framework grew a fifth metric in 2025 for basically this reason: deployment rework rate, alongside the familiar four — deployment frequency, lead time, change failure rate, recovery time. It exists because change failure rate alone wasn’t catching everything; a team could ship fast and “successfully” while quietly generating rework nobody was counting. Their 2025 report on AI-assisted teams found close to what you’d predict from everything earlier in this series: AI boosted individual output, and at the same time, change failure rate and rework rate got measurably worse at the team level — for teams whose delivery system wasn’t ready to absorb the extra volume. Same finding as the very first post in this series, from a completely different source. The difference was never the AI. It’s what’s underneath it.
One metric worth tracking that DORA doesn’t: the percentage of AI-generated code that gets substantially rewritten before or after it ships. Not “accepted” — actually still there, unchanged, a month later. If that number’s high, the loop isn’t validating anything. It’s just relabeling human rewrite work as AI-assisted.
The Actual Bet
I think the teams that end up ahead here won’t be the ones with the best coding agents. Coding agents are converging fast enough that the model you’re using this quarter is a temporary advantage at best. The teams that end up ahead will be the ones with the best change-validation and delivery systems wrapped around whatever agent they’re using, because that’s the part that compounds, and it’s the part that doesn’t get automatically obsoleted by the next model release.
Which is really just the thesis this whole series has been circling: AI can give a team extraordinary feature velocity. Whether that velocity turns into shipped, reliable software instead of an expensive mess depends entirely on whether DevOps, QA, observability, and automation can keep up with the volume of change AI makes possible. The AI was never the hard part. It was always going to be everything downstream of it.