Stripe, "Minions" · human-reviewed, no human-written code
…in a Ruby codebase
Plain Ruby, not Rails, with Sorbet · hundreds of millions
of lines · over $1 trillion a year
How do they trust that much?
The bottleneck moved
Producing code was the expensive part.
Review was the cheap thing you squeezed in
before standup.
Now producing is nearly free, and review is the
constraint.
This is not an AI talk
No agent demos. Nothing to sell you. An architecture talk that
AI made urgent.
The goal is to claim the leverage from the use of agents but
without any compromise on the quality of the software.
— Andrej Karpathy, February 2026
How do I keep my old quality bar at this new speed?
Spotify ranked their failure modes
1 No PR appears.A minor annoyance.
2 A PR that fails CI.Frustrating.
3 A PR that passes
CI and is functionally wrong.Erodes trust. Hides in
review. Breaks production.
Refactor billing and update
dependencies741 files changed+18,204 −9,613
AI
agent
I have no idea why I deleted half the codebase and
refactored the other half.
VT
reviewer
I have no idea what I'm looking at.
Anyway — merge.
Merge pull request+ All checks have passed
Confident code and correct code look identical
Excellent at the first 80% of a feature. Unreliable at the last
20% — edge cases, failure recovery, scaling assumptions.
No tone of voice.
What's good for humans is good for agents
Stripe, Part 1: "if it's good for humans, it's good for LLMs,
too."
Four properties
Gates — checks nobody can skip, fast
enough that nobody wants to
Budgets — performance as an assertion, not
an aspiration
Boundaries — so reviewing one change means
reading one thing
Blessed paths — so the default already
carries the standard
Property one
Gates
A check that runs every time and cannot be skipped. Not by an
agent, not by a contractor, not by me at five on a Friday.
Example 1 — the test that proves nothing
class DiscountCalculator
def initialize(order)
@order = order
end
def total
subtotal = @order.items.sum(&:price)
subtotal - (subtotal * rate)
end
private
def rate
return 0.20 if @order.customer.vip?
return 0.10 if @order.items.count > 5
0.0
end
end
Example 1 — the test that proves nothing
class DiscountCalculator
def initialize(order)
@order = order
end
def total
subtotal = @order.items.sum(&:price)
subtotal - (subtotal * rate)
end
private
def rate
return 0.20 if @order.customer.vip?
return 0.10 if @order.items.count > 5
0.0
end
end
The test it shipped with
it "applies the VIP discount" do
order = build_order(customer: vip, prices: [100])
expect(DiscountCalculator.new(order).total).to eq(80)
end
Green. 100% line coverage on that method.
Example 2 — the gate that catches it
$ bin/mutant-shipped
def rate
- if @order.customer.vip?+ if true
return 0.2
end
if @order.items.count > 5
Mutations: 61 Kills: 20Alive: 41 <- your suite never noticedCoverage: 32.78%
Example 3 — the verifier pattern
module Verify
Result = Struct.new(:ok, :summary, :details) do
def self.pass(summary) = new(true, summary, [])
def self.fail(details)
new(false, "#{details.size} problems", details)
end
def ok? = ok
end
end
One verifier
class Rubocop
# The verifier decides for itself whether it applies.
def applies?(root)
File.exist?(File.join(root, ".rubocop.yml"))
end
def call(root)
report = JSON.parse(`cd #{root} && rubocop --format json`)
offenses = report["files"].flat_map { |f| f["offenses"] }
return Result.pass("rubocop clean") if offenses.empty?
# The few lines that matter, not the whole report.
Result.fail(offenses.first(5).map { |o|
"#{o.dig('location', 'line')}: #{o['message']}"
})
end
end
One verifier
class Rubocop
# The verifier decides for itself whether it applies.
def applies?(root)
File.exist?(File.join(root, ".rubocop.yml"))
end
def call(root)
report = JSON.parse(`cd #{root} && rubocop --format json`)
offenses = report["files"].flat_map { |f| f["offenses"] }
return Result.pass("rubocop clean") if offenses.empty?
# The few lines that matter, not the whole report.
Result.fail(offenses.first(5).map { |o|
"#{o.dig('location', 'line')}: #{o['message']}"
})
end
end
#!/usr/bin/env ruby
# .git/hooks/pre-push (chmod +x)
changed = `git diff --name-only --diff-filter=d origin/main...HEAD`
.split("\n").grep(/\.rb\z/)
exit 0 if changed.empty?
# Fix what can be fixed, instead of only complaining.
system("rubocop --autocorrect-all #{changed.join(' ')}")
# Then fail on whatever is left.
exit 1 unless system("rubocop #{changed.join(' ')}")
Example 4 — move the gate earlier
#!/usr/bin/env ruby
# .git/hooks/pre-push (chmod +x)
changed = `git diff --name-only --diff-filter=d origin/main...HEAD`
.split("\n").grep(/\.rb\z/)
exit 0 if changed.empty?
# Fix what can be fixed, instead of only complaining.
system("rubocop --autocorrect-all #{changed.join(' ')}")
# Then fail on whatever is left.
exit 1 unless system("rubocop #{changed.join(' ')}")
Example 5 — types at the boundary
# sig/pricing/quote.rbs
module Pricing
class Quote
def initialize: (order: Order, currency: String) -> void
def total_cents: () -> Integer
def breakdown: () -> Array[LineItem]
end
end
What the signature catches
$ bundle exec rspec spec/boundaries
x Pricing::Quote matches its signature signature drift: Pricing::Quote#total_cents:
declared in the signature, missing from the class Pricing::Quote#initialize:
signature says keywords: order, class takes keywords: order_id
Gates — recap
Line coverage says what ran. Mutation coverage says what's
verified.
Scope the expensive gates narrowly: money, auth,
permissions.
Let the check decide when it applies. Remove the option to
skip.
Short failure output, or the gate gets disabled.
Fix before you complain. Run before CI.
Property two
Generated code is good at correct and bad at
cheap
Example 6 — the invisible cost
class ReportsController < ApplicationController
def monthly
orders = Order.where(created_at: 1.month.ago..)
@rows = orders.map do |order|
{
id: order.id,
customer: order.customer.name,
country: order.customer.address.country,
total: order.items.sum(&:price)
}
end
end
end
Example 6 — the invisible cost
class ReportsController < ApplicationController
def monthly
orders = Order.where(created_at: 1.month.ago..)
@rows = orders.map do |order|
{
id: order.id,
customer: order.customer.name,
country: order.customer.address.country,
total: order.items.sum(&:price)
}
end
end
end
Example 7 — make the cost assertable
RSpec::Matchers.define :stay_under_query_budget do |max|
match do |block|
@count = 0
sub = ActiveSupport::Notifications
.subscribe('sql.active_record') do |*, payload|
@count += 1 unless payload[:name] =~ /SCHEMA|TRANSACTION/
end
block.call
ActiveSupport::Notifications.unsubscribe(sub)
@count <= max
end
failure_message do
"expected at most #{max} queries, observed #{@count}"
end
supports_block_expectations
end
Example 7 — make the cost assertable
RSpec::Matchers.define :stay_under_query_budget do |max|
match do |block|
@count = 0
sub = ActiveSupport::Notifications
.subscribe('sql.active_record') do |*, payload|
@count += 1 unless payload[:name] =~ /SCHEMA|TRANSACTION/
end
block.call
ActiveSupport::Notifications.unsubscribe(sub)
@count <= max
end
failure_message do
"expected at most #{max} queries, observed #{@count}"
end
supports_block_expectations
end
Using it
it "renders the monthly report within budget" do
create_list(:order, 25, :with_items)
expect { get :monthly }.to stay_under_query_budget(6)
end
prosopite or bullet — find
the N+1 and name the line that raised it
the budget — stops the number creeping back
afterwards
brakeman — the same idea for security, and
it reads the app rather than running it
All three are gates. None of them is a
code review.
Example 8 — the cost nobody sees until 3am
$ kubectl logs -f web-7d4f
[1] ! Terminating timed out worker: 12
[1] - Worker 0 (PID: 12) booted in 61.4s
[1] ! Terminating timed out worker: 19
[1] - Worker 1 (PID: 19) booted in 63.1s[1] ! Worker 0 (PID: 26) failed to boot within 60 seconds
One require. Eight more libraries.
$ bin/boot-profile $ bin/boot-profile
boot: 0.007s boot: 0.057s
libraries at boot: 2 libraries at boot: 10
digest x csv x date
json x date_core x English
x forwardable x stringio
x strscan x time
as it stands after one require "csv"
The gate: a manifest, not a stopwatch
# spec/boot_budget_spec.rb
it "loads exactly the libraries the manifest declares" do
report = BootReport.call
expect(report.libraries).to eq(manifest),
"boot loaded libraries nothing declared: " \
"#{(report.libraries - manifest).join(', ')}"
end
Then fix the boot
require: false, then require at the point of
use
Never dial out at load time — memoize the client, let the
first request pay
preload_app!, and reopen sockets in
on_worker_boot
Then set worker_boot_timeout to
something honest
Example 9 — the budget that doesn't flake
it "serializes without allocating wildly" do
order = build(:order, items_count: 50)
report = MemoryProfiler.report do
OrderSerializer.new(order).to_json
end
expect(report.total_allocated).to be < 12_000
end
When a budget fails, profile
A budget tells you something got worse. It doesn't tell you
why. vernier rather than stackprof these days — it
understands threads, and will show you a Ractor properly.
Property three
How much do I need to know to say yes?
Example 10 — shape one
class SubscriptionsController < ApplicationController
def upgrade
sub = current_user.subscription
if sub.trial? && params[:plan] == 'annual'
sub.update!(plan: 'annual', trial_ends_at: nil)
Billing.charge(current_user, AnnualPlan::PRICE)
current_user.update!(onboarded: true)
Slack.notify("#{current_user.email} upgraded")
UpgradeMailer.confirm(current_user).deliver_later
elsif sub.active?
sub.update!(plan: params[:plan])
Billing.prorate(current_user, params[:plan])
else
return redirect_to billing_path, alert: 'Cannot upgrade'
end
redirect_to account_path, notice: 'Upgraded'
end
end
Shape two — same behaviour
class Subscriptions::Upgrade
Result = Struct.new(:ok, :subscription, :error)
def initialize(subscription:, plan:, clock: Time)
@subscription = subscription
@plan = plan
@clock = clock
end
def call
return failure(:not_upgradable) unless upgradable?
ActiveRecord::Base.transaction do
@subscription.update!(next_attributes)
Billing.charge(@subscription, price)
end
success(@subscription)
rescue Billing::Declined => e
failure(:declined, e.message)
end
end
Example 11 — did it stay where it said?
# spec/architecture/scope_spec.rb
it "keeps pricing changes inside pricing" do
touched = `git diff --name-only origin/main...HEAD`.split("\n")
allowed = %r{\A(app/services/pricing/|spec/services/pricing/)}
strays = touched.reject { |path| path.match?(allowed) }
expect(strays).to be_empty,
"pricing-only change also touched:\n#{strays.join("\n")}"
end
$ bin/verify --audience
+ gates nobody can skip them 8s+ budgets asserted, not hoped for 6s+ boundaries one change, one thing 4s+ blessed paths the default carries it 2sx questions audience#curiosity ∞
expected at most 0 questions, observed
Q&A
Scope check disabled for the next five
minutes. Ask me anything — strays are welcome.👀