The Coffee
Experiment

Why I built this

My pricing class taught me how supply-and-demand curves set prices for physical commodities. I wanted to point those methods at software.

In the microeconomics class of my INSEAD Executive MBA, we worked through the classics — demand curves, elasticity, the profit-maximising price. But every example was a physical thing: oil, aluminium, Apple Watches, physical units shipped. I’m a PM. I wanted to do two things the coursework didn’t: take those methods to a software product, and test out A/B testing — a validation toolkit I’d thought about for years and never actually run.

So I made up an app to do it on. “Cafezinho” is a fictional app, which is exactly the point: with an invented product I can test various scenarios and extend the example with data I create, while basing it in reality.

Fictional app, illustrative numbers, real method. I’m working in the open — every figure here is one I’d defend or revise out loud. — Sally

One price isn't an answer. It's a point on a curve.

Cafezinho sells a monthly subscription. Charge too little and they leave money in the tip jar; charge too much and customers walk. Somewhere between free and absurd sits a price that does the most work. Part 1 finds it. Part 2 asks the harder question: once you have a candidate price, how do you prove it's better (without fooling yourself by developing contrived tests)?

One founder's question, answered in three moves

The founder wants to jump the price from R$14.9 to R$24.9. Before I touch a slider or run a test, I work through three questions in order — price first, then whether a premium tier earns its build, then whether any of it can be proven with an experiment. Here's the path, and where each move lives on the page.

We're setting the price of Cafezinho's paid tier — the Recipe Suggester — and optimising for one thing: monthly profit. Drag the price. Watch four business metrics move at once, and notice they don't agree on where the peak is.

Demand model is illustrative — linear demand, marginal cost R$2.5/sub.

Four metrics. Four different "best" prices.

Each metric peaks somewhere else on the same curve. Picking which one to chase is a leadership call — not a math output.

Elasticity: how hard does demand push back?

Below the unit-elastic line (E = 1), a price rise grows revenue. Above it, customers flee faster than price climbs. The profit peak always lands in the elastic region — to the right of E = 1.

Step 2: don't pick one price. Build a menu.

Once you've set the single best price (the baseline), layer a tiered menu on top of it. Different drinkers will pay different amounts; the menu lets willingness-to-pay self-select. The thinking is — you design the ladder, customers sort themselves.

Discount the elastic. Hold the line on the inelastic.

Not every drinker reacts the same way. The rule of thumb: cut price where demand is springy, keep it where demand is sticky — always measured at the baseline you just set.

To A/B test this credibly, aim for ~7× the traffic

An honest test of the lift we care about needs more visitors than Cafezinho has today.

Once Cafezinho reaches ~1,059/arm, this exact test becomes trustworthy: 80% power, and a tight confidence interval instead of the noisy one you'd get today. How it grows to that traffic — marketing, growth, funnel work — is a separate plan, out of scope here.

The peeking trap

To expose the trap honestly we run an A/A test — two groups that get the exact same offer (same price, same everything). The true difference between them is zero by construction, so any "significant" result is a false alarm. This matters because peeking — stopping the moment a result looks good — is how you convince yourself a phantom win is real. We could easily ship a change that does nothing (economically), peek at the results too early, and end up burning engineering time and momentum chasing further improvements, only to find that the results from the full test contained different indications. Let's test it out — press run, watch the days tick by, and see how often peeking fools you.

We start with an A/A, not A/B: in an A/B test, group B is a real change. Here both groups are identical — so we know the honest answer is "no difference", and that the instrumentation and test mechanism are honest and will lead to a valuable A/B.

And check the split before you trust anything

Before you read a result, check the split it's built on.

What each testing step bought us

Five checks, in order — each one either earns the right to run the test or tells us to stop. Here's the ledger: what I assessed, and what it told me for Cafezinho.

So can Cafezinho just A/B its way to the price?

Not at today's size. Power says the test is blind until traffic grows ~7×, and peeking before then would manufacture false wins. The honest playbook: start from the modelled price (Part 1), roll it out to new sign-ups rather than secretly charging identical users two prices, pre-register what you'll measure, clear the SRM gate, and watch the guardrails — cancellations, refunds, lifetime value — over time. An experiment supports the decision; it rarely makes it for you.