Chewy's customers skew older and deeply attached to their pets. When orders arrive late or not at all, it's personal. I noticed a pattern in CX call data: customers weren't just complaining about late deliveries. The root cause was wrong or ambiguous addresses entered at checkout.
I pulled the data and ran a batch validation against Melissa Data's address database. The problem was 3-4x bigger than direct complaints suggested. Thousands of addresses per day were incorrect or incomplete, silently causing mis-deliveries, re-ships, and customer churn.
The obvious solution was real-time address validation at checkout. The non-obvious part was the politics: the checkout team owned conversion rate. Adding any friction to checkout was a direct threat to their metrics. They pushed back hard.
I empathized with their position and proposed an A/B test with hard guardrails: we would measure checkout completion rate and latency alongside mis-delivery rate, cost, and customer sentiment. If conversion regressed beyond an agreed threshold, we'd kill it.
I integrated Melissa Data's real-time API (<500ms latency) into the checkout flow for the treatment group. For measurement, I built three pillars: system metrics (latency, availability), structured feedback (CX reps + 50 customer interviews every two weeks during the test), and outcome tracking (did corrected addresses actually get delivered correctly).
The test showed zero conversion regression. Mis-deliveries dropped sharply. The $2M in direct mis-delivery costs disappeared, and the downstream retention impact was $20M annualized. The feature is still live on Chewy today.
This case study is about navigating a two-sided tradeoff: improving one dimension of the experience (delivery accuracy) without regressing another (checkout conversion). The key was not arguing about hypotheticals, but designing a rigorous test that let the data decide.