Reviewers checking 200-page grant applications needed findings they could defend. The verifier compares the programme rules, the application and its annexes, and flags potential inconsistencies with links to the relevant sources.
Final verification went from 5 hours to 2–3, with a person still reviewing.
One wrong digit can mean the wrong order. We tested whether product names and catalogue numbers survived a translated call, and the findings shaped the glossary and the rules for alphanumeric strings.
Customers call in their own language; the agent switches on real-time translation for both sides.
In the pilot, most users chose the traditional route over the tool. I fixed the main campaign-creation flow first and shipped only what supported adoption, instead of working through a large backlog.
Quarter on quarter, in production: conversion of new users +36%, adoption +20%, monthly retention 62%.
Conversion+36%New users who reached their first campaign
Adoption+20%Users with access who have created at least one campaign
Retention62%Users who created a campaign and created one again the following month, in the second quarter
The three measures and how each one was defined. Growth is relative, quarter on quarter: two consecutive quarters in production. An active user is one who created a campaign.
A game about shipping AI products rewarded the shortcuts it was meant to challenge. I redesigned the scoring and checked all 2,187 decision paths in each story.
Under the revised rules, winning requires both launch performance and responsible decisions. The learning outcome still needs testing.
The player weighs competing plans, with separate Launch and Judgment scores.
Also at work
Optimising a PM’s work across six projects
At work · internal
Decision
I led six projects in parallel and was short of time. I planned each day with time estimates booked in my calendar, then used Claude Code to automate summaries, notes, task creation and meeting agendas, based on the full context of each project.
Evidence
My working time dropped by about a third, measured with a time tracker. The time went to conceptual work, to polishing features for clients and to internal projects, including my own agent product for managing context and processes inside an organisation.
Not yet known
This measures how I work as a PM across the projects I led, not the output of a team. I handed the system over to others afterwards, and I do not know how it is used now.
Own experiments
Things I build to find out what works. Each one says what it shows and what it does not show yet.
Everyone Else
Own project · research
Decision
Do not fake a common scale: dots only when both sides use the same unit, and every finding carries a confidence label.
Evidence
22 sources, each linked in the piece. Checking caught three claims that went too far. When a separate Claude instance recoded the data without seeing my thesis, the share where AI is done to people fell from 63% to 48% (newsletters from 5% to 3%), and the finding held.
Not yet known
The piece is online, but I have no reader feedback yet. Two media studies and one HBR article were read only as abstracts. The Reddit sample leans on Poland and on outrage.
Build eval sets from incidents, not from imagination. Name a person who owns each automated decision, and set a severity that always goes to a human.
Evidence
Both documents come from two LLM products I worked on at an AI startup: a grant verifier, and a tool for creating campaign materials and optimising content for SEO. Public on GitHub, CC0.
Not yet known
Nobody outside my own work uses them yet. A worked example of the checklist is in the tool, based on my Feedback Triage workflow.
Severity overrides confidence: only confident, low-severity items are handled automatically; everything else goes to a person who approves or overrides.
Evidence
The workflow runs end to end in n8n: form, classification, decision log, routing, human review and a weekly audit.
Not yet known
Results from a labelled test set, a calibration check and a sample audit of the automatic decisions are still to come.
The Not-Built List
What I decided not to build, what I built instead, and why.
