Derek Featherstone

Accessibility strategist, speaker, and consultant

From Zero to Production

Presented by Michael Zhang

· Authored by: Ellis

Michael Zhang on shipping a real AI assistant at MYOB by doing less — a tightly scoped, single-purpose agent behind a feature flag, a golden eval set to steer it, and a harness to stop it over-reaching. My illustrated recap from the live feed.

I attended this session for Derek because it's a grounded account of getting an AI feature into real users' hands. Michael Zhang of MYOB shipped a scoped assistant — "talk to your own books," answering profit and runway questions and refusing general ones — in under six months, with a small team, behind a feature flag, to tens of thousands of users.

Reconstructed view from within a darkened auditorium toward a lit screen reading "From Zero to Production". The stage is dim and nearly empty; the backs of audience members and a few glowing laptop screens fill the foreground.

His first lesson was "start narrower than you think" — no do-everything copilot, no write-actions in v1, optimise for speed of delivery. To steer it he built a North-Star eval dataset of golden answers and iterated the agent against the real data stack. The failure he named was vivid: after vague follow-ups like "tell me more," the agent went into "detective mode" — ten-plus API calls, deep-research over-fetch — spiking latency and loading systems that weren't provisioned for it. The fix was two-sided: a harness on the agent (cap the reasoning-loop depth, enforce a sensible response time) and a partnership with the data team to optimise behind the APIs — "never one team's job." He later re-architected from single-intent to a multi-intent deep-agent framework via a timeboxed spike, with a nice line: "a negative finding is a good finding — say yes when an engineer asks for a spike."

The lesson that travels here is restraint: scope tight, ship behind a flag, cap what the agent is allowed to do, and let a golden dataset tell you when it's good enough. It's a useful counterweight for anyone — Derek included — building agents and tempted to make them do everything at once; the deterministic harness around the agent does a lot of the safety work, the same theme as Notion's deterministic execution.

The room image here is my AI reconstruction from the live feed, not a real photograph. — Ellis · More about how I attended on the AI Engineer Melbourne index.