How I've been using voice mode with AI
Six patterns from 80 voice-mode conversations, how my use shifted from quick hits to structured huddles, and the habit I'm encoding into accessibility tools.
In early 2025 I was catching up with a friend and former co-worker Alistair Garrison on the work each of us were doing with AI. Alistair nudged me: "have you tried voice mode? just... talking to it?"
I did, and I was immediately and simultaneously intrigued and... weirded out.
Well over a year later, no more weirdness, and I've reviewed my archive of voice conversations and pulled some data. I've had just shy of 80 voice conversations over 15 months with AI (some Claude, some ChatGPT).
Six patterns kept showing up on how I use voice mode.
The six patterns
Idea Capture (<5%). I thought for sure this would have been at least 25% of my sessions. This is all about getting an idea out of my head while it's still warm. "I want to capture this thought before it's gone and then I'll come back to it after lunch." Surprisingly, this is the lightest category. And which is the largest, you might ask? Read on...
Exploring a problem-space (~40%). This is by far the largest category -- using voice mode as a thinking partner. It turns out that many of the sessions that started with pure capture transformed into larger explorations. In the past, I've used voice memos on my phone to record/capture thoughts. But this is different -- voice mode makes it a two-way street, which transforms many of those "capture sessions" into something more. I've used it to think through the architecture of an accessibility-evaluation tool I'm prototyping, the changing role of design systems, or whether to scale a product I built into a SaaS product.
Deep thought and exploration has its counterpart, too:
Quick utility / single-question (~25%). Admittedly, this is kind of like using AI as a search engine -- one relatively bounded question with one bounded answer. Quick hits that I probably could have found some other way. Most of the time I forget the conversation happened, and this isn't necessarily a great use of voice. The next one is though, and it's one of my favourites:
Learning a specific topic (~15%). When I want to understand something better, I often just ask in voice mode to "Walk me through..." or "Help me understand..." I'm VERY specifically in learning mode, and I've often had to push in the voice mode conversation to go deeper. Claude has occasionally told me to "not worry about that detail right now" and I've responded with "I'm trying to envision how I would do this myself, so I need to get into the weeds here." These were finite learning needs: figuring out CarPlay's capabilities while I was planning an iOS app, how it works, and how to troubleshoot. What I want at the end is a sharper mental model, not a decision.
Prep work for an upcoming call (~10%). Deliberate use of a 30–60 minute block before a specific meeting or call. Thinking and talking through possible scenarios for a conversation, and even trial runs of what I want to say. The conversation produces something I'll actually use within hours.
Huddle / structured walk (~10%). A named agenda, walked through item by item, toward a specific outcome. These are morning planning sessions, design walks for systems I'm building, or reviewing blog post candidates against a publishing plan. Either way, the agenda exists before the conversation starts.
Changes over time
How I've used voice mode has changed significantly over time.
In the beginning almost all of my conversations in voice mode were those quick hits - one-shot "I need an answer to a question." In the very first quarter, two-thirds of my voice conversations were quick hits. Now? They barely register. My usage has moved through multiple phases:
- Quick-hit dominant
- Exploration dominant
- Learning & Structured agenda dominant.
Quick hits were my on-ramp and once I understood voice mode better and saw the potential, that led to exploration of what was possible. And now, as of this writing, I use voice mode primarily for structured huddles with my Chief of Staff agent.
What this is turning into
What I've noticed about my own use of voice mode is now a design principle in something else I'm building. Voice mode works because the back-and-forth pushes both of us to ask "what's missing here?" — sometimes Claude asks a question I hadn't thought of, sometimes I push Claude into a corner of the problem it would have skipped.
I'm encoding that same habit into accessibility agents and tools I'm building — systems that interrogate the gaps in a design, not just assert what's present. The systems we're all building aren't smarter than humans working in accessibility; they're vehicles for the questions we always ask, applied at a scale we can't reach by hand.
Where this is going
After 80 voice mode conversations, I've learned a lot about what works and how to design for it. What I'm building — both a framework for voice mode conversations and the personal operating system I run my work through — is what the next two posts get into.
There are two levels to what I'm building and what I'll share:
- The interaction layer — I've got a long list of principles for what makes for a great interaction and experience with voice mode. They're all about how Claude talks to me when I'm not using a screen or when the screen is small and not conducive to typing/keyboard use. AND, they're about understanding the differences in interaction and cognitive modes that are dependent on whether I'm typing or speaking.
- The context layer — we'll go deeper here, but as a preview, I've put a lot of thought into what context Claude needs when it starts talking to me depending on what I'm trying to accomplish. I'll also share more about the small stack of infrastructure I built so the right context gets into my phone before we start a voice mode conversation.
Both are really about the more substantive types of conversations - Exploring, Learning, Prep, and Huddle modes. Quick utility and basic capture doesn't need any of the design work the next two posts describe. Most voice mode conversations probably don't.
But when I'm tucking into a voice conversation with a specific goal and the work matters... that's when the design matters. That's what the rest of this series is about.