I tried ChatGPT Voice’s keyboard-free future. It still needs a keyboard

OpenAI plans for its revamped voice mode are even more ambitious than I imagined. Rather than just having ChatGPT hang out and shoot the breeze with you, OpenAI has its sights set on a loftier goal: replacing our keyboards.

During a podcast with tech journalist Joanna Stern, OpenAI President Greg Brockman predicted that we’ll eventually prefer speaking to our computers rather than typing for the “vast majority of what we do,” and that AI voice modes like the revamped ChatGPT Voice will be a key part of that sea change.

Of course, saying that ChatGPT Voice will completely replace the keyboard is hyperbole; even Brockman admits that he still likes typing when it comes to “a very quick message.” And let’s not forget that OpenAI’s very first hardware product was, in fact, a mini keyboard for Codex users.

Still, I was intrigued by the notion of using ChatGPT Voice to actually get stuff done rather than just aimlessly talking. Now that voice mode works with ChatGPT Work and Codex on the ChatGPT desktop app, I wanted to try a keyboardless vibe-coding session — just ChatGPT Voice, me, and my personal self-hosted projects, with me talking with ChatGPT while I folded my laundry.

Welcome to another edition of Prompt Mode, your weekly AI newsletter.

I’m your host, Ben Patterson. Each week on Prompt Mode, I’ll be serving up analysis of the AI trends that matter to everyday users like you and me. Stay tuned for practical AI tips, hands-on experiences with the latest AI tools, and–you guessed it–prompts to help you get the most out of your AI assistants.

Thanks for reading, and if you like what you see, just sign up right here.

My one-hour voice-coding session started awkwardly, as I tried to establish whether ChatGPT was working in a specific directory on my computer. “OK, so now are we in the project?” I kept repeating, while later it was tricky asking ChatGPT to move up a level in the directory structure (“what files are you seeing now?”). I was during these strained interactions when I missed my keyboard the most.

But eventually I got the hang of it, and soon I discovered one of ChatGPT Voice’s superpowers: the ability to dispatch agents to perform other tasks in different projects, all while we were still talking. As I continued folding towels, I asked ChatGPT to chase down a bug in my Jellyfin streaming-video instance, then I asked it to check the health of my aging Raspberry Pi 3. ChatGPT deployed agents to perform both those tasks, giving me status updates as the agents reported back.

Now, there were some awkward silences while waiting for ChatGPT’s agents to finish their errands. “Hey, you still there?” I kept asking, and while ChatGPT was still there, I found myself wanting reassurance that something was happening in the background.

Once I realized that ChatGPT was in fact ready and waiting, we kicked off a brainstorm session for a new web project, using a previously abandoned project as a jumping-off point. I asked ChatGPT to look in the old directory, scan the files, and help me sketch out ideas for new directions. Once we arrived at a new premise (it’s about how local AI models deal with uncertainty when answering questions — stay tuned), I asked ChatGPT to assemble scaffolding documents and drop them into a fresh directory, all ready for a new build.

Impressive, but does this mean I’m replacing my keyboard with ChatGPT Voice? No way. A brainstorming session is one thing, but when it comes to navigating directories or (especially) writing stories, I need the precision of a keyboard. Indeed, I don’t think I could ever dictate an article with ChatGPT Voice; for me, typing is just too integral to the thinking process.

Still, OpenAI and plenty of other AI providers seem intent on making AI voice modes happen, and if they haven’t replaced our keyboards today, I wonder if they’ll be sitting alongside them tomorrow.

Prompt of the week: The “On a Budget” prompt

Ever get the feeling that your AI chatbot is creating work for work’s sake? I’ve been noticing that with the latest Claude and ChatGPT models, which have been finding a suspicious number of bugs that urgently need fixing — and the more bugs you squash, the faster you burn through your usage limits.

With (reluctant) help from Claude and ChatGPT, I devised a prompt that makes those AI models more respectful of your time and tokens, separating real problems from cosmetic ones while scaling back the scope of any fixes you do need to implement. Whether you’re debugging code or reviewing an Excel spreadsheet, the “on a budget” prompt will keep Claude, ChatGPT, and other AI chatbots from wasting your time and money.