Your PC can now see what you’re looking at on the screen, if you choose to let it. Copilot Vision is now generally available for users in the United States, Microsoft said Thursday.
Copilot Vision sort of replaces the kinds of how-to articles, TikToks, and short videos you turn to when you need help. Basically, it “looks over your shoulder” at your PC’s screen when you (orally) ask it a question, and it walks you through what you need to do.
Copilot Vision debuted at Microsoft’s 50th anniversary party at its headquarters in Redmond, and later was released into Microsoft’s Insider channels, where I was able to go hands-on with Copilot Vision, with middling success. With certain applications, like Photoshop, the app was able to direct me to a specific menu and commands when I needed help editing a photo. But it completely fumbled the ball with basic recognition on apps like Microsoft’s venerable Solitaire.
Now, Microsoft is promising that Copilot Vision is more fully baked. Two things have improved since I tested it: First, it can visually highlight what you need to click or do, making it easier to find that command or shortcut. Second, you can now use it in two different applications at once.
Mark Hachman / Foundry
Copilot Vision is part of Copilot, and needs to be turned on. You then need to specify what applications you want it to “see.” Unlike Windows Recall, which periodically takes snapshots of your screen, Copilot Vision has no long-term memory. In my tests, it saw what I saw — if I scrolled to the bottom of a web page, it didn’t see or remember the top. It’s not clear how it will see two applications — a split screen, maybe?
It’s easy to call Copilot Vision “AI,” but it’s not clear if it fits the classical definition. Microsoft implied that the technology requires the use of an NPU, but the company’s blog post announcing Copilot Vision’s release made no mention of any hardware requirements. In my tests, I first used a laptop with an older Core Ultra Series 1 processor with just a handful of TOPS. It ran extremely slowly, with Copilot reacting after 10 seconds or more. With a more modern Copilot+ PC, Copilot Vision was pretty snappy.
Will Copilot Vision be met with the same distrust as Recall? It might. But its built-in limitations might reassure those who worry that it’s just another iteration of spyware. For me, it just wasn’t that effective.
