Apple’s visual intelligence technology could be a step toward Apple Glasses

Apple’s new “Visual Intelligence” feature was one of the most impressive things shown off at Monday’s iPhone 16 event. The tool lets users scan the world around them with their iPhone camera to identify a dog breed, copy event details from a poster, or search for almost anything around them.

It’s a handy feature that fits in perfectly with the iPhone’s new camera button. But it could also pave the way for bigger products in the future: It’s the exact kind of thing Apple will need for future tech like augmented reality glasses.

It’s not hard to imagine how visual intelligence could help you on a device that sees everything you see. Take the idea of ​​learning more about a restaurant, as Apple demonstrated visual intelligence on the iPhone: Instead of pulling your phone out of your pocket to look up information about a new place, with Glasses, you can just look at the restaurant and ask a question, and the Glasses will tell you more.

Meta has already proven that AI-powered computer glasses can be a good and useful tool for identifying objects. It’s no surprise that Apple is doing something similar with a very high level of fit and finish for the glasses. Apple will certainly make the glasses connect to all your apps and personal context on your iPhone as well, making the visual intelligence even more useful.

Of course, Apple already has a camera-covered headset: the Vision Pro. But most people don’t walk around with their headsets outside their homes, and they probably already know about the stuff in their homes. It’s long been reported that Apple wants to develop a pair of true augmented reality glasses, and this seems to be the ultimate destination for this type of technology.

The truth is, Apple’s AR glasses may be a long way off. BloombergMark Gurman of The New York Times reports: In June Apple said it has set a 2027 launch date for its in-development glasses, but noted that “no one I’ve spoken to inside Apple believes the glasses will be ready in a few years.”

But when these glasses do hit the market, they’ll need software—and you can see how Apple is building the foundation for that here. Visual intelligence could be Apple’s first step toward a truly PC-powered glasses application, and by starting now, Apple could have years to perfect the feature before it shows up in glasses.

It wouldn’t be unprecedented for Apple to take this approach. The company has been iterating on AR tech on the iPhone for years before launching the Vision Pro. Yes, the Vision Pro is arguably more of a VR headset than an AR device, but it’s clearly a first step toward something that could turn into AR glasses. As Apple refines the device, it could work on software features like visual intelligence on the iPhone as well, and when the time is right, it could package the best ideas into a glasses-like product..