Google announced that AI Mode now lets you search by uploading a photo or image from your device. Before the new AI Mode only allowed you to search with text, but now, like you can with other Google Search features, you can use multimodal capabilities in Lens with AI Mode.
Google wrote, "With AI Mode’s new multimodal understanding, you can snap a photo or upload an image, ask a question about it and get a rich, comprehensive response with links to dive deeper. This experience brings together powerful visual search capabilities in Lens with a custom version of Gemini, so you can easily ask complex questions about what you see."
Here is what it looks like:
Here is some more of the technical details:
AI Mode builds on our years of work on visual search and takes it a step further. With Gemini’s multimodal capabilities, AI Mode can understand the entire scene in an image, including the context of how objects relate to one another and their unique materials, colors, shapes and arrangements. Drawing on our deep visual search expertise, Lens precisely identifies each object in the image. Using our query fan-out technique, AI Mode then issues multiple queries about the image as a whole and the objects within the image, accessing more breadth and depth of information than a traditional search on Google. The result is a response that’s incredibly nuanced and contextually relevant, so you take the next step.
As a reminder, Google has rolled out AI Mode beyond its premium subscribers a couple of weeks ago but I guess Google is doing a larger roll out. Google's Robby Stein wrote, "we’ve now started to make AI Mode available to millions more Labs users in the U.S."
Happy to share that AI Mode is now available to millions more Labs users and integrating Lens so you can get AI-powered responses and ask follow-up questions on your visual searches! 🎉
— Rajan Patel (@rajanpatel) April 7, 2025
Our Lens index has a rich understanding of billions of images from across the web to provide you with the most accurate visual information. And with AI Mode’s advanced multimodal capabilities, it can better understand multiple objects in an image and how they relate to each…
— Rajan Patel (@rajanpatel) April 7, 2025
Forum discussion at X.
