Your Phone’s Smartest AI Yet? Google’s Gemini Can Now Read and Watch in Real-Time
Google’s Gemini AI Gets Major Upgrade: Now Understands Your Screen and Camera View!
Google Rolls Out New Gemini Features for Smarter AI Interaction
Google is introducing groundbreaking new features for Gemini, its generative AI-powered chatbot. As part of Project Astra, these updates enable Gemini to analyze what’s on your phone screen and camera viewfinder in real-time, providing instant and relevant information.
Gemini AI Can Now See and Analyze Your Screen
First reported by 9to5Google and a Reddit user who spotted the feature on a Xiaomi device, Gemini now has the ability to read and interpret content displayed on your smartphone screen. This means users can ask the AI chatbot questions about what they’re seeing on their device, making it easier to get explanations, translations, or additional context without switching between apps.
Use Your Camera to Get Real-World Answers
A second major feature rolling out as part of Project Astra is camera integration. With this feature, users can point their phone camera at objects, animals, landmarks, or anything else and receive instant answers from Gemini. The AI can recognize and analyze the objects in real time, making it a powerful tool for learning, exploration, and everyday convenience.
To use this function, users can open the Gemini Live fullscreen interface and start a video stream. This enhances real-world interactions, making it easier to identify unknown objects, read foreign text, or even troubleshoot technical issues just by showing them to Gemini.
Exclusive to Google One AI Premium Users (For Now)
Currently, these advanced features are only available for Google One AI Premium subscribers, which starts at Rs 1,950 per month. Google had initially planned to roll them out exclusively for Pixel devices, but recent reports suggest a random rollout to other Android smartphones as well.
If you don’t have access yet, don’t worry—Google is expected to expand availability to a broader audience soon, potentially including free Gemini users in future updates.
Why These Features Matter
The integration of screen reading and camera-powered AI assistance gives Gemini a competitive edge over rivals like ChatGPT and Microsoft Copilot. It allows for more seamless, interactive, and efficient AI experiences, eliminating the need to copy-paste text or manually describe what’s on your screen.
By leveraging these updates, Google is pushing AI boundaries, making Gemini more intuitive and responsive to users' daily needs.
What’s Next for Gemini?
Google is expected to refine and expand these features in the coming months, possibly bringing more real-world AI applications like live voice assistance, enhanced object recognition, and deeper app integration. Stay tuned for official announcements from Google regarding wider availability