The latest advancements in AI are redefining the capabilities of smartphones. With Google AI at the core of Android’s operating system, users can now interact with their devices in groundbreaking ways. Google has announced several updates that will enhance the user experience on Android devices, leveraging the power of AI.
Circle to Search Now Helps Students with Homework
Circle to Search, integrated directly into the user interface, allows users to search anything on their phone with a simple gesture, without interrupting their current activity. Since its debut at Samsung Unpacked, the feature has expanded to include full-screen translation and is now available on more Pixel and Samsung devices.
Starting today, Circle to Search can assist students with their homework. By circling a prompt, students receive step-by-step instructions to solve various physics and math word problems directly on their devices. Later this year, the feature will support more complex problems involving symbolic formulas, diagrams, and graphs, thanks to the enhancements from Google’s LearnLM effort.
Currently, Circle to Search is available on over 100 million devices, with plans to double that number by the end of the year.
Enhanced Contextual Assistance with Gemini
Gemini, a new kind of assistant on Android, uses generative AI to boost creativity and productivity. The assistant is becoming even better at understanding the context of what’s on your screen and which app you’re using. Soon, users will be able to overlay Gemini on any app to utilize its features more seamlessly. For example, users can drag and drop generated images into Gmail or Google Messages, or ask specific questions about YouTube videos. Gemini Advanced users will also have the option to quickly get answers from PDFs without scrolling through multiple pages.
This update will roll out to hundreds of millions of devices over the next few months, with continuous improvements to provide dynamic suggestions based on screen content.
Introducing Gemini Nano with Multimodality
Android is the first mobile operating system to include a built-in, on-device foundation model. Gemini Nano will debut later this year on Pixel devices, bringing multimodal capabilities that allow phones to understand text, sights, sounds, and spoken language.
Enhanced Descriptions with TalkBack
Gemini Nano’s multimodal capabilities will soon enhance TalkBack, providing richer descriptions for people who are blind or have low vision. This update will offer detailed descriptions of images, whether it’s a photo from family or friends or the details of clothes while shopping online. These descriptions are processed on-device, ensuring speed and functionality even without a network connection.
Real-Time Scam Alerts During Calls
In response to the alarming rise in fraud, Google is testing a new feature that uses Gemini Nano to detect scam patterns during phone calls. Users will receive real-time alerts if suspicious conversation patterns are detected, such as urgent requests for money transfers or personal information. This feature processes data on-device to maintain privacy and will be available later this year.
More AI Innovations Coming to Android
Google continues to integrate AI into every aspect of the Android experience. Developers can explore the latest AI models and tools, such as Gemini Nano and Gemini in Android Studio, by visiting the Android Developers blog. More updates on Android 15 and other ecosystem advancements are expected on May 15.
