Home Internet, Software & Apps Google I/O 2024 announcements: Ask Photos, Gmail Summarize, Gemini Live, Veo and more explained

Google I/O 2024 announcements: Ask Photos, Gmail Summarize, Gemini Live, Veo and more explained

By G.S Vasan -

Published On: May 15, 2024

Highlights

Google is improving its Gemini integration into Photos, Gmail, Chrome, Search, and other apps.
It has announced the Veo video generator, a competitor to OpenAI's Sora.
Google will make Android 15 announcements today, i.e., May 15th.

Google said AI (artificial intelligence) 121 times at its I/O 2024 keynote yesterday. So as expected, we got a lot of Gemini updates and feature announcements. The company has better integrated Gemini into Chrome, Search, Photos, Gmail, and other Workspace apps. It also unveiled Veo, Imagen 3, Project Astra and several new AI-driven products. Let's have a look!

Show Full Article

Ask Photos

Ask Photos Search is a new Google Photos feature that uses Gemini AI to help you find specific photos and videos in your library. You just have to ask it questions about the said photo or video.
You can ask naturally like "find photos from my anniversary party last month" or "show me videos of the hills I saw on my trip last November."
It will be available in the coming months.

Gmail Summarize, Q&A, Contextual Smart Replies

Gmail is getting upgraded with Gemini 1.5 Pro-powered features like Summarize, Q&A, and Smart Replies. You can access them via a sidebar on the Gmail desktop or by tapping the Gemini button on Gmail mobile.
Tap the "summarize this email" to get a brief of the email or email thread you've received. Google is rolling this out to Workspace Labs users this month, and to all Gemini for Workspace customers and Google One AI Premium subscribers starting next month.
Tap the "Enter a prompt here" from the Gemini sidebar or Gemini button on Gmail and ask it for anything related to emails in your inbox. It will give you a relevant answer. Gmail Q&A will be coming to Workspace Labs users in July.
Based on the context from the email thread, Gemini will give you context-aware responses. You just have to use them as it is. It will be released to Workspace Labs users in July.

Veo video generator

Veo is an alternative to OpenAI's Sora video generator.
It can create over a minute-long 1080p video based on your text, image or video prompts.
Videos can be in different styles like timelapses or aerial shots. You can tweak them with further prompts too.
It's public availability hasn't been disclosed yet. Following OpenAI's steps, Google will be pitching Veo to Hollywood creators.

Gemini Live

Gemini Live lets you have a natural chat with Gemini. Just tap the mic button and have a 2-way conversation with it.
It will be available for Gemini Advanced subscribers in the coming months.
Later this year, you will be able to use the camera feed to show Gemini what you are seeing and have contextual conversations related to what you are seeing.

Gems custom chatbot

Gems is like OpenAI's GPTs for ChatGPT. In other words, Gems are custom chatbots that you can create using Gemini.
You can inject personality and get it to do a particular task repeatedly. For example, it can act like a gym buddy, shrink, or motivational coach.
It will soon be available to Gemini Advanced subscribers.

Project Astra

This isn't technically ready for the public, but it is where Google is leading many of its AI efforts.
Project Astra is like a real-time, multimodal universal AI assistant that can see the world and is contextually aware of your reality. So, it should be able to answer all your queries and help you do things.
Here's a demo, which Google claims it hasn't doctored in any way:

Other Google AI announcements at I/O 2024

Search Generative Experience is now named AI Overview on Google Search. It can understand your Google Search query and provide a specially designed results page with summarised answers from the web. AI Overview will be released to US users this week.
Circle to Search on Android can now break down math problems into steps so that you can understand and get to the answer.
Gemini on Android will gain more multimodal capabilities soon. It will be available to Gemini Advanced users in the coming months.
Google Chrome will get Gemini Nano-based AI assistant starting Chrome 126.
Later this year, Google will start rolling out Gemini Nano-based AI scam detection on Android.
Gemini Advanced users can get Gemini to scan a PDF and give responses to their queries based on the information in that PDF.
Gemini 1.5 Pro sidebar is now part of apps like Gmail, Drive, Docs, Sheets, and Slides for users of Workspace Labs and Gemini for Workspace Alpha. Starting next month, it will roll out on desktops for businesses and consumers through Gemini for Workspace add-ons and the Google One AI Premium plan.
Gemini 1.5 Pro's context window has been increased from 1 million to 2 million tokens. This refers to how much context/knowledge related to something it can carry in its mind if you will.
Gemini 1.5 Flash is the new multimodal AI model that is as capable as Gemini 1.5 Pro but designed to work for "narrow, high-frequency, low-latency tasks." Differently put, it can respond faster.