8 Exciting Abilities of Google’s Project Astra

At their I/O event on Tuesday, Google demoed a very early version of Astra, a multimodal, universal, voice and vision-based AI assistant. This assistant is the brainchild of DeepMind, Google’s AI division.

Astra is paving the way for the future of AI chatbots. It’s clear from both the OpenAI and the Google launches that AI is moving on from chatbots and is heading towards personal AI assistants. Its amazing functionalities even at such a nascent stage in its development are a marvel to watch.

Astra truly demonstrates the possibility of bringing to life concepts that were only seen in science fiction movies till now. The Astra demos seen so far all work in real-time and with vision functionalities so far beyond what we have seen yet.

8 Amazing Use Cases of Google’s Astra

Let’s take a look at 8 of the most impressive use cases of Astra users have found

1. Identifying and explaining objects from a doodle

Astra showed the unique capability of identifying the type and parts of the car simply from a basic drawing of it. It correctly recognized the drawing of a Formula 1 car, and also described the functions of the parts of the car when the user zoomed in on that. It also provided the user with additional information about the car when asked.

Astra also identified a famous figure ( Albert Einstein ) from a child-like drawing and explained his achievements in the specified year. 

2. Recognizing elements from a travel journal

Astra saw the sketches for barely one second and was successfully able to recognize as well as remember all the monuments sketched in the journal. The memory capabilities are amazing! Even I did not remember all the monuments upon watching the video once.

3. Helping in the use of a fancy espresso machine

Logan Kilpatrick, the product lead for Google AI studio posted this use case on X where Astra helped him by explaining how to use a complex espresso machine.

Astra gave step-by-step instructions, explaining each part of the machine and the different tools used. It also corrected him when he went wrong and told him the correct way to utilize the instrument.

4. Discussions on art

Logan also demonstrated many more conversations on his X profile, one of which was his conversation with Astra about a piece of art.
Astra identified the art piece and the material used in making it, as well as speculated on the different possible inspirations behind the piece and also came up with a story for the same.
Astra even suggested different flowers that could fit in the piece when prompted by the user.

5. Identifying drawings in real-time

Conor Grennan, the founder and CEO of AI Mindset, posted a short video of a private testing at the Google event of the Astra project.

In the test, Astra identified what a user was drawing (an apple) and even identified a squiggly green line as a worm in the apple. Not just that, but its conversational tone, and ability to modulate voice so organically is so impressive. The experience was fully multimodal with almost no lag!

6. Remembering the objects it sees

Astra has the incredible capability of remembering the objects that it sees even if it was only visible for a few seconds.

Astra remembered every single object shown to it for a few seconds, identified them correctly. It identified items like a fish, a seashell, a crab, a spiky gold ball, and a plant.

7. Correctly identifying types of plants in real-time

Astra provided a completely humanlike interactive experience while identifying the genus of different plants from the characteristics of the leaves. It described the scientific as well as the common name of plants. While not demonstrated in this video, Astra can also suggest the most useful plants and describe the features of a plant it identifies.

8. Taking Astra for a spin around the Google grounds

Dave Burke, the VP of engineering for Google, took Astra out for a spin to interact with the world around it.

Astra held a continuous conversation with Dave while providing information about physics concepts, different cars, and objects it saw around the area at the Google IO.

Conclusion

Keeping in mind that this is the weakest technology will ever be, the capabilities of Astra still feel out of this world! If this is the first edition, imagine what capabilities future versions will have. While there are many areas of improvement for this new project, its abilities have never been seen before.