ChatGPT Just Got 2 Amazing Features: Voice and Vision

Now you can have a voice conversation with ChatGPT or show it what you’re talking about!

Image made with Canva

Hi! Here’s another free guest article from Diana. This one is about a recent big update to ChatGPT. I’m looking forward to testing it out!

Yesterday OpenAI released voice and image capabilities in ChatGPT. This is huge because now we can snap pictures and share them with ChatGPT to know how to solve a problem, ask questions about the picture, and start a conversation about it.

Voice and image capabilities in ChatGPT will definitely give us more ways to use ChatGPT in our everyday lives. Although these new features will be available to ChatGPT Plus users in the next 2 weeks, today we’ll see what we can do with these new features.

Now you can speak with ChatGPT (and have it talk back)

The new voice feature turns ChatGPT into a voice assistant similar to Siri. You only need to press a button to start a back-and-forth conversation with ChatGPT.

You can ask ChatGPT to tell you a bedtime story, ask questions about a topic, ask for advice, etc. The amazing thing is that after ChatGPT gives you a response, you can keep the conversation going by asking another question.

Here’s a demo of this feature made by OpenAI.

If you’re one of the lucky ChatGPT plus users that have this feature available, go ahead and enable it following the steps below in the mobile app:

• Go to Settings

• Select New Features

• Opt into voice conversation

• Tap the headphone button shown below

In case you’re curious about how this new feature works, OpenAI is using its speech recognition system Whisper to transcribe your spoken words into text. Note that the model is proficient at transcribing English text but performs poorly with some other languages (especially those with non-roman script).

You can choose your preferred voice out of five different voices and I’m very impressed with the AI voices available for this feature. They sound very similar to real human voices. Kudos to OpenAI and the voice actors. My favorite voice is Cove because it reminds me of one of the narrators in some audiobooks I listened to in the past.

Here’s a demo of the Cove voice extracted from the OpenAI blog.

By the way, did you know Spotify is partnering with OpenAI to use this tech for the pilot of their Voice Translation feature? This allows podcasters to translate their content into multiple languages in the podcasters’ own voices. Crazy, right?

Now you can chat about images

ChatGPT is now able to read your images and start a conversation about anything that is on the image. This can help you deal with tasks you might encounter in your daily life such as fixing an artifact that isn’t working, planning a meal based on the ingredients in an image, and even analyzing a graph with data.

A video is worth a million words. Here’s a demo from OpenAI.

If you’re one of the lucky ChatGPT Plus users who has this feature available, you’ll see these new icons when using ChatGPT.

Tap the photo button to capture or choose an image. You can upload multiple images and even use the drawing tool to guide your assistant!

OpenAI Blog

This will take DIY tutorials to the next level! 

Good times are ahead of us thanks to AI, so for more like this consider subscribing to my newsletter below.