My Honest Review of ChatGPT Agent
OpenAI just released its AI agent
OpenAI just released ChatGPT agent, a new tool designed to actively complete work for you rather than just answer questions.
ChatGPT agent can take actions on your behalf. It can browse websites and complete multi-step tasks from start to finish. Unlike OpenAI’s Operator, it’s more reliable and incorporates the best aspects of Deep Research.
As a programmer who has been working in web automation for the past few years, I couldn’t resist the temptation to put it to the test—and here’s my honest review.
For more reviews like this, subscribe (and consider supporting us by becoming a paid subscriber)👇
ChatGPT Agent Use Cases
ChatGPT agent is an evolution of Operator and Deep Research. Operator is an AI tool designed to browse the web and visually interact with websites, while Deep Research focuses on analyzing information deeply to generate comprehensive reports.
ChatGPT agent can do both. It can seamlessly switch between reasoning and action—it knows when to stop and think and when to click or type something to move a task forward.
The main use cases of ChatGPT agent are:
Use the web: It can do different tasks for you, such as making bookings/appointments, comparison, analyzing sites, etc
Generate reports: It can produce a PowerPoint deck, a formatted Excel spreadsheet, or a PDF report as the end result of a task
Connect to your apps and take action: It can connect to apps such as Google Calendar, Gmail, Maps, and take action on your behalf
I’m going on vacation soon, so I asked ChatGPT agent to find the best restaurants in the city I’m visiting and create a new saved list on Google Maps, including notes about the signature dishes to try at each place
The video’s sped up. It took the ChatGPT agent about 23 minutes to finish the task.
First, ChatGPT agent searches for the best restaurants in the city, selects 10 of them, and collects information about their signature dishes. Then it pauses and asks me to take control to enter my Google login credentials. After that, it creates a list on Google Maps with notes for each restaurant and saves it.
All of this is amazing! That said, the ChatGPT Agent isn’t ideal for every type of task. Let’s take a look at what it does well — and where it falls short.
ChatGPT Agent: Where it shines and disappoints
OpenAI’s demos emphasize the idea of users giving a task to the agent, then stepping away to focus on other tasks while the agent completes the work independently.
While ChatGPT agent can often operate independently, it occasionally requires the user to be present at their computer. Whether it's to enter login credentials, solve captchas manually, or confirm a critical action, user involvement is sometimes necessary.
Other times, the agent doesn’t need you to take control, but it may take so long to perform a simple action—like failing to select an item from a dropdown after several minutes—that you end up stepping in to help move the automation forward.
Below is an example of this. I asked ChatGPT agent to go to a real estate site and search for homes to buy in Marbella, Spain, within a price range of €100k to €400k. It was doing a good job, but suddenly it failed to locate the maximum price in the dropdown. After 14 minutes, I took control of the site and did it myself.
I have to say, though, that was the only issue. It went through 13 homes, collected the data I asked for, and exported it to an Excel file. I double-checked the site and confirmed that the data was accurate—no hallucinations!
ChatGPT agent amazes at how well it handles tasks on the web. You can even see it pause to think before clicking or moving a task forward. However, all of this comes with one downside—it’s slow.
It took ChatGPT agent around 20 minutes (with one intervention from me) to get the job done. That makes it well-suited for some tasks, but not so great for others.
If you need a tool to perform urgent web tasks, it's better not to use ChatGPT agent. Even simple actions like clicking, selecting elements, and searching can take the agent several seconds—or even minutes
However, if you need a tool for non-time-sensitive tasks, ChatGPT agent is your best friend. You can use it to do research, take action, generate spreadsheets and slides, collect data, and more.
In the past, I had to write Python code to build a bot that could handle all of this for me. But now, I can just give instructions in plain English to the agent and get similar results.
Just remember that ChatGPT agent isn’t fully autonomous, so don’t close your laptop, grab a coffee, and expect everything to be done when you come back.
ChatGPT agent is available for Pro and Plus users. Pro users have 400 messages to use ChatGPT agent per month, while Plus users get 40 messages monthly.
I think the number of credits you get is a bit low for what you actually get. Especially considering that one credit is used for every interruption, whether it's you intervening (to solve a captcha or help when it gets stuck), or ChatGPT agent pausing to ask you to enter login credentials or confirm an action.
Let me know in the comments what you think about OpenAI’s agent.
Interesting times
Interesting. But like you say, intervention still needed. The unpaid ChatGPT also can make slides but they aren’t very good.