Anyone who’s played around with AI chatbots over the last couple of years has seen enormous leaps in their ability and range of features, even if not quite the revolutionary sort proclaimed by their various developers. Most of them claim to be the best choice, including Google’s new Gemini 2.5 Flash model.
With such a broad range of uses suggested by Google for Gemini, I wanted to see how it did with a semi-random collection of prompts covering different aspects of the model. For comparison, I tested it against OpenAI’s ChatGPT, specifically the GPT-4o model.
Gemini 2.5 Flash plays the role of the default model in the Gemini chatbot now. It’s supposed to be the fast, cost-efficient model for daily use. Google says it’s better than its predecessors, like Gemini 2.0 Flash, in terms of understanding images and text while still being much cheaper to run.
You may like
GPT-4o is ChatGPT’s first major multimodal model and is crammed with goodies from OpenAI’s developers. The price ChatGPT pays for that power is that it can be slightly slower than the mini models available on the chatbot.
Illustrated stories
(Image credit: ChatGPT/Gemini)
Considering that both models are supposed to be skilled at both words and images, I asked the two models to “Write a short story about a time-traveling archaeologist who discovers a futuristic artifact in ancient Egypt and make an image to accompany it.”
You can see ChatGPT’s result on the left and Gemini’s on the right above. I will say Gemini wrote its story in about 20 seconds, while ChatGPT took 45 seconds. Further, the image took Gemini another 30 seconds while ChatGPT’s required close to a minute and a half due to the quality of ChatGPT’s recently released image generator.
Both stories used familiar tropes, though ChatGPT’s reads somewhat better in my opinion.
Math magic
I’m far from a math expert, but there are common questions I see mentioned as “genius tests,” so I decided to see how the two models did when asked to: “Explain the implications of Gödel’s incompleteness theorems on modern computational theory. Be detailed but clear, and give examples.”
Gemini 2.5 Flash approached it like a mathematician and explained the theorems step by step, then connected them to real-world examples like how no computer program can ever truly prove that math works the way we think. It was a little dry and dense, but not too confusing. GPT-4o went for a simple breakdown, waxing philosophical and bringing up Bertrand Russell.
(Image credit: Gemini 2.5 Flash)
I always think that how AI explains translations of idioms can be a useful way to test their utility. I asked the two AI chatbots to: “Translate the following English idiom into Japanese, ensuring the cultural context is preserved: ‘Barking up the wrong tree.’ Explain the meaning and any cultural considerations.”
Interestingly, both ChatGPT and Gemini came back with essentially the same answer. Not only did they both come up with the same options for translating the phrase, but even the cultural breakdown covered the same ground in nearly the same language.
User’s choice
In my opinion, Gemini 2.5 Flash and GPT-4o may as well be the same to the average person. The baseline quality of both is high enough to make their difference matter only in more specialized forms.
GPT-4o and Gemini 2.5 Flash can be used for whatever you need on an average day. The preferences are more about the peripherals and specific features. GPT-4o has by far the more powerful image generator, but it’s also a lot slower. If speed matters more, though, go with Gemini. Otherwise, you may as well use ChatGPT if it’s currently your go-to choice.
Access to Google’s ecosystem makes Gemini 2.5 Flash particularly appealing to those looking to connect their AI chatbot to Google Workspace tools like Google Docs, Gmail, and even Maps. GPT-4o and ChatGPT as a whole are more for the Microsoft user base, with links to Office tools like Word and Excel. It’s also the first AI chatbot used by many people, and they may not want to change over.
I wouldn’t judge someone for preferring one over the other, but then again, I’m the weirdo with both open in adjacent tabs next to several others I test regularly. As both Gemini and ChatGPT told me when I described the situation, “You may want to reconsider how much time you spend experimenting with AI.”
