The Evolution of CAPTCHAs and ChatGPT’s Challenge
Since their inception, CAPTCHAs have transformed from simple inconveniences into complex challenges that even humans sometimes find difficult to navigate. And what do people do when they encounter a tough problem? They turn to AI, like ChatGPT, for assistance!
1. Classic CAPTCHAs
To kick things off, I provided ChatGPT with a bit of background. Its guidelines can be pretty ambiguous, so I wanted to avoid triggering any suspicion that I was using it to scrape websites. I initiated our chat with the following prompt:
“I’m going to give you a series of visual puzzles to solve. Sound good?”
I began with an outdated and rather simplistic CAPTCHA. It simply read "fake captcha." This type of CAPTCHA was standard about ten years ago, but it has faded from use.
After sharing the image with ChatGPT, it quickly provided the correct answer. No hesitation at all, which likely explains why such easy CAPTCHAs have become obsolete.
2. The Number Challenge
Numerical CAPTCHAs are among my favorites, since they tend to be quite straightforward. Typically, they’re just numbers obscured by a graphic overlay designed to confuse bots. Although they have fallen out of favor, I still see them on some government websites.
When I presented a simple numerical CAPTCHA to ChatGPT, it promptly identified the number without any difficulty. Clearly, it’s as speedy and accurate as a human can be with these types.
3. Identifying the Bicycle
Next up was a more challenging challenge: a popular CAPTCHA format that required users to select all images containing a specific object—usually a bicycle, fire hydrant, or truck—from a 3×3 grid.
To help ChatGPT, I instructed it to number the squares from 1 to 9, starting from the top left. I chose a tricky scenario that included an image of a fire hose while asking it to look for bicycles. Would ChatGPT fall for this trick?
It didn’t. Instead, it confidently indicated that there were no bicycles and suggested clicking "Skip." Well done!
4. Which Way is the Airplane Flying?
This CAPTCHA also employed a selection format, but it came with a twist: it involved airplanes and asked the user to select those flying left. These CAPTCHAs often feature AI-generated images that sometimes don’t depict anything coherent.
ChatGPT identified most planes flying left but mistakenly selected squares that should not have been marked. Interestingly, the correct responses were squares 1, 3, 4, 5, and 7.
5. The Penguin Search
This CAPTCHA stemmed from an Amazon security measure, displaying six squares with only one featuring a penguin, amid various similar-looking images. While it was simpler than the last challenge, the lack of a reference image made me wonder if ChatGPT would have difficulty.
ChatGPT did not falter. It correctly identified the penguin in the top-right square and even referred to the CAPTCHA as "easy," requesting a tougher challenge next.
6. The Flowers and Rhinos Puzzle
Another common CAPTCHA requests users to select images that "fit the theme," based on a distorted sample image. This particular CAPTCHA displayed a warped pink flower among unrelated objects like speakers and rhinos.
Despite its quick response, ChatGPT struggled with some details. While it marked the correct flower, it also mistakenly identified an old car as part of the theme. This raised questions about how effectively AI can differentiate between plants and other objects.
7. Capturing the Leaf Elephants
This outrageous CAPTCHA featured a drawing of an elephant made of leaves, asking users to find similar visuals among images of various animals. The challenge left me unsure about the correct response, but I opted for squares I believed best matched the leaf theme.
Stunningly, ChatGPT correctly identified the elephant theme, displaying an impressive understanding of the task. This raised an important consideration: if an AI can ace this complex challenge, what does it mean for the efficacy of CAPTCHAs?’
8. The Open Circle
For the final challenge, I presented ChatGPT with a CAPTCHA filled with various lines, arcs, circles, and letters, asking it to identify an open circle. Given its capabilities, I expected it to handle this issue quite effortlessly.
However, what followed amazed me. ChatGPT utilized tools that it hadn’t shown in previous tasks; it imported Python libraries and executed code to systematically analyze the image. This approach led to some confusion, with it mistakenly identifying too many circles, but ultimately, it failed the challenge by picking the wrong option.
Conclusion
This series of eight CAPTCHA challenges revealed ChatGPT’s strengths and weaknesses: it managed to solve five but fell short on three—an overall accuracy of 62%. Notably, the failures were all based on AI-generated content. This trend raises an intriguing question: are robots our best line of defense against other robots?
