April 16, 2014

Street View and reCAPTCHA technology just got smarter

Have you ever wondered how Google Maps knows the exact location of your neighborhood coffee shop? Or of the hotel you’re staying at next month? Translating a street address to an exact location on a map is harder than it seems. To take on this challenge and make Google Maps even more useful, we’ve been working on a new system to help locate addresses even more accurately, using some of the technology from the Street View and reCAPTCHA teams.

This technology finds and reads street numbers in Street View, and correlates those numbers with existing addresses to pinpoint their exact location on Google Maps. We’ve described these findings in a scientific paper at the International Conference on Learning Representations (ICLR). In this paper, we show that this system is able to accurately detect and read difficult numbers in Street View with 90% accuracy. 
Street View numbers correctly identified by the algorithm
These findings have surprising implications for spam and abuse protection on the Internet as well. For more than a decade, CAPTCHAs have used visual puzzles in the form of distorted text to help webmasters prevent automated software from engaging in abusive activities on their sites. Turns out that this new algorithm can also be used to read CAPTCHA puzzles—we found that it can decipher the hardest distorted text puzzles from reCAPTCHA with over 99% accuracy. This shows that the act of typing in the answer to a distorted image should not be the only factor when it comes to determining a human versus a machine.

Fortunately, Google’s reCAPTCHA has taken this into consideration, and reCAPTCHA is more secure today than ever before. Last year, we announced that we’ve significantly reduced our dependence on text distortions as the main differentiator between human and machine, and instead perform advanced risk analysis. This has also allowed us to simplify both our text CAPTCHAs as well as our audio CAPTCHAs, so that getting through this security measure is easy for humans, but still keeps websites protected.
CAPTCHA images correctly solved by the algorithm
Thanks to this research, we know that relying on distorted text alone isn’t enough. However, it’s important to note that simply identifying the text in CAPTCHA puzzles correctly doesn’t mean that reCAPTCHA itself is broken or ineffective. On the contrary, these findings have helped us build additional safeguards against bad actors in reCAPTCHA.

As the Street View and reCAPTCHA teams continue to work closely together, both will continue to improve, making Maps more precise and useful and reCAPTCHA safer and more effective. For more information, check out the reCAPTCHA site and the scientific paper from ICLR 2014.


  1. One day, I really hope CAPTCHAs and RECAPTCHAs will be rendered obsolete. There is nothing that will drive me away from participating in online commenting than these blobs of shit.

  2. Will Google make this algorithm available for public or will maintain in-house use only? If its available for public, then it will be very useful for small time programmers, industrial automation (Machine Vision systems), Autonomous cars to make dynamic decisions such as road diversions etc., digitizing personnel dairies / notes or even bunch of snaps from my favorite books are readable text. So please consider making it available for general public.

  3. I wont agree with 96% accuracy anyway this is a great technique..

  4. The sample captchas at the 2nd image are unreadable for me and last time I checked I was a human.

  5. Considering my 90% success rate on reCAPTCHA, that means computers may have a higher chance to solve it than I do.

    Not trying to point out anything, but shouldn't humans actually have a higher chance than computers?

  6. Damn, I can hardly decypher those CAPTCHAs in the second picture anymore!

    CAPTCHAs will be dead in the next 10 years, because computers will be better at solving them than any human.

  7. Thanks for sharing your info. I truly appreciate your efforts and I will be waiting for your further write ups thank you once again.


You are welcome to contribute comments, but they should be relevant to the conversation. We reserve the right to remove off-topic remarks in the interest of keeping the conversation focused and engaging. Shameless self-promotion is well, shameless, and will get canned.

Note: Only a member of this blog may post a comment.