New Research: Some Tough Questions for ‘Security Questions’

Posted by Elie Bursztein, Anti-Abuse Research Lead and Ilan Caron, Software Engineer

What was your first pet’s name?
What is your favorite food?
What is your mother’s maiden name?

What do these seemingly random questions have in common? They’re all familiar examples of ‘security questions’. Chances are you’ve had to answer one these before; many online services use them to help users recover access to accounts if they forget their passwords, or as an additional layer of security to protect against suspicious logins.

But, despite the prevalence of security questions, their safety and effectiveness have rarely been studied in depth. As part of our constant efforts to improve account security, we analyzed hundreds of millions of secret questions and answers that had been used for millions of account recovery claims at Google. We then worked to measure the likelihood that hackers could guess the answers.

Our findings, summarized in a paper that we recently presented at WWW 2015, led us to conclude that secret questions are neither secure nor reliable enough to be used as a standalone account recovery mechanism. That’s because they suffer from a fundamental flaw: their answers are either somewhat secure or easy to remember—but rarely both.

Click infographic for larger version

Easy Answers Aren’t Secure

Not surprisingly, easy-to-remember answers are less secure. Easy answers often contain commonly known or publicly available information, or are in a small set of possible answers for cultural reasons (ie, a common family name in certain countries).

Here are some specific insights:

• With a single guess, an attacker would have a 19.7% chance of guessing English-speaking users’ answers to the question "What is your favorite food?" (it was ‘pizza’, by the way) 
• With ten guesses, an attacker would have a nearly 24% chance of guessing Arabic-speaking users’ answer to the question "What’s your first teacher’s name?"
• With ten guesses, an attacker would have a 21% chance of guessing Spanish-speaking users’ answers to the question, "What is your father’s middle name?"
• With ten guesses, an attacker would have a 39% chance of guessing Korean-speaking users’ answers to the question "What is your city of birth?" and a 43% chance of guessing their favorite food.

Many different users also had identical answers to secret questions that we’d normally expect to be highly secure, such as "What’s your phone number?" or "What’s your frequent flyer number?". We dug into this further and found that 37% of people intentionally provide false answers to their questions thinking this will make them harder to guess. However, this ends up backfiring because people choose the same (false) answers, and actually increase the likelihood that an attacker can break in.

Difficult Answers Aren’t Usable

Surprise, surprise: it’s not easy to remember where your mother went to elementary school, or what your library card number is! Difficult secret questions and answers are often hard to use. Here are some specific findings:

• 40% of our English-speaking US users couldn’t recall their secret question answers when they needed to. These same users, meanwhile, could recall reset codes sent to them via SMS text message more than 80% of the time and via email nearly 75% of the time.
• Some of the potentially safest questions—"What is your library card number?" and "What is your frequent flyer number?"—have only 22% and 9% recall rates, respectively.
• For English-speaking users in the US the easier question, "What is your father’s middle name?" had a success rate of 76% while the potentially safer question "What is your first phone number?" had only a 55% success rate.

Why not just add more secret questions?

Of course, it’s harder to guess the right answer to two (or more) questions, as opposed to just one. However, adding questions comes at a price too: the chances that people recover their accounts drops significantly. We did a subsequent analysis to illustrate this idea (Google never actually asks multiple security questions).

According to our data, the ‘easiest’ question and answer is "What city were you born in?"—users recall this answer more than 79% of the time. The second easiest example is "What is your father’s middle name?", remembered by users 74% of the time. If an attacker had ten guesses, they’d have a 6.9% and 14.6% chance of guessing correct answers for these questions, respectively.

But, when users had to answer both together, the spread between the security and usability of secret questions becomes increasingly stark. The probability that an attacker could get both answers in ten guesses is 1%, but users will recall both answers only 59% of the time. Piling on more secret questions makes it more difficult for users to recover their accounts and is not a good solution, as a result.

The Next Question: What To Do?

Secret questions have long been a staple of authentication and account recovery online. But, given these findings its important for users and site owners to think twice about these.

We strongly encourage Google users to make sure their Google account recovery information is current. You can do this quickly and easily with our Security Checkup. For years, we’ve only used security questions for account recovery as a last resort when SMS text or back-up email addresses don’t work and we will never use these as stand-alone proof of account ownership.

In parallel, site owners should use other methods of authentication, such as backup codes sent via SMS text or secondary email addresses, to authenticate their users and help them regain access to their accounts. These are both safer, and offer a better user experience.