Project Wycheproof

Posted by Daniel Bleichenbacher, Security Engineer and Thai Duong, Security Engineer
We’re excited to announce the release of Project Wycheproof, a set of security tests that check cryptographic software libraries for known weaknesses. We’ve developed over 80 test cases which have uncovered more than 40 security bugs (some tests or bugs are not open sourced today, as they are being fixed by vendors). For example, we found that we could recover the private key of widely-used DSA and ECDHC implementations. We also provide ready-to-use tools to check Java Cryptography Architecture providers such as Bouncy Castle and the default providers in OpenJDK.

The main motivation for the project is to have an achievable goal. That’s why we’ve named it after the Mount Wycheproof, the smallest mountain in the world. The smaller the mountain the easier it is to climb it!

In cryptography, subtle mistakes can have catastrophic consequences, and mistakes in open source cryptographic software libraries repeat too often and remain undiscovered for too long. Good implementation guidelines, however, are hard to come by: understanding how to implement cryptography securely requires digesting decades' worth of academic literature. We recognize that software engineers fix and prevent bugs with unit testing, and we found that many cryptographic issues can be resolved by the same means.

These observations have prompted us to develop Project Wycheproof, a collection of unit tests that detect known weaknesses or check for expected behaviors of some cryptographic algorithm. Our cryptographers have surveyed the literature and implemented most known attacks. As a result, Project Wycheproof provides tests for most cryptographic algorithms, including RSA, elliptic curve crypto, and authenticated encryption.

Our first set of tests are written in Java, because Java has a common cryptographic interface. This allowed us to test multiple providers with a single test suite. While this interface is somewhat low level, and should not be used directly, we still apply a "defense in depth" argument and expect that the implementations are as robust as possible. For example, we consider weak default values to be a significant security flaw. We are converting as many tests into sets of test vectors to simplify porting the tests to other languages.

While we are committed to develop as many tests as possible and external contributions are welcome — if you want to contribute, please read CONTRIBUTING before sending us pull requests — Project Wycheproof is by no means complete. Passing the tests does not imply that the library is secure, it just means that it is not vulnerable to the attacks that Project Wycheproof tries to detect. Cryptographers constantly discover new weaknesses in cryptographic protocols. Nevertheless, with Project Wycheproof developers and users now can check their libraries against a large number of known attacks without having to sift through hundreds of academic papers or become cryptographers themselves.

For more information about the tests and what you can do with them, please visit our homepage on GitHub.