Learnings from kCTF VRP's 42 Linux kernel exploits submissions

Tamás Koczka, Security Engineer

In 2020, we integrated kCTF into Google's Vulnerability Rewards Program (VRP) to support researchers evaluating the security of Google Kubernetes Engine (GKE) and the underlying Linux kernel. As the Linux kernel is a key component not just for Google, but for the Internet, we started heavily investing in this area. We extended the VRP's scope and maximum reward in 2021 (to $50k), then again in February 2022 (to $91k), and finally in August 2022 (to $133k). In 2022, we also summarized our learnings to date in our cookbook, and introduced our experimental mitigations for the most common exploitation techniques.

In this post, we'd like to share our learnings and statistics about the latest Linux kernel exploit submissions, how effective our mitigations are against them, what we do to protect our users, and, finally, how we are changing our program to align incentives to the areas we are most interested in.

Learnings and Statistics

Since its inception, the program has rewarded researchers with a total of 1.8 million USD, and in the past year, there has been a clear trend: 60% of the submissions exploited the io_uring component of the Linux kernel (we paid out around 1 million USD for io_uring alone). Furthermore, io_uring vulnerabilities were used in all the submissions which bypassed our mitigations.

 

Limiting io_uring

To protect our users, we decided to limit the usage of io_uring in Google products: 

• ChromeOS: We disabled io_uring (while we explore new ways to sandbox it).

• Android: Our seccomp-bpf filter ensures that io_uring is unreachable to apps. Future Android releases will use SELinux to limit io_uring access to a select few system processes. 

• GKE AutoPilot: We are investigating disabling io_uring by default.

• It is disabled on production Google servers.

While io_uring brings performance benefits, and promptly reacts to security issues with comprehensive security fixes (like backporting the 5.15 version to the 5.10 stable tree), it is a fairly new part of the kernel. As such, io_uring continues to be actively developed, but it is still affected by severe vulnerabilities and also provides strong exploitation primitives. For these reasons, we currently consider it safe only for use by trusted components.

Transparency

Currently, we make vulnerability details public on our spreadsheet (which now also includes CVE details), and we have summarized different exploitation techniques in our cookbook. In the future, to make our efforts more transparent and give faster feedback to the community, we will ask researchers to open-source their submissions, including the code they used.

Introducing kernelCTF

To better align incentives with our areas of interest, we are shifting our focus from GKE and kCTF to the latest stable kernel and our mitigations. As a result, starting today we will handle kernel exploit submissions under a new name, "kernelCTF," with its own reward structure and submission process. The maximum total payout for kernelCTF is still $133,337 per submission. While the specific GKE kernel configuration is still covered by the new kernelCTF, exploits affecting non-kernel components like the full GKE stack (including Kubernetes), the container runtime, and GKE itself, are now separately eligible for vulnerability rewards under the kCTF VRP which is returning to its original reward amounts and conditions.

Conclusion

Our goal remains the same: we are building a pipeline to analyze, experiment, measure, and build security mitigations to make the Linux kernel as safe as possible, with the help of the security community. We hope that over time, we will be able to implement security mitigations that make it more difficult to exploit Linux kernel vulnerabilities.

With the name change, we have moved our communication channel to #kernelctf on Discord, with a separate #kernelctf-announcements channel. Please join us there for the latest updates regarding kernelCTF.