Privileged pod escalations in Kubernetes and GKE

May 18, 2022

Posted by GKE and Anthos Platform Security Teams

Privileged pods within the context of GKE security

While privileged pods can pose a security issue, it’s important to look at them within the overall context of GKE security. To use a privileged pod as a “trampoline” in GKE, there is a major prerequisite – the attacker has to first execute a successful application compromise and container breakout attack.

Because the use of privileged pods in an attack requires a first step such as a container breakout to be effective, let’s look at two areas:

features of GKE you can use to reduce the likelihood of a container breakout
steps the GKE team is taking to minimize the use of privileged pods and the privileges needed in them.

Reducing container breakouts

There are a number of features in GKE along with some best practices that you can use to reduce the likelihood of a container breakout:

Use GKE Sandbox to strengthen the container security boundary. Over the last few months, GKE Sandbox has protected containers running it against several newly discovered Linux kernel breakout CVEs.
Adopt GKE Autopilot for new clusters. Autopilot clusters have default policies that prevent host access through mechanisms like host path volumes and host network. The container runtime default seccomp profile is also enabled by default on Autopilot which has prevented several breakouts.
Subscribe to GKE Release Channels and use autoupgrade to keep nodes patched automatically against kernel vulnerabilities.
Run Google’s Container Optimized OS, the minimal and hardened container optimized OS that makes much of the disk read-only.
Incorporate binary authorization into your SDLC to require that containers admitted into the cluster are from trusted build systems and up-to-date on patching.
Use Secure Command Center’s Container Threat Detection or supported third-party tools to detect the most common runtime attacks.

More information can be found in the GKE Hardening Guide.
How GKE is reducing the use of privileged pods.

While it’s not uncommon for customers to install privileged pods into their clusters, GKE works to minimize the privilege levels held by our system components, especially those that are enabled by default. However, there are limits as to how many privileges can be removed from certain features. For example, Anthos Config Management requires permissions to modify most Kubernetes objects to be able to create and manage those objects.

Some other privileges are baked into the system, such as those held by Kubelet. Previously, we worked with the Kubernetes community to build the Node Restriction and Node Authorizer features to limit Kubelet's access to highly sensitive objects, such as secrets, adding protection against an attacker with access to the Kubelet credentials.

More recently, we have taken steps to reduce the number of privileged pods across GKE and have added additional documentation on privileges used in system pods as well as information on how to improve pod isolation. Below are the steps we’ve taken:

We have added an admission controller to GKE Autopilot and GKE Standard (on by default) and GKE/Anthos (opt-in) that stops attempts to run as a more privileged service account, which blocks a method of escalating privileges using privileged pods.
We created a permission scanning tool that identifies pods that have privileges that could be used for escalation, and we used that tool to perform an audit across GKE and Anthos.
The permission scanning tool is now integrated into our standard code review and testing processes to reduce the risk of introducing privileged pods into the system. As mentioned earlier, some features require privileges to perform their function.
We are using the audit results to reduce permissions available to pods. For example, we removed “update nodes and pods” permissions from anetd in GKE.
Where privileged pods are required for the operation of a feature, we’ve added additional documentation to illustrate that fact.
We added documentation that outlines how to isolate GKE-managed workloads in dedicated node pools when you’re unable to use GKE Sandbox to reduce the risk of privilege escalation attacks.

In addition to the measures above, we recommend users take advantage of tools that can scan RBAC settings to detect overprivileged pods used in their applications. As part of their presentation, the Palo Alto researchers announced an open source tool, called rbac-police, that can be used for the task. So, while it only takes a single overprivileged workload to trampoline to the cluster, there are a number of actions you can take to minimize the likelihood of the prerequisite container breakout and the number of privileges used by a pod.

I/O 2022: Android 13 security and privacy (and more!)

May 11, 2022

Posted by Eugene Liderman and Sara N-Marandi, Android Security and Privacy Team
Every year at I/O we share the latest on privacy and security features on Android. But we know some users like to go a level deeper in understanding how we’re making the latest release safer, and more private, while continuing to offer a seamless experience. So let’s dig into the tools we’re building to better secure your data, enhance your privacy and increase trust in the apps and experiences on your devices. Low latency, frictionless security Regardless of whether a smartphone is used for consumer or enterprise purposes, attestation is a key underpinning to ensure the integrity of the device and apps running on the device. Fundamentally, key attestation lets a developer bind a secret or designate data to a device. This is a strong assertion: "same user, same device" as long as the key is available, a cryptographic assertion of integrity can be made. With Android 13 we have migrated to a new model for the provisioning of attestation keys to Android devices which is known as Remote Key Provisioning (RKP). This new approach will strengthen device security by eliminating factory provisioning errors and providing key vulnerability recovery by moving to an architecture where Google takes more responsibility in the certificate management lifecycle for these attestation keys. You can learn more about RKP here.

We’re also making even more modules updatable directly through Google Play System Updates so we can automatically upgrade more system components and fix bugs, seamlessly, without you having to worry about it. We now have more than 30 components in Android that can be automatically updated through Google Play, including new modules in Android 13 for Bluetooth and ultra-wideband (UWB). Last year we talked about how the majority of vulnerabilities in major operating systems are caused by undefined behavior in programming languages like C/C++. Rust is an alternative language that provides the efficiency and flexibility required in advanced systems programming (OS, networking) but Rust comes with the added boost of memory safety. We are happy to report that Rust is being adopted in security critical parts of Android, such as our key management components and networking stacks. Hardening the platform doesn’t just stop with continual improvements with memory safety and expansion of anti-exploitation techniques. It also includes hardening our API surfaces to provide a more secure experience to our end users. In Android 13 we implemented numerous enhancements to help mitigate potential vulnerabilities that app developers may inadvertently introduce. This includes making runtime receivers safer by allowing developers to specify whether a particular broadcast receiver in their app should be exported and visible to other apps on the device. On top of this, intent filters block non-matching intents which further hardens the app and its components. For enterprise customers who need to meet certain security certification requirements, we’ve updated our security logging reporting to add more coverage and consolidate security logs in one location. This is helpful for companies that need to meet standards like Common Criteria and is useful for partners such as management solutions providers who can review all security-related logs in one place. Privacy on your terms Android 13 brings developers more ways to build privacy-centric apps. Apps can now implement a new Photo picker that allows the user to select the exact photos or videos they want to share without having to give another app access to their media library.

With Android 13, we’re also reducing the number of apps that require your location to function using the nearby devices permission introduced last year. For example, you won’t have to turn on location to enable Wi-fi for certain apps and situations. We’ve also changed how storage works, requiring developers to ask for separate permissions to access audio, image and video files. Previously, we’ve limited apps from accessing your clipboard in the background and alerted you when an app accessed it. With Android 13, we’re automatically deleting your clipboard history after a short period so apps are blocked from seeing old copied information. In Android 11, we began automatically resetting permissions for apps you haven’t used for an extended period of time, and have since expanded the feature to devices running Android 6 and above. Since then, we’ve automatically reset over 5 billion permissions. In Android 13, app makers can go above and beyond in removing permissions even more proactively on behalf of their users. Developers will be able to provide even more privacy by reducing the time their apps have access to unneeded permissions. Finally, we know notifications are critical for many apps but are not always of equal importance to users. In Android 13, you’ll have more control over which apps you would like to get alerts from, as new apps on your device are required to ask you for permission by default before they can send you notifications. Apps you can trust Most app developers build their apps using a variety of software development kits (SDKs) that bundle in pre-packaged functionality. While SDKs provide amazing functionality, app developers typically have little visibility or control over the SDK code or insight into their performance. We’re working with developers to make their apps more secure with a new Google Play SDK Index that helps them see SDK safety and reliability signals before they build the code into their apps. This ensures we're helping everyone build a more secure and private app ecosystem. Last month, we also started rolling out a new Data safety section in Google Play to help you understand how apps plan to collect, share, and protect your data, before you install it. To instill even more trust in Play apps, we're enabling developers to have their apps independently validated against OWASP’s MASVS, a globally recognized standard for mobile app security.

We’re working with a small group of developers and authorized lab partners to evolve the program. Developers who have completed this independent validation can showcase this on their Data safety section. Additional mobile security and safety Just like our anti-malware protection Google Play, which now scans 125 billion apps a day, we believe spam and phishing detection should be built in. We’re proud to announce that in a recent analyst report, Messages was the highest rated built-in messaging app for anti-phishing and scams protection. Messages is now also helping to protect you against 1.5 billion spam messages per month, so you can avoid both annoying texts and attempts to access your data. These phishing attempts are increasingly how bad actors are trying to get your information, by getting you to click on a link or download an app, so we are always looking for ways to offer another line of defense.

Last year, we introduced end-to-end encryption in Messages to provide more security for your mobile conversations. Later this year, we’ll launch end-to-end encryption group conversations in beta to ensure your personal messages get even more protection. As with a lot of features we build, we try to do it in an open and transparent way. In Android 11 we announced a new platform feature that was backed by an ISO standard to enable the use of digital IDs on a smartphone in a privacy-preserving way. When you hand over your plastic license (or other credential) to someone for verification it’s all or nothing which means they have access to your full name, date of birth, address, and other personally identifiable information (PII). The mobile version of this allows for much more fine-grained control where the end user and/or app can select exactly what to share with the verifier. In addition, the verifier must declare whether they intend to retain the data returned. In addition, you can present certain details of your credentials, such as age, without revealing your identity. Over the last two Android releases we have been improving this API and making it easier for third-party organizations to leverage it for various digital identity use cases, such as driver’s licenses, student IDs, or corporate badges. We’re now announcing that Google Wallet uses Android Identity Credential to support digital IDs and driver’s licenses. We’re working with states in the US and governments around the world to bring digital IDs to Wallet later this year. You can learn more about all of the new enhancements in Google Wallet here. Protected by Android We don’t think your security and privacy should be hard to understand and control. Later this year, we’ll begin rolling out a new destination in settings on Android 13 devices that puts all your device security and data privacy front and center.

The new Security & Privacy settings page will give you a simple, color-coded way to understand your safety status and will offer clear and actionable guidance to improve it. The page will be anchored by new action cards that notify you of critical steps you should take to address any safety risks. In addition to notifications to warn you about issues, we’ll also provide timely recommendations on how to enhance your privacy. We know that to feel safe and in control of your data, you need to have a secure foundation you can count on. Because if your device isn’t secure, it’s not private either. We’re working hard to make sure you’re always protected by Android. Learn more about these protections on our website.

Taking on the Next Generation of Phishing Scams

May 11, 2022

Posted by Daniel Margolis, Software Engineer, Google Account Security Teambrowsers get betterencryption becomes ubiquitous on the Weba recent phishing attack on the U.S. Department of Laborat I/O we announced

Left: legitimate Google SMS verification. Right: spoofed message asking victim to share verification code.

In other cases, attackers have leveraged more sophisticated dynamic phishing pages to conduct relay attacks. In these attacks, a user thinks they're logging into the intended site, just as in a standard phishing attack. But instead of deploying a simple static phishing page that saves the victim's email and password when the victim tries to login, the phisher has deployed a web service that logs into the actual website at the same time the user is falling for the phishing page.

The simplest approach is an almost off-the-shelf "reverse proxy" which acts as a "person in the middle", forwarding the victim's inputs to the legitimate page and sending the response from the legitimate page back to the victim's browser.

The Solution Space

Over the past year, we've started to automatically enable device-based two-factor authentication for our users. This authentication not only helps protect against traditional password compromise but, with technology improvements, we can also use it to help defend against these more sophisticated forms of phishing.

Taking a broad view, most efforts to protect and defend against phishing fall into the following categories:

Browser UI improvements to help users identify authentic websites.

Password managers that can validate the identity of the web page before logging in.

Phishing detection, both in email—the most common delivery channel—and in the browser itself, to warn users about suspicious web pages.

Preventing the person-in-the-middle attacks mentioned above by preventing automated login attempts.

Phishing-resistant authentication using FIDO with security keys or a Bluetooth connection to your phone.

Hardening the Google Prompt challenge to help users identify suspicious sign-in attempts, or to ask them to take additional steps that can defeat phishing (like navigating to a new web address, or to join the same wireless network as the computer they're logging into).

Expanding phishing-resistant authentication to more usersFIDO AllianceTitan Security Keyannounced

But don't worry

Hardening existing challenges against phishin

Bringing it all together

The Package Analysis Project: Scalable detection of malicious open source packages

April 28, 2022

Posted by Caleb Brown, Open Source Security Team
Despite open source software’s essential role in all software built today, it’s far too easy for bad actors to circulate malicious packages that attack the systems and users running that software. Unlike mobile app stores that can scan for and reject malicious contributions, package repositories have limited resources to review the thousands of daily updates and must maintain an open model where anyone can freely contribute. As a result, malicious packages like ua-parser-js, and node-ipc are regularly uploaded to popular repositories despite their best efforts, with sometimes devastating consequences for users.

Google, a member of the Open Source Security Foundation (OpenSSF), is proud to support the OpenSSF’s Package Analysis project, which is a welcome step toward helping secure the open source packages we all depend on. The Package Analysis program performs dynamic analysis of all packages uploaded to popular open source repositories and catalogs the results in a BigQuery table. By detecting malicious activities and alerting consumers to suspicious behavior before they select packages, this program contributes to a more secure software supply chain and greater trust in open source software. The program also gives insight into the types of malicious packages that are most common at any given time, which can guide decisions about how to better protect the ecosystem.

To better understand how the Package Analysis program is contributing to supply chain security, we analyzed the nearly 200 malicious packages it captured over a one-month period. Here’s what we discovered:
Results
All signals collected are published in our BigQuery table. Using simple queries on this table, we found around 200 meaningful results from the packages uploaded to NPM and PyPI in a period of just over a month. Here are some notable examples, with more available in the repository.
PyPI: discordcmdThis Python package will attack the desktop client for Discord on Windows. It was found by spotting the unusual requests to raw.githubusercontent.com, Discord API, and ipinfo.io.

First, it downloaded a backdoor from GitHub and installed it into the Discord electron client.

Next, it looked through various local databases for the user's Discord token.

Finally, it grabbed the data associated with the token from the Discord API and exfiltrated it back to a Discord server controlled by the attacker.
NPM: @roku-web-core/ajax
During install, this NPM package exfiltrates details of the machine it is running on and then opens a reverse shell, allowing the remote execution of commands.

This package was discovered from its requests to an attacker-controlled address.

Dependency Confusion / Typosquatting
The vast majority of the malicious packages we detected are dependency confusion and typosquatting attacks.

The packages we found usually contain a simple script that runs during an install and calls home with a few details about the host. These packages are most likely the work of security researchers looking for bug bounties, since most are not exfiltrating meaningful data except the name of the machine or a username, and they make no attempt to disguise their behavior.

These dependency confusion attacks were discovered through the domains they used, such as burpcollaborator.net, pipedream.com, interact.sh, which are commonly used for reporting back attacks. The same domains appear across unrelated packages and have no apparent connection to the packages themselves. Many packages also used unusual version numbers that were high (e.g. v5.0.0, v99.10.9) for a package with no previous versions.

Conclusions
The short time frame and low sophistication needed for finding the results above underscore the challenge facing open source package repositories. While many of the results above were likely the work of security researchers, any one of these packages could have done far more to hurt the unfortunate victims who installed them.

These results show the clear need for more investment in vetting packages being published in order to keep users safe. This is a growing space, and having an open standard for reporting would help centralize analysis results and offer consumers a trusted place to assess the packages they’re considering using. Creating an open standard should also foster healthy competition, promote integration, and raise the overall security of open source packages.
Over time we hope that the Package Analysis program will offer comprehensive knowledge about the behavior and capabilities of packages across open source software, and help guide the future efforts needed to make the ecosystem more secure for everyone. To get involved, please check out the GitHub Project and Milestones for opportunities to contribute.

How we fought bad apps and developers in 2021

April 27, 2022

Posted by Steve Kafka and Khawaja Shams, Android Security and Privacy Team Providing a safe experience to billions of users continues to be one of the highest priorities for Google Play. Last year we introduced multiple privacy focused features, enhanced our protections against bad apps and developers, and improved SDK data safety. In addition, Google Play Protect continues to scan billions of installed apps each day across billions of devices to keep people safe from malware and unwanted software. We continue to enhance our machine learning systems and review processes, and in 2021 we blocked 1.2 million policy violating apps from being published on Google Play, preventing billions of harmful installations. We also continued in our efforts to combat malicious and spammy developers, banning 190k bad accounts in 2021. In addition, we have closed around 500k developer accounts that are inactive or abandoned. In May we announced our new Data safety section for Google Play where developers will be required to give users deeper insight into the privacy and security practices of the apps they download, and provide transparency into the data the app may collect and why. The Data safety section launched this week, and developers are required to complete this section for their apps by July 20th. We’ve also invested in making life easier for our developers. We added the Policy and Programs section to Google Play Console to help developers manage all their app compliance issues in one central location. This includes the ability to appeal a decision and track its status from this page. In addition, we continued to partner with SDK developers to improve app safety, limit how user data is shared, and improve lines of communication with app developers. SDKs provide functionality for app developers, but it can sometimes be tricky to know when an SDK is safe to use. Last year, we engaged with SDK developers to build a safer Android and Google Play ecosystem. As a result of this work, SDK developers have improved the safety of SDKs used by hundreds of thousands of apps impacting billions of users. This remains a huge investment area for our team, and we will continue in our efforts to make SDKs safer across the ecosystem. Limiting access The best way to ensure users' data stays safe is to limit access to it in the first place. As a result of new platform protections and policies, developer collaboration and education, 98% of apps migrating to Android 11 or higher have reduced their access to sensitive APIs and user data. We've also significantly reduced the unnecessary, dangerous, or disallowed use of Accessibility APIs in apps migrating to Android 12, while preserving the functionality of legitimate use cases. We also continued in our commitment to make Android a great place for families. Last year we disallowed the collection of Advertising ID (AAID) and other device identifiers from all users in apps solely targeting children, and gave all users the ability to delete their Advertising ID entirely, regardless of the app. Pixel enhancements For Pixel users, we had even more great features to help keep you safe. Our new Security hub helps protect your phone, apps, Google Account, and passwords by giving you a central view of your device’s current configuration. Security hub also provides recommendations to improve your security, helping you decide what settings best meet your needs. In addition, Pixels now use new machine learning models that improve the detection of malware in Google Play Protect. The detection runs on your Pixel, and uses a privacy preserving technology called federated analytics to discover bad apps. Our global teams are dedicated to keeping our billions of users safe, and look forward to many exciting announcements in 2022.

How to SLSA Part 3 - Putting it all together

April 14, 2022

Posted by Tom Hennen, software engineer, BCID & GOSST
12

The Squirrel package ‘foo’

The Oppy package ‘baz’

A custom executable, ‘bar’, written by Acme employees.

source attestation

bar is compiled from source code stored in the same source repo as the Dockerfile.

acorn install downloads ‘foo’ from the Squirrel repo, verifying the VSA, and recording the use of acorn://foo@abc and its VSA in the build.

acme_oppy_get install (a custom script made by Acme) downloads the latest version of the Oppy ‘baz’ package and queries its SLSA provenance and other attestations from Rekor. It then performs a full verification checking that it was built by ‘https://oppy.example/slsa/builder/v1’ and the publicized key. Once verification is complete it records the use of oppy://baz@def and the associated attestations in the build.

The build process assembles the SLSA provenance for the container by:

Recording the Acme git repo the bar source and Dockerfile came from, into materials.
Copying the reported dependencies of acorn://foo@abc and oppy://baz@def into materials and adding their attestations to the output in-toto bundle.
Recording the CI/CD entrypoint as the invocation.
Creating a signed DSSE with the SLSA provenance and adding it to the output in-toto bundle.

SBOMthe threats affecting the software supply chain todayShow us

How to SLSA Part 2 - The Details

April 13, 2022

Posted by Tom Hennen, software engineer, BCID & GOSST
In our last post we introduced a fictional example of Squirrel, Oppy, and Acme learning to use SLSA and covered the basics of what their implementations might look like. Today we’ll cover the details: where to store attestations and policies, what policies should check, and how to handle key distribution and trust.

Attestation storage

Attestations play a large role in SLSA and it’s essential that consumers of artifacts know where to find the attestations for those artifacts.

Co-located in repo

Attestations could be colocated in the repository that hosts the artifact. This is how Squirrel plans to store attestations for packages. They even want to add support to the Squirrel CLI (e.g. acorn get-attestations foo@1.2.3).

Acme really likes this approach because the attestations are always available and it doesn’t introduce any new dependencies.
Rekor
Meanwhile, Oppy plans to store attestations in Rekor. They like being able to direct users to an existing public instance while not having to maintain any new infrastructure themselves, and the in-depth defense the transparency log provides against tampering with the attestations.

Though the latency of querying attestations from Rekor is likely too high for doing verification at time of use, Oppy isn’t too concerned since they expect users to query Rekor at install time.

Hybrid

A hybrid model is also available where the publisher stores the attestations in Rekor as well as co-located with the artifact in the repo—along with Rekor’s inclusion proof. This provides confidence the data was added to Rekor while providing the benefits of co-locating attestations in the repository.

Policy content

‘Policy’ refers to the rules used to determine if an artifact should be allowed for a use case.

Policies often use the package name as a proxy for determining the use case. An example being, if you want to find the policy to apply you could look up the policy using the package name of the artifact you’re evaluating.

Policy specifics may vary based on ease of use, availability of data, risk tolerance and more. Full verification needs more from policies than delegated verification does.

Default policy

Default policies allow admission decisions without the need to create specific policies for each package. A default policy is a way of saying “anything that doesn’t have a more specific policy must comply with this policy”.

Squirrel plans to eventually implement a default policy of “any package without a more specific policy will be accepted as long as it meets SLSA 3”, but they recognize that most packages don’t support this yet. Until they achieve critical mass they’ll have a default SLSA 0 policy (all artifacts are accepted).

While Oppy is leaving verification to their users, they’ll suggest a default policy of “any package built by ‘https://oppy.example/slsa/builder/v1’”.

Specific policy

Squirrel also plans to allow users to create policies for specific packages. For example, this policy requires that package ‘foo’ must have been built by GitHub Actions, from github.com/foo/acorn-foo, and be SLSA 4.

scope: 'acorn://foo'

target_level: SLSA_L4

allow_github_actions {

workflow: 'https://github.com/gossts/slsa-acorn/.github/workflows/builder.yml@main'

source_repo: 'https://github.com/foo/acorn-foo.git'

allow_branch: 'main'

}

Squirrel will also allow packages to create SLSA 0 policies if they’re not using SLSA compliant infrastructure.

scope: 'acorn://qux'

target_level: SLSA_L0

Policy auto generation
Squirrel has an enormous number of existing packages. It’s not feasible to get all those package maintainers to create specific policies themselves. Therefore, Squirrel plans to leverage process mining to auto generate policies for packages based on the history of the package. E.g. “The last 10 times Squirrel package foo was published it was built by GitHub Actions from github.com/foo/acorn-foo, and met SLSA 4 (this is the policy above). Let’s create a policy that requires that and send it to the maintainers to review.”
Policy add-ons

Policy evaluation could do more than just evaluate the SLSA requirements. The same policies that check SLSA requirements are well placed to check other properties that are important to organizations like “was static analysis performed”, “are there any known CVEs in this artifact”, “was integration testing successful”, etc…

Acme is really interested in some of these policy add-ons. They’d like to avoid the embarrassing situation of publishing a new container image with known CVEs. They’re not sure how to implement it yet but they’ll be on the lookout for tools that can help them do so.

Delegated policies

When using delegated verification there’s much less that actually needs to be checked and they can be hard-coded directly in tooling. A minimal delegated verification policy might be “allow if trusted-party verified this artifact (identified by digest) as <package name>”. This can be tightened further by adding requirements on the artifact & its dependencies SLSA levels (data which is available in the VSA). For example, “allow if trusted-party verified this artifact as <package name> at SLSA 3 and it doesn’t have any dependencies less than SLSA 2”.

# Delegated verification implicitly checks that the package name we're

# checking matches the VSA's subject.name field.

allow_delegated_verification {

trusted_verifier: 'https://delegatedverifier.com/slsa/v1'

minimum_level: SLSA_L3

minimum_dependency_level: SLSA_L2

}

Policy storage

When using specific, non-default, policies verifiers need to know where to find the policy they need to evaluate.

Co-located in repo

Squirrel plans to store specific policies as a property of the package in the repository. This makes them very easy for users and their tooling to find. It also allows the maintainer of the package to easily set the policy (they already have write permissions!).

A potential downside is that the write permissions are the same as for the package itself. An attacker that compromises the developer’s credentials could also change the policy. This may not be as bad as it seems. Policies are human-readable so anyone paying attention would notice that package foo’s policy now says that it can be built from github.com/not-foo/acorn-foo. Squirrel plans to notify interested parties (including the maintainer!) when the policy changes, potentially letting them “sound the alarm” if anything nefarious happens.

A similar approach is taken in a number of contact-change workflows. For example, when you change your address with your bank, the bank will send you an email (and a letter to the old address) letting you know the address has been changed. This type of notification would alert the maintainer to a potential compromise.

Squirrel would also consider requiring a second person to review any policy changes for packages with over 10,000 users.
Public canonical Git repo

Another option might be to just create a canonical git repo (e.g. github.com/slsa-framework/slsa-acorn-policies) and let people publish proposed policies there. This has the advantage of using a separate ACL control mechanism from the package repository itself, but the disadvantages of being difficult to ensure the author of the policy is actually allowed to set the policy for that package and not scaling well as the repo grows.

The approach outlined in policy auto generation could help here. Automation in the repo could just look at the last N releases of the package and determine if the proposed policy matches what’s actually been published. Proactive changes to the policy (like deciding to switch from GitHub Actions to CircleCI) would be harder to coordinate however.
Org specific repo

Acme plans to establish their own org specific repo for policy storage. This gives them a single place to store all their policies, regardless of ecosystem type, and lets them provide more specific policies for packages provided by upstream repos. Since Oppy doesn’t have any plans to provide package-specific policies this gives Acme a place to store their own policies for Oppy packages (if they ever get around to it).

Organizations can also use their policy repo to vet any upstream changes to policy and potentially add additional checks (e.g. “doesn’t have any known vulnerabilities”).

Trusted Verifier

Acme wants to use delegated verification and that relies on having trusted verifiers to make decisions for downstream users. Who are these trusted verifiers?

Public verifier

A public repo is in a great position to act as a trusted verifier for their users. Users already trust these repos and they may already be doing verification on import.

Squirrel plans to make use of this by making VSAs available for each artifact published, publicizing their verifier ID (i.e. ‘https://squirrel.example/slsa-verifier’) and the public key used to sign the VSAs. They even plan to build VSA verification directly into the Squirrel tooling, so that users can get SLSA protection by default.
Org-wide verifier

While Acme is happy to use Squirrel’s verifier (and the verification built into the tooling) they still need their own verifier so they can publish VSAs to Acme customers. So Acme plans to stand up their own verification service and publish their verifier ID (i.e. ‘https://acme.example/private-verifier’) and signing key. Acme customers can then verify the software they get from Acme.

In the future Acme could require all software used throughout the company to be verified with this verifier (instead of relying on public verifiers). They’d do the verification and generate VSAs whenever artifacts are imported into their private Artifactory instance. They could then configure this ID/key pair for use throughout Acme and be confident that any software used has been verified according to Acme policy. That’s not Acme’s highest priority at the moment, but they like having this option open to them.

Key distribution & Trust

Both full and delegated verification depend upon key distribution to the users doing the verification. Depending on the specifics and what’s getting verified this can be a difficult problem.

Org-specific keys

When using delegated verification this could be the easiest case. Squirrel can just build the key they used for delegated verification directly into the Squirrel tooling. Acme can also fairly easily configure the use of their keys through the company using existing configuration control mechanisms.

When using full verification this can be harder. If there are multiple builders that could be accepted the keys that sign the attestations need to be distributed to everyone that might use that builder. For Squirrel this would be really difficult since they plan to allow package maintainers to use whatever builder they want. How those keys get configured would be tricky just for Squirrel, and much more difficult if downstream Squirrel users wanted to do full verification of the Squirrel packages.

The situation is easier, however, for Oppy. That’s because Oppy plans to only accept artifacts built by their autobuilder network. Oppy can configure this network to use a single (or small set) of keys and then publish those keys (and the SLSA level Oppy believes it meets) for downstream users.

Fulcio

Squirrel plans to solve the problem of which keys they accept by leveraging Fulcio. Squirrel will build support for Fulcio root keys into their verifier and then express which Fulcio subject is allowed to sign attestations in the specific policy of each package. E.g. “Squirrel package ‘foo’ must have been built & signed by ‘spiffe://foobar.com/foo-builder, from github.com/foo/acorn-foo, and be SLSA 4”.

scope: 'acorn://foo'

target_level: SLSA_L4

allow_fulcio_builder {

id: 'spiffe://foobar.com/foo-builder'

source_repo: 'https://github.com/foo/acorn-foo.git'

allow_branch: 'main'

allow_entrypoint: 'package.json'

}

The Update Framework (TUF)

The above methods could be further enhanced with TUF to allow the secure maintenance of keys. TUF metadata could include all the SLSA keys, the build services and other entities they’re valid for, and the SLSA levels they’re qualified at. Oppy is considering using TUF to let verifiers securely fetch and update keys used by the Autobuilder network. Oppy would use a TUF delegation to indicate that these keys should only be used for the builder id ‘https://oppy.example/slsa/builder/v1’. Squirrel might do something similar to allow for updating the Fulcio key in its tooling.

Recording & verifying dependencies

Acme wants to record and verify the dependencies that go into its container into the SLSA provenance. Acme would prefer that this functionality were just built-in their build service, but that feature isn’t available yet. Instead they’ll need to do something themselves. They have a few options at their disposal:

Tool wrappers

Since Oppy doesn’t build SLSA into it’s tooling Acme will create wrapper scripts for dependency import/installation that record and verify (using cosign) dependencies as they’re installed. Acme will update their build scripts to replace all instances of Oppy package installation with the wrapper script and then use the recorded results to help populate the materials section of the provenance.

A downside is that this approach, if run in the build itself, is not guaranteed to be complete and cannot meet the “non-falsifiable” requirement (since the results reported by the wrapper could be falsified by the build process), relegating this approach to SLSA 2. Still, it allows Acme to make progress SLSA-fying their builds and provides a starting point for achieving higher SLSA levels.
Built into ecosystem tooling

Since Squirrel does build verification into their tooling, Acme can just use acorn install to verify the dependencies and record what was installed. Acme can use this information to populate the Squirrel packages installed in the materials section of the provenance and it can include the attestations of those dependencies in the in-toto bundle for their container image.

As with tool wrappers, if this method is used in the build itself it cannot meet “non-falsifiable” requirement.

Proxied verification

Acme considered creating a proxy for their existing builder to proxy outbound connections. This proxy could verify everything fetched and use its logs to populate the provenance. Since this proxy is trusted it would be easier to meet “non-falsifiable” requirement. Unfortunately it’s also a lot of work for Acme so they’re going to defer this idea for now.

Next time

In the first two parts of this series, we’ve covered the basics of getting started with SLSA and the details of policy and provenance storage, policy verification, and key handling. In our next post we’ll cover how Squirrel, Oppy, and Acme put this all together to protect a heterogeneous supply chain.

How to SLSA Part 1 - The Basics

April 12, 2022

Posted by Tom Hennen, Software Engineer, BCID & GOSST
Supply-chain Levels for Software Artifacts

How and when do you verify a package with SLSA?

How to handle artifacts without provenance?

Where is the provenance stored?

Where is the appropriate policy stored and who should verify it?

What should the policies check?

How do you establish trust & distribute keys?

What does a secure, heterogeneous supply chain look like?

The Situation

The Squirrel package ‘foo’

The Oppy package ‘baz’

A custom executable, ‘bar’, written by Acme employees

Basics

In order to SLSA, Squirrel, Oppy, and Acme will all need SLSA capable build services. Squirrel wants to give their maintainers wide latitude to pick a builder service of their own. To support this, Squirrel will qualify some build services at specific SLSA levels (meaning they can produce artifacts up to that level). To start, Squirrel plans to qualify GitHub Actions using an approach like this, and hopes it can achieve SLSA 4 (pending the result of an independent audit). They’re also willing to qualify other build services as needed. Oppy on the other hand, doesn’t need to support arbitrary build services. They plan to have everyone use their Autobuilder network which they hope to qualify at SLSA 4 (they’ll conduct the audit/certification themselves). Finally, Acme plans to use Google Cloud Build which they’ll self-certify at SLSA 4 (pending the result of a Google-conducted audit).

Squirrel, Oppy, and Acme will follow a similar qualification process for the source control systems they plan to support.

Verification options

Full verification

At some point, one or more of the organizations will need to do full verification of each artifact to determine if it is acceptable for a given use case. This is accomplished by checking if the artifact meets the appropriate policy.

Typically, full verification would take place with SLSA provenance, source attestations, and perhaps other specialized attestations (like vulnerability scan results). While having to coordinate this data for all of its dependencies seems like a lot of work to Acme, they’re prepared to do full verification if Squirrel and Oppy are unable to.
Delegated verification

When Acme isn’t using full verification, they can instead use delegated verification where they check if an artifact is acceptable for a use case by checking if some other trusted party who performed a full verification (such as Squirrel or Oppy) believes the artifact is acceptable.

Delegated verification is easier to perform quickly with limited data and network connectivity. It may also be easier for some users who value if someone they trust verified the artifact is good.

Squirrel likes how easy delegated verification would make things for their users and plans to support it by creating a Verification Summary Attestation (VSA) when they perform full verification.

When to verify

Verification (full or delegated) could happen at a number of different times.

On import to repo

Squirrel plans to perform full verification when an artifact is published to their repo. This will ensure that packages in the repo have met their corresponding policy. It’s also helpful because all the required data can be gathered when latency isn’t critical.

If this were the only time verification is performed, it would put the repository's storage in the trusted computing base (TCB) of its users. Squirrel’s plans to use delegated verification (and issue VSAs) can prevent this. The signature on the VSA will prevent the artifacts from being tampered with while sitting in storage, even if they’re just SLSA 0. Downstream users will just need to verify the VSA.

Acme also wants to do some sort of verification on the import to their internal repo since it simplifies their security story. They’re not quite sure what this will look like yet.

On install

Acme also wants to do verification when an artifact is actually installed since it can remove a number of intermediaries from their TCB (their repo, the network, upstream storage systems).

If they perform full verification at install then they must gather all the required information. That could be a lot of data, but it might be simplified by gathering the data from external sources and caching it in their internal repo. A larger problem is that it requires Acme to have established trust in all parties that produced that information (e.g. every builder of every package). For a complex supply chain that may be difficult.

If Acme performs delegated verification, they only need the VSA for the packages being installed and to explicitly trust a handful of parties. This allows the complex full verification to be performed once while allowing all users of that package to perform a much simpler operation.

Given these tradeoffs Acme prefers delegated verification at install time. Squirrel also really likes the idea and plans to build install time verification directly into the Squirrel tool.
On use
Verification could also take place each time an artifact is actually used. In this model, latency and reliability are very important (a sudden increase in site traffic may necessitate a scaling operation launching many new jobs).

Time of use verification allows the most context with which decisions can be made (“is this job allowed to run this code and is it free from vulns right now?”). It also allows policy changes to affect already built & installed software (which may or may not be desirable).

Acme wants their users to be able to verify on use without too many dependencies so they plan to provide VSAs users can use to perform delegated verification when they start the container (perhaps using something like Kyverno).

How to handle artifacts without provenance?

Inevitably a build or system may require that an artifact without ‘original’ provenance is used. In these cases it may be desirable for the importer to generate provenance that details where it got this artifact. For example, this generated provenance shows that http://example.com/foo.tgz with sha256:abc was imported by ‘auto-importer’:

Such an artifact would likely not be accepted at higher SLSA levels, but the provenance can be used to: 1) prevent tampering with the artifact after it’s been imported and 2) be a data point for future analysis (e.g. should we prioritize asking for foo.tgz to be distributed with native SLSA provenance?).

Acme might be interested in taking this approach at some point, but they don’t need it at the moment.

Next time

In our next post we’ll cover specific approaches that can be used to answer questions like “where should attestations and policies be stored?” and “how do I trust the attestations that I receive?”