Apple Private Cloud Compute: core concepts and an open alternative

Apple recently introduced "Private Cloud Compute" (PCC), a cloud-based system aimed at ensuring privacy while using AI.

LLMs and GenAI apps like Siri, ChatGPT, and other personal assistants handle sensitive data, combining information that was previously disaggregated. This consolidation increases privacy risks but is necessary for LLMs to be effective. Moreover, sophisticated requests require larger models that must be run in the cloud. PCC aims to maintain data privacy and security for LLM requests that have to be handled in the cloud.

In this blog post, we will explore how Apple wants to match the stringent on-device privacy standards when sending user data to their servers. Subsequently, we explain how you can do the same.

Apple's goals with PCC

In their announcement, Apple explains how PCC aims to achieve the same security standards for cloud data as for data processed on Apple devices. This is difficult because traditionally, processing data always meant having access to it in clear text. Let's look at the design primitives that Apple used for PCC to achieve this:

• Stateless computation: User data is used only for fulfilling user requests, inaccessible to Apple staff, and not retained after response delivery. PCC relies on custom silicon deployed in their own data centers. This silicon encrypts data during processing, making statelessness meaningful by ensuring data is inaccessible during processing.
• Enforceable guarantees: A minimal set of components should be responsible for guaranteeing the overall system’s security. Minimizing the number of components makes checking the security of the system possible.  
• No privileged runtime access: Apple staff must not be able to extend their privilege, even when dealing with an outage or severe incident.
• Non-targetability: Even if an attacker compromises a server, they cannot steer a specific user's traffic to it. This ensures that a subsystem's exploitation won't compromise the whole privacy of a user.
• Provide verifiable transparency: Notably, PCC hardware is set to be verifiably transparent. This means, that not only do multiple Apple teams cross-check the components, but they are also monitored by a third party not affiliated with Apple. Crucially, Apple plans to release most binary artifacts code for scrutiny.

This system aims to secure data on the cloud for the Apple ecosystem. But how can non-Apple users gain this privacy, already now, and at scale?

The future of handling sensitive data: Confidential AI

 
PCC is enabled by confidential computing, a technology we at Edgeless Systems have been focusing on for quite some time. Confidential computing ensures that data is always encrypted, even at runtime. Unlike alternative encryption technology, it only incurs low overhead, and already works in production, protecting virtually any type of workload. Confidential computing features have already been available in CPUs from Intel and AMD for multiple years. Recently, Nvidia introduced this technology to H100 GPUs, enabling this technology for AI applications.

We have developed Continuum AI, a platform designed to securely deploy AI models, by leveraging the aforementioned Nvidia GPUs. Read more on the collaboration on Nvidia’s blog post. Continuum AI ensures that user prompts and responses, remain encrypted and shielded from model owners and infrastructure providers. Visit our confidential computing wiki to learn more about the technology.

Now we will see how Continuum AI, our LLM framework predating PCC, offers an alternative that is production-ready and will be released as open source in H2/2024.

Understanding Continuum AI

Continuum AI is a platform that hosts any model that can be deployed as a container. It transparently adds the privacy guarantees discussed for PCC to your workload. Key Continuum features include:

• Hardware-backed isolation and runtime encryption (stateless computation): Like PCC, Continuum provides encryption of data in-use. It uses AMD SEV-SNP technology, securing data within Confidential VMs (CVMs). SEV-SNP encrypts the VM memory. This ensures data confidentiality and integrity, even if the hypervisor or other VMs are compromised.
• Prompt encryption (enforceable guarantees). When a user sends a prompt, Continuum encrypts the message before sending it to the provider. Data is decrypted for processing only within encrypted Continuum workers. The results are encrypted again before sending them back to the user. We use Authenticated Encryption through AES-GCM for this.
• Sandboxing (enforceable guarantees): Like PCC, Continuum leverages sandboxing. The inference code, sourced externally, undergoes frequent updates, making regular remote attestation and reviews impractical. The code runs within a confidential computing environment, accessing user data. To prevent data leaks, it operates in a sandbox within this environment, isolated from the host via gVisor.
• Remote attestation (verifiable transparency): Continuum verifies that CVMs are running on trusted hardware before allowing access. Measured boot ensures the integrity of the boot process, making sure any changes are detectable.
• No one can access your data (no privileged runtime access): Like PCC, our software doesn’t allow privileged roles, ensuring that even cloud admins cannot bypass security measures.
• Scalability and compatibility: In contrast to PCC, Continuum is compatible with diverse deployment scenarios and highly scalable. 

Leverage Confidential AI yourself already today

If PCC interests you, Continuum is the open alternative. It uses the same underlying technology — confidential computing, to protect your AI data from the infrastructure, the model owner, the service provider, and others, already today.

With Continuum AI, you can deploy any AI solution with privacy standards like Apple’s PCC. Leveraging Nvidia’s GPUs, we provide remote attestation, encrypted communication, and memory isolation. Your data is always processed within a confidential-computing environment, ensuring the highest level of security for businesses.

As Apple invests in confidential computing, it marks a significant advancement for the industry, and it will push other companies to do the same. In the meantime, get started with Continuum! Try out our public preview and chat with encrypted prompts here. You can also join the waitlist on the same page to get enterprise access, or contact our experts to talk about Confidential AI!

Author: Otto Bittner