Continuum AI is now public. Try out the first confidential LLM platform!
Blog
Otto Bittner
Apple recently introduced "Private Cloud Compute" (PCC), a cloud-based system aimed at ensuring privacy while using AI.
LLMs and GenAI apps like Siri, ChatGPT, and other personal assistants handle sensitive data, combining information that was previously disaggregated. This consolidation increases privacy risks but is necessary for LLMs to be effective. Moreover, sophisticated requests require larger models that must be run in the cloud. PCC aims to maintain data privacy and security for LLM requests that have to be handled in the cloud.
In this blog post, we will explore how Apple wants to match the stringent on-device privacy standards when sending user data to their servers. Subsequently, we explain how you can do the same.
In their announcement, Apple explains how PCC aims to achieve the same security standards for cloud data as for data processed on Apple devices. This is difficult because traditionally, processing data always meant having access to it in clear text. Let's look at the design primitives that Apple used for PCC to achieve this:
This system aims to secure data on the cloud for the Apple ecosystem. But how can non-Apple users gain this privacy, already now, and at scale?
PCC is enabled by confidential computing, a technology we at Edgeless Systems have been focusing on for quite some time. Confidential computing ensures that data is always encrypted, even at runtime. Unlike alternative encryption technology, it only incurs low overhead, and already works in production, protecting virtually any type of workload. Confidential computing features have already been available in CPUs from Intel and AMD for multiple years. Recently, Nvidia introduced this technology to H100 GPUs, enabling this technology for AI applications.
We have developed Continuum AI, a platform designed to securely deploy AI models, by leveraging the aforementioned Nvidia GPUs. Read more on the collaboration on Nvidia’s blog post. Continuum AI ensures that user prompts and responses, remain encrypted and shielded from model owners and infrastructure providers. Visit our confidential computing wiki to learn more about the technology.
Now we will see how Continuum AI, our LLM framework predating PCC, offers an alternative that is production-ready and will be released as open source in H2/2024.
Continuum AI is a platform that hosts any model that can be deployed as a container. It transparently adds the privacy guarantees discussed for PCC to your workload. Key Continuum features include:
If PCC interests you, Continuum is the open alternative. It uses the same underlying technology — confidential computing, to protect your AI data from the infrastructure, the model owner, the service provider, and others, already today.
With Continuum AI, you can deploy any AI solution with privacy standards like Apple’s PCC. Leveraging Nvidia’s GPUs, we provide remote attestation, encrypted communication, and memory isolation. Your data is always processed within a confidential-computing environment, ensuring the highest level of security for businesses.
As Apple invests in confidential computing, it marks a significant advancement for the industry, and it will push other companies to do the same. In the meantime, get started with Continuum! Try out our public preview and chat with encrypted prompts here. You can also join the waitlist on the same page to get enterprise access, or contact our experts to talk about Confidential AI!