Continuum AI: the first confidential LLM platform

Edgeless Systems to launch first confidential LLM platform Continuum with NVIDIA H100 Tensor Core GPUs

Bochum, Germany — July 1, 2024

Edgeless Systems has launched Continuum, an LLM platform that leverages confidential computing to keep user prompts verifiably protected, unlocking the benefits of generative AI for sensitive enterprise data.

AI is becoming ubiquitous and large language models (LLMs) like ChatGPT are elevating the productivity of organizations worldwide. Despite the known benefits of LLMs, concerns over data leaks can be barriers to adoption, especially in enterprise contexts.

Confidential computing is a computing paradigm that keeps data encrypted and protected throughout – even during processing. Further, confidential computing makes it possible to verify the integrity of workloads.

Edgeless Systems is a pioneer in this groundbreaking technology and develops open-source software that allows companies to use confidential computing at scale. To fundamentally address concerns around data leaks, Edgeless Systems has launched Continuum, the first confidential LLM platform, available at https://ai.confidential.cloud/. Continuum comes with an intuitive chat interface and serves the state-of-the-art open-source models, currently Mistral 7B. In the upcoming weeks, the platform will expand to include additional LLMs such as Llama 3 70B, ensuring that users have access to the latest advancements in AI while maintaining full confidentiality.

Confidential computing is widely available with current server CPUs. With the NVIDIA Hopper architecture-based H100 Tensor Core GPU, NVIDIA launched the first AI accelerator that provides confidential-computing features.

Continuum ensures the end-to-end confidentiality of all user data. For this, Continuum leverages the confidential-computing features of NVIDIA H100 GPUs, confidential computing-enabled virtual machines, and advanced sandboxing technology.

This technology enables users to share sensitive information with an LLM without having to worry about data leaks and unauthorized re-training. Organizations interested in using Continuum in production can already join Edgeless Systems’ waitlist. Further, Edgeless Systems will soon make Continuum available as open source.

About Edgeless Systems

Edgeless Systems develops the open-source software stack for confidential computing.