Project Description

Trusted Private AI Services

Customer Need

The client wanted to use AI Services (chat, image, and video) for software development and production use cases on data which have secrecy and compliance requirements.

Issues

The client had three significant areas of concern:

  • Trust – Required high level of trust in AI applications and data (AV scans, CVE reports, SBOM, cybersecurity artifacts, and integrity).
  • Privacy – Concerned protected / proprietary info could be compromised using public AI systems.
  • Isolated Operation – Needed to connect AI services with isolated enclave systems.

Solution

The client worked with Red Rake to deploy Trusted Private AI Services by Real Obvious.
  • Trust – Real Obvious AI is a Trusted AI service. Apps and data are tested, validated, and certified before delivery to customer.
  • Privacy – Real Obvious AI is a Private AI service. Self-hosted support enables deployment on-premise or within a customer’s cloud.
  • Isolated Operation – Real Obvious AI is designed to deploy, run, and update in isolated enclave environments.

Details

The client recognized the potential benefits of using AI Services to analyze, update, and adjust (feedback loop) their real time application. Because of the proprietary nature / compliance requirements of their application and data, they were required to operate the AI Services offline (air-gapped / disconnected from the Internet).

While it was possible for them to deploy AI Services on their own, they recognized that long term operations and maintenance (software updates) would place a significant burden on their internal IT team. They were also very concerned about the quality (level of trust) of the AI applications and data being loaded onto their isolated system initially and on each update.

The solution involved the design and deployment of an AI Services architecture which would operate in an isolated environment. Several Open Source (OS) AI applications and data (large language models) were  selected for deployment. Using DevSecOps best practices, a Software “Trust” Factory  was built to transition OS AI applications and data from a untrusted –> trusted state and from a public –> private level of operation. All software was developed using Red Rake’s Real Obvious technologies.

Technologies

Results

The initial release of the project was completed on time and within budget. Work continues on the project to accommodate enhancements and upgrades to the Real Obvious Trusted Private AI services.

Related Links