How do you use AI for development in high security environments?

Question

I'm curious how this is done effectively assuming no source should be sent to a model hosted remotely. Are there foundational model toggles for using inference and embedding but not using data for training?

daemonologist · Accepted Answer

All the big providers offer no-training/retention guarantees (either by default, or as a toggle, or upon request). For many high security environments though I'd expect everything to be hosted on-prem or at minimum on company-controlled instances, which does limit your model options somewhat.My employer has such contracts for some use cases, but actually forbids use of code completion/generation due to IP concerns.

scarface_74 · Answer

I&rsquo;ve worked with companies that would never trust publicly hosted models. But don&rsquo;t have any issues with hosted models on AWS or Azure. But I work in cloud consulting so they already have to trust the cloud provider.Yes this includes GovCloud implementations that have citizenship requirements and you can&rsquo;t connect outside of the US.I have not admittedly worked on any projects in the &ldquo;secret&rdquo; regions.https://aws.amazon.com/federal/secret-cloud/

sky2224 · Answer

As someone else stated, there are enterprise services that offer solutions that make it so your company data isn't consumed, however, I think pretty soon we're going to see a lot of companies maintaining models locally in-house.I think this is especially true given that Intel is shifting its focus toward an affordable in-house solution for training AI models locally with its upcoming GPUs.

2rsf · Answer

Repeating what others have written based on my experience at the bank i work for- business offering will not use or save you data, and for more sensitive material we simply host it on prem