A hosted AI solution typically requires less setup and maintenance for the implementing teams, as the provider takes on the responsibility for many components
The provider will manage the hosting service, and the customer will secure the access and data integration, design the identity management, and monitor the solution
Operating in the customer’s Azure tenant offers more control over the network infrastructure, identity design and logging, but does not give direct management of the models or services themselves
When reviewing the Enterprise AI deployment options ask: What does “our environment” mean for every design element?
Is it dictated by the data center location, traffic flows, model access or incident response?
With a managed service the provider will typically take care of the physical infrastructure and the elements they are responsible for.
The customer will manage the permissions, protect the secrets, design the data-handling rules, and ensure that the logs are properly evaluated.
Thus, the infrastructure costs and maintenance burden are reduced but not eliminated: when comparing the Enterprise AI deployment options it is important to also evaluate the data-handling terms, the location of data, the identity integration possibilities, logging, and legal terms.
Operating in the customer’s Azure tenant
The solution can be designed with the appropriate Microsoft Entra ID integration, networking, monitoring, and policy engine, depending on the particular service architecture.
This can be beneficial for enterprises that already operate extensive Azure environments, but at the cost of increased responsibility: the tenant-based design requires the enterprise to maintain the network infrastructure, define the roles, manage the secrets, monitor the solution, develop and maintain the pipelines, perform incident response, evaluate the models, update prompts, ensure the quality of retrieval data, and manage the application updates as appropriate.
The Enterprise AI deployment options should be compared based on the ability to manage these elements.
Security, data, and compliance
Define the data the enterprise AI service will process.
Does it include public data, enterprise documents, customer personal data, or any other sensitive information?
Mapping the data flows from collection, through processing, prompts, model data, logging, and outputs will help to understand what data is used for what purpose.
Ask about the possibility to use the data for training, its location, retention, and deletion policies. Least-privilege access, network restrictions, audit trails, and manual reviews of critical processing steps are examples of security controls that should be in place for enterprise AI deployment.
Consider Enterprise AI deployment options that feature the security controls your organization needs, and that help to demonstrate security posture and regulatory compliance.
Compare the costs associated with the different Enterprise AI deployment options, including the expenses related to the consumption of AI services, networking, storage, monitoring, security technologies, support, and maintenance.
A managed cloud AI service may require fewer resources but have higher costs per request, while a more controlled setup may incur additional expenses due to the required infrastructure and maintenance.
When considering Enterprise AI deployment options it is important to estimate the total cost of ownership based on the organization’s needs and capabilities.
Conduct a pilot project to analyze the quality of results, response time, utilization, costs, and failure responses.
Define the ownership of access reviews, model updates, incident response, and monitoring responsibilities.
Select a managed cloud AI service when rapid deployment, reduced infrastructure maintenance, and increased level of provider support are needed.
Choose the tenant-based design when the enterprise needs to manage the network infrastructure closely and have control over the data processing boundaries.
A hybrid approach can be considered when different aspects of the solution require different levels of control: lower-risk areas can be managed by a cloud provider, while higher-risk elements remain in the enterprise’s control.
However, hybrid Enterprise AI deployment options typically require additional resources and maintenance compared to a single-cloud approach.
The choice between Enterprise AI deployment options should be based on the responsibility matrix that includes the hosting, identity, data access, network security, application protection, monitoring, model evaluation, incident response, and compliance evidence.
Consider the deployment options that best align with your organization’s ability and desire to manage these elements.
The key takeaway is to choose an AI hosting model that reflects the sensitivity and security needs of the data being processed, and that defines the roles and responsibilities between the provider and the enterprise.
Microsoft’s shared responsibility model that outlines the security, data, and compliance responsibilities depends on the particular service.
Azure AI services have documentation for each service that details the data processing and privacy-related aspects relevant to the particular offering.
Is AI in our Azure tenant more secure by default?
No, there is no automatic security advantage based on the location of processing.
The security depends on the configuration, permissions, network design, application logic, monitoring, and data handled by the AI service.
Doesn’t a managed cloud AI service take care of everything?
No, the organization retains responsibility for the application security, data handling, and proper use of the service.
What should we compare when evaluating Enterprise AI deployment options?
The choice between Enterprise AI deployment options depends on the data sensitivity, compliance requirements, identity management, network security, and operations-related factors, as well as the responsibilities that the organization is prepared to accept.
Can we utilize both options?
Yes, the organization can host different workloads in different environments, provided there is an understanding of the data segmentation, security controls, owners, and monitoring requirements.
