How to Secure Azure AI Foundry Hub Project Connectivity
&edge_flat=false)
- Scenario: I found that a lot of customers have started deploying AI models in Azure AI foundry and they’re deploying them with endpoints set to public. When it comes to Private Endpoint deployment in AI foundry, the configurations can be tricky as there are multiple Private endpoints that will get created. If you’re using a hub-based approach, then this blog is for you. We’ll explain how the Private Endpoints (PE) of the AI Foundry service are created in the managed VNET. How you’ll need to create that in customer VNET, along with that PE for additional services need to be created in customer VNET and so on.
AI Foundry project-based deployment is different from hub-based deployment. Please note, Microsoft recommends choosing an AI Foundry project instead of a hub-based project, however there are some features which are only available in Hub based project. If you want to learn the difference between the two options, you can check the article below.
Let’s proceed assuming you’ve done your assessment and selected AI Foundry Hub based project.
Services Placement
Download the block diagram
Before we discuss all the private endpoints, let’s first look at where the services will be placed.
- This is the on-premises workload which will access the AI Model present in AI Foundry or connect to Applications hosted on Azure which will then talk to models hosted in AI Foundry.
- Customer HUB VNET is where all the connectivity related components will be deployed like Express Route/ VPN gateway, private endpoint, firewall etc.
- This is the spoke subscription where your PaaS services will be hosted, these PaaS services are often dependent services like AI search service, blob storage which are required for your application along with AI Foundry. Private endpoints of these services will be deployed in Hub VNET so that on-premises can talk to these services. In an enterprise environment, these private endpoints will be hosted in the VNETs of the spoke subscription in a separate subnet, not in the hub. We’ve simplified and shown the PEs in the Hub VNET.
- This is where all the additional services which get deployed along with AI Hub project, will be hosted.
- This is where your AI Foundry managed VNET will be hosted. Serverless, prompt flow and managed endpoints will be hosted.
Additional services part of AI Foundry
As mentioned in the 4th point above, when you deploy a hub-based project, additional PaaS services will also be deployed.
AI services will be created 
Container registry, blob storage and key vault is created. 
This screenshot shows all the services which get deployed as part of AI hub based project. 
Connectivity Method for AI Foundry
We’ll be deploying the AI Hub service in private mode which is “Allow only approved outbound” and public connectivity will be disabled. This will ensure that you can only access the AI Foundry portal through a connected network and not publicly. Along with that any outbound connectivity needs to be explicitly allowed in the outbound rules. In the backend an Azure firewall will be deployed which will enable this functionality of allow and deny.
Making sure your public connectivity is disabled for the workspace. 
private endpoint is deployed in customer vnet and also a PE is created in managed VNET. The one with _sys is for managed VNET. 
Validating the network setting of “Allow only approved outbound” is selected for workspace. 
Private Endpoint Scenarios
PE for AI foundry and additional PaaS services in customer VNET
Download the architecture diagram
This architecture shows two scenarios of private endpoints, first scenario is PE of search service, storage account or PaaS services which are deployed in customer’s environment that is required by the application. PE will get deployed in the HUB VNET in a subnet. This is typical scenario which you must be using PaaS service and it’s private endpoint.
The second scenario is of PE that is created for the AI Hub itself and for additional PaaS services deployed as part of the AI Foundry Hub project. These are AI Search, keyvault, container registry. AI Hub private endpoint is required without which you cannot connect to AI Foundry portal.
PE for additional services and customer PaaS services in managed VNET
Download the architecture diagram
This section is of PE which gets deployed in the managed VNET and it gets deployed as part of AI Hub project. A managed hub is where all your PE will get created along with Agent, serverless or compute instances.
Whenever your compute instance in the AI Foundry wants to connect to additional PaaS services securely it’ll connect to private endpoint present in the managed VNET and hence for container registry, AI services, blob and key vault individual PE are deployed in the managed VNET itself. This is created automatically as soon as managed VNET is provisioned.
This image shows the _Sys private endpoint which are created automatially. 
you’ll see this as inactive, the reason is while deploying the AI Hub, managed VNET is not deployed, it’ll only get triggered once compute instance is deployed or if you have instructed during the deployment to provision the managed VNET. This is to save the cost of Private endpoints and also the firewall resource. These PE are automatically created and you don’t have to manually create it.
Second scenario is of PE for customer deployed PaaS services, these PE are also created in managed VNET, for compute or serverless instances which need to communicate with PaaS services privately, a PE is required. Creation is not automatic and need to be created manually. Depending upon the services which you’ve deployed in your spoke subscription and based on the project you’ll need to create private endpoint of those in managed VNET. Below are the steps to create it.
Adding a new connection from AI foundry portal. 
Searching for desired service for which you need to make connection, here we’re searching for ai search 
Enter the name of the service which you want to add. 
I have observed, for couple of service private endpoint is created automatically if managed vnet is provisioned already. However in my scenario it is not provisioned hence creating a PE manually. 
Validate private endpoint is created successfully. 
AI foundry hub showing outbound rules, PE is created. 
Outbound connectivity from AI Hub to on-premises resources
As AI Hub compute resources are in managed VNET, you cannot directly peer that with HUB vnet to reach on-premises. There is no such option. One way is to create application gateway in Hub VNET and put on-premises resource IP address in the backend pool of the application gateway, and create private endpoint of this application gateway in the managed VNET. This way compute instance in AI Hub project can connect to on-premises resources.
Download the architecture diagram
please check below article to know more https://learn.microsoft.com/en-us/azure/ai-foundry/how-to/access-on-premises-resources
Securing Additional PaaS services
The services which are deployed as part of the AI Hub accepts public connections, if you have selected the option of “Approved Only outbound” that is only for AI Hub resource and not for all the additional services which are deployed as part of AI Foundry. Hence you need to ensure each individual service is not publicly exposed. Along with that you need to make sure to check the option to allow trusted Microsoft services.
Making sure public connectivity to container registry is disabled. 
Making sure public connectivity to AKV is disabled. 
Making sure public connectiivty to blob storage is disabled. 
Inactive PE
During initial deployment if you’ve not selected to provision the managed VNET, hence you’ll see that the PEs are inactive because the VNET and firewall aren’t deployed. Best approach is to select the checbox during the deployment itself.
This is where you can select “provision managed vnet” during AI foundry hub project. 
You can deploy it manually after the AI Hub is provisioned through CLI.
I hope you deploy your AI Foundry Hub Project securely. Share the blogpost if you like it.
Happy Learning!
Subscribe to my newsletter for the latest posts and summaries. Your privacy is paramount, and your email stays with us securely. click the link to submit your email. https://forms.office.com/r/6ysKm4nkp4