Use Azure Active Directory (AAD) to authenticate all users and restrict access to the private endpoint.

Implement Azure Private Link to create a private endpoint for the data pipeline, and use Azure Firewall to control public access.

Configure Azure Virtual Network Service Endpoints to provide private access to Azure services, and use Azure API Management for public access with rate limiting.

Set up Azure ExpressRoute to create a private connection to the data pipeline, and use Azure Policy to enforce public endpoint security configurations.