Formerly known as Azure AI Services or Azure Cognitive Services is a unified collection of prebuilt AI capabilities within the Microsoft Foundry platform
Hey Oguz Kokes,
Good day. Thank you for confirming that issue is not replicable anymore.
Emphasizing on “Production scenario” context as you requested.
These error (error 715-123420) are auto triggered from Backend team when system detects and intends to surface in below scenario
- Anomalous resource creation with unsupported templates/API versions or
- There is anomalous usage at customer side (Un-authorized from outside players)
- Recursive inputs violating content filtering or chain of thought violation
Best practice would be to monitor below perspectives
On Anomalous usage
- Enable Entra credential instead of key based authentication. Reference - How to migrate to OpenAI Python v1.x (classic) - Microsoft Foundry (classic) portal | Microsoft Learn
- Create custom monitor alerts to monitor over-usage Reference - Monitor usage and spending with cost alerts in Cost Management - Microsoft Cost Management | Microsoft Learn
- Secure with VNET if you are using public resources.
- Adopt least point privileging access for user through proper RBAC
On prompts
Sanitize the prompts and file with content safety resource and sanitize based on probably of detection before reaching model endpoint
Enable tracing in SDK to flag the prompts and save it in a separate storge and view it workbooks
Use Advanced abuse monitoring feature (as a Managed customer) if you are dealing sensitive info like medical records for your use case
Reference
Tracing in Azure OpenAI
Advanced abuse monitoring form
Resiliency in production case
Create multi region deployment and create runbook automation/use APIM to load balance properly based on signal received in
Relevant guide
Architecture
You can use AI gateway to setup alerts and routing points for Azure OpenAI deployment (Slightly affordable for developers compared to APIM gateway)
AI gateway in Azure API Management | Microsoft Learn
You need to enable this feature as per documentation prior usage though.
AI gateway in Azure API Management | Microsoft Learn
Please let us know we can consider this resolved with positive feedback if all above queries addressed your additional queries on fool proofing with respect to production scenario.
Thank you for inputs on forum.