Be AI Confident with New Observability Solution

This paper presents a unified observability architecture for on-prem AI environments based on Cisco Secure AI Factory with NVIDIA, including Cisco AI PODs as the underlying infrastructure, together with Splunk Observability Cloud and Splunk Enterprise. This unified architecture is based on SoftServe’s experience as MSP running SOC and NOC using Splunk for our clients and internal purposes, and Cisco validated architectures.

The solution is built on a split-plane observability model tailored by SoftServe for regulated on-prem AI deployments:

• Operational telemetry, such as metrics, distributed traces, and derived AI quality signals, is analysed in Splunk Observability Cloud for real-time visibility into system performance and AI workflows.
• Sensitive raw AI interaction content, audit records, and other governed logs can remain in Splunk Enterprise under enterprise governance controls.
• Correlation across both planes is achieved using shared identifiers, enabling end-to-end troubleshooting without moving sensitive data.