Scaling the Cisco AI Assistant for Support with Splunk

Cisco needed to scale its virtual support engineer that assists its technical support teams around the world. By leveraging its own Splunk technology, Cisco was able to scale the AI assistant to support more than 1M cases and free up engineers to concentrate on more complex cases, creating a 93+% customer satisfaction rating, and ensuring the critical support continues running in the face of any disruption. 

If you’ve ever opened a support case with Cisco, it’s likely that the Technical Assistance Center (TAC) came to your rescue. This around-the-clock, award-winning technical support team services online and over-the-phone support to all of Cisco’s customers, partners, and distributors. In fact, it handles 1.5 million cases around the world every year.

Fast, accurate, and consistent support is critical to ensuring the customer satisfaction that helps us maintain our high standards and grow our business. However, major events like critical vulnerabilities or outages can cause spikes in the volume of cases that slow response times and quickly swamp our TAC teams, impacting customer satisfaction as a result.  we’ll dive into the AI-powered support assistant that helps to ease this issue, as well as how we used our own Splunk technology to scale its caseload and increase our digital resilience. 

Building an AI Assistant for Support

team of elite TAC engineers with a passion for innovation set out to build a solution that could accelerate issue resolution times by augmenting an engineers’ ability to detect and solve customer problems. the was created — it’s more than an AI bot and less than a human, designed to work alongside the human engineer. 

Fig. 1: All cases are analyzed and directed to the AI Assistant for Support or the human engineer based on which is most appropriate for resolution.

By directly plugging into the case routing system to analyze every case that comes in, the AI Assistant for Support evaluates which ones it can easily help solve, including license transactions and procedural problems, and responds directly to customers in their preferred language. 

With such great success, we set our eyes on even more support for our engineers and customers. While the AI Assistant for Support was originally conceived to help with the high-volume events that create a significant influx of cases, it quickly expanded to include more day-to-day customer issues, helping to reduce response times and mean time to resolution while consistently maintaining a 93+% customer satisfaction score. 

However, as the use of the AI Assistant grew, so did the complexity and volume of cases it handled. A solution that once handled 10-12 cases a day quickly ballooned into hundreds, outgrowing the methodology originally in place for monitoring workflows and sifting through log data.  

Initially, we created a methodology known as “breadcrumbs” that we tracked through a WebEx space. These “breadcrumbs,” or actions taken by the AI Assistant for Support during a case from end to end, were dropped into the space so we could manually go back through the workflows to troubleshoot. When our assistant was only taking a small number cases a day, this was all we needed.  

The problem was it couldn’t scale. As the assistant began taking on hundreds of cases a day, we outgrew the scale at which our “breadcrumbs” method was effective, and it was no longer feasible for us to manage as individuals.  

Identifying where, when, and why something went wrong had become a time-consuming challenge for the teams operating the assistant. We quickly realized we needed to: 

• Implement a new methodology that could scale with our operations 
• Find a solution that would provide traceability and ensure compliance

Scaling the AI Assistant for Support with Splunk 

We decided to build out a logging methodology using Splunk, where we could drop log messages into the platform and build a dashboard with case number as an index. Instead of manually sifting through our “breadcrumbs,” we could instantaneously locate the cases and workflows we needed to trace the actions taken by the assistant. The troubleshooting that would have taken us hours with our original methodology could be accomplished in seconds with Splunk.  

The Splunk platform offers a robust and scalable solution for monitoring and logging that enables the capabilities required for more efficient data management and troubleshooting. Its ability to ingest large volumes of data at high rates was crucial for our operations. As an industry leader in case search indexing and data ingestion, Splunk could easily manage the increased data flow and operational demands that our previous methodology could not.   

Tangible benefits of Splunk

Splunk unlocked a level of resiliency for our AI Assistant for Support that positively impacted our engineers, customers, and business.

Fig. 2: The Splunk dashboard offers clear visibility into functions to ensure optimized performance and stability. 

With Splunk, we now have: 

• Scalability and efficiency: Splunk monitors the assistant’s activities to ensure it’s working correctly and provides the ability for TAC engineers to monitor and troubleshoot workflows, allowing the assistant to efficiently scale. The AI Assistant for Support has successfully worked on over one million cases to date. 
• Enhanced visibility: With dashboards that allow for quick access to case histories and workflow logs of our assistant, the TAC engineers overseeing the processes save time on case reviews to deliver faster than ever customer support. 
• Optimized processes with real-time metrics: The visibility into resource allocation allows us to optimize our business processes and workflows, as well as demonstrate the value of our solution with real-time metrics. 
• Proactive monitoring: Splunk ensures all APIs are fully functioning and monitors logs to alert us of potential issues that could impact our AI Assistant’s ability to operate, allowing for quick remediation before customer experience is impacted. 
• Higher employee and customer satisfaction: Engineers are equipped to handle higher caseloads and efficiently reprioritize efforts, reducing burnout while optimizing customer experience. 
• Reduced complexity: The dashboards have a simple interface, making it much easier to train and onboard new employees. The ease of use also serves to improve the capabilities of the humans operating our AI Assistant by enhancing their accuracy and efficiency. 

By providing a scalable and traceable solution that helps us stay compliant, Splunk has enabled us to maintain our commitment to exceptional customer service through our AI Assistant for Support.

Additional Resources:

PS:  Attending Cisco Live in San Diego this June? 

You’ll have a special opportunity to talk live with Cisco IT experts to dive into these success stories and other deployments! Look for Cisco on Cisco in each of the showcases and be sure to search Cisco on Cisco in the session catalog to add our sessions to your schedule!

Share: