Automated Quality Management with Speech Recognition – Why Contact Centers Need It



Rising call volumes, increasingly complex calls and difficulties finding and retaining qualified staff; these are three of the most complex challenges facing contact centers today. Their business objectives, however, are perhaps more ambitious than they’ve ever been.

According to a 2022 McKinsey and Company study on the state of customer care in 2022, those objectives include improving the customer experience, reducing contact center volume, and introducing AI Assistance, all while increasing service call revenue generation. 

Bottom line - contact center leadership is being asked to do more with less - raising performance while lowering costs and generating additional revenue. 

Focus on Automation

Contact centers continuously monitor and measure customer satisfaction. It’s no secret why; delivering the best customer experience possible is Job #1 for a contact center, and that’s why agent performance is also front-and-center in the minds of managers, supervisors and team leads. 

In many contact centers, agents’ interactions with customers are manually evaluated by supervisors and team leaders. The crucial word here is "manually". Supervisors must listen to calls in order to understand whether or not the agent followed the call script, a review activity that takes virtually the same amount of time as the call itself. 

The result? Contact centers using a manual process typically can’t evaluate more than 1% of their calls, dramatically limiting their understanding of the quality of service they provide to their customers and compromising their ability to identify and address any issues. Leaders are essentially operating blind, making decisions – or not making them – absent the proper insights. There’s an inability to identify training needs, employee turnover increases and the customer experience and the organization’s brand suffer.

Enter automation. The WFO software market now boasts a number of automated tools developed to simplify and streamline daily processes and analyze interactions without the need for manual intervention. Enterprise companies were early adopters, but now even mid and small-sized contact centers are leveraging automation.

Automatic Speech Recognition

Automatic Speech Recognition (ASR) is a technology that converts spoken language (an audio signal) into written text (a transcript). The technology is also capable of providing additional outputs useful for analyzing an agent/customer interaction, such as emotion and sentiment detection, along with the detection of other acoustic parameters.

ASR outputs:

  • Transcription – dialog between call participants
  • Emotion and sentiment – identified for each utterance on the transcript and aggregated for each call participant
  • Acoustic parameters – identified for the overall conversation, as well as for each participant. These values are also available as absolute values (for instance, cross-talk time) and relative values (for instance, the ratio of cross-talk time against overall handling time)

    • Crosstalk time - participants talking at the same time
    • Silence time - nobody is talking
    • Number of interruptions - when one participant was interrupted by another
    • Speed of speech - number of words per minute
    • Speech phrases - predefined speech tags and phrases identified and highlighted on top of the transcription 
    • Gender

A visualization of all available ASR outputs is provided in the images here:

Automatic Speech Recognition Use Cases

With the help of ASR a user can easily leverage the available outputs in the following ways:

  • Understanding the content of the call at a glance (when looking at key attributes) without the need to playback the call, reviewers can answer the following questions:

    • What was the overall mood of the customer? What was their emotional state at the beginning of the call, and how did it change as the call progressed?
    • Was the agent moody? Were they rude? Polite? 
    • How many seconds was the overall cross-talk time?
    • How many times was the customer interrupted by the agent and vice versa?
    • Was total silence time suspiciously long (was the agent pretending to work)?
    • What was the overall silence time (Was the agent able to help the customer)? 
    • Did the agent speak clearly? Do they talk too quickly? 

  • Finding calls based on these attributes:

    • Find calls based on the available attributes quickly.
    • Real-time, full-text search allows for content to be found in any conversation transcription.

  • Visualization of acoustic parameters and transcription during playback of the call:

    • Speech phrases and the detected emotion are highlighted within the interaction player´s waveform
    •  Captions are shown in the player during playback

  • Leverage emotion and acoustic parameters when automatically evaluating conversation´s agents by the system (refer to the section Automated Quality Management for more details!)


Automated Quality Management

As mentioned previously, the manual review process is very time-consuming, and contact center managers are only able to evaluate an extremely small percentage of customer interactions. The ultimate goal for every contact center is to evaluate - or at least to understand - the experience of up to 100% of their conversations.

Having an automated solution that can analyze all interactions on a regular basis can identify emerging issues, find new opportunities and highlight the outstanding and poor agent performance. Additionally, alerts can provide quick information about both outstanding and problematic conversations.

Automated Quality Management (AQM), sometimes also called Analytics-enabled QM automatically evaluates all conversations and the agents involved based on various parameter categories such as: 

  • Metadata captured from CCaaS platform (handling time, attached flags, information about hangup party, etc.)
  • Metadata analyzed by Automatic Speech Recognition (emotion, acoustic parameters, and transcription; see above for more details)
  • Post-call assigned metadata (tags)
  • Metadata assigned by external systems, typically a Survey tool (NPS score) or CRM system (case-related information)

The logical structure of AQM is shown in the picture below:



Automatic Quality Management Use Cases

With the help of AQM, Eleveo will automatically assign a score to all conversations based on a predefined set of rules set by the user. Typically, contact center managers want to reward agents who correctly follow the call scripts and other internal policies. 

The following table shows examples of how AQM works. However, rules for evaluations are fully configurable by the customer.


An integrated alert system is an important part of the tool, and the system notifies a predefined user or the agent´s supervisor if an automatically assigned score is less/greater than the defined threshold.

Looking Forward

Natural Language Processing
Natural language processing (NLP) refers to the branch of computer science—more specifically, the branch of artificial intelligence or AI—concerned with giving computers the ability to understand text and spoken words in much the same way human beings can.

Speaking of AQM and its new capabilities, the goal is to classify every single transcription according to the customer´s call script. Therefore, every question included as part of the call script (scorecard/questionnaire if preferred) will be automatically answered by the tool. The advantage of using this approach is that it is likely that the AI/Machine Learning tool will find the correct answer if trained using an appropriate dataset. The requirements to provide a data set and subsequently training the tool (goes hand in hand with costs) are potential downsides and have to be considered during the implementation phase for every single customer.

Text classified by Natural Language Processing (actually, the output of the classification) is intended to be used as an input parameter in AQM and will therefore have a direct impact on the conversation evaluation.

Contact center managers and supervisors must be able to monitor, measure, follow trends, and work with aggregated data. The AQM and ASR results must be available in the form of reports as well as KPIs on the Eleveo Dashboard.

Even Greater Automation
Once the system understands that something went wrong in a conversation it can propose an appropriate follow-up action. Such an action might include the scheduling of a manual review or the scheduling of agent training. On the other hand, the system can also reward outstanding agent performance, perhaps by letting them choose their preferred shifts. 


In summary, the reasons for including more advanced technologies and tools, such as Automated Quality Management (AQM) and Automatic Speech Recognition (ASR), are compelling. 

Eleveo tackles the various challenges in this area with the introduction of a new smart tool that simplifies the daily routine of contact centers. With the help of ASR and AQM tools, contact centers will be able to analyze what happened in each and every conversation, identify potential issues almost immediately, and constantly improve workforce performance – all of which are keys to improving customer satisfaction, and that is Job #1 for contact centers. 

Back to Blog

Related Articles

What Does It Take to be a Cloud Contact Center? A Plan.

As cloud contact centers become more prevalent, methodologies for migrating from on-prem to cloud...

Cloud Contact Center Providers

With the right cloud contact center provider, businesses can centralize their customer...

Elevēo Receives 2021 Contact Center Technology Award | Elevēo


About Elevēo

Elevēo was formed to provide easy to use, results oriented software & expertise for customers who don’t want features that they do not need clogging up their workflow.

Our products do only what you need to get results, are built using modern frameworks and cloud native technologies and are priced based on how much you use them. Elevēo products are birthed from ZOOM International with its rich WFO history and award-winning products, services and reputation for service.

We believe that Workforce Management can and should be an intuitive and easy process that contributes to employee engagement while supporting an exemplary customer experience.