Listen to This Before You Build or Buy an AI Auto QA Platform

00:00I advice from a call center geek. a weekly, our focus on all the things call center. We'll cover it all from call center operations, hiring culture, technology, and education. We're here to give you actionable items to improve. I like the quality of yours and your customers' experience. This is an evolving industry with creative minds and ambitious and ambitious people like. Like this guy, not only is his passion call center operations, but he's He's the CEO of Expivia Inter a Interaction Marketing Group and the call center geek himself, Tom L rd. What's up everybody? Welcome to another episode of advice from a call center geek. The call center podcast we try to give you some actionable items to take back in your contacts and improve the overall quality, quality, improve the agent experience, improve the customer experience. What's going on. I hope. I hope. I-on. I hope everybody's going on. I hope everybody

[Gap 01:10–01:40: no reliable transcription]

01:40to take a couplebraska minutes here, and I think that this should be a really cool value ad for everyone and kind of talk and talk about where Quality Assurance is today, you know, and where it's moving for kind of in the future, where I kind I'm kind of a see it. And again, if you're looking for any tool that automate your quality assurance, and where it's So, I can't... I can't, can you close the door? I can't, that it's almost been 24 going over a little bit, 24 months, that we started our AutoQ. QA process and started the program and thought, you know, we could really do something cool. So the amount that we've learned in the AI, you know, has been just valuable. I can't really put even a finger on how- I can't really put even a finger on how quickly this thing is moving

[Gap 02:51–03:13: no reliable transcription]

03:13take a week, we're coming up with some other technique, technique tool. So when we really just

[Gap 03:31–04:01: no reliable transcription]

04:01halls is kind of the revolutionary endpoint, but it's not, it's the beginning step. You have to be able to a score with AI to do all the really cool other things. But most people I see are stopping there, right? So they're getting 80 to 90% accurate quality assurance scores at scale. They're thinking that's great. And. I'm telling you it's just there's so much more that you can do and there's so much more out there. First thing is you have to have your ITT and you have to be able to push back and tell them or the vendor that you're working with that 80 to 90% accurate isn't good enough Because it's not. And there's lots of different techniques that can It's the first step, right, but let's say, and there's a lot of really cool vendors and What I see then Rene is...B-B-B-B-B-B-B-B-B-B to data that we want to collect to try to get to different levels.

05:47For instance, we said, hey, let's just the score calls and tell it why it scored it, but let's coaching tips. Those coaching tipsB are now, building building our full coaching platform. That's the engine from the data that we're collecting, because we're doing that naturally. We said, hey, we need but kind of give you a real AI customer journey. to I'm like, what? What did you learn? Right? Right. Now, maybe there's a dashboard and you can see some lower reps or some higher reps.

[Gap 06:59–07:22: no reliable transcription]

07:22And again, when people, when customers, like Tom, can you help us? Our quality form isn't up to where we think it, it should be. Right. So if you're if you have one of those forms that is not great, then I think the problem is you're just scaling garbage. Right. You're scaling GARBA. Question coming in here, right? What are some things that AI, gets wrong? And how does he, how does he even find out what's How does anyone even find out it is wrong? Well, there's multiple things. And again, I can only tell you what we're doing at Auto Q we're still not at a full voice inflection deal yet, right? So there, there have to prompt for sarcasm

[Gap 08:32–09:02: no reliable transcription]

09:02that I've struggled with, well, there's two other things. One But it's I would say, the screen. Put in the proper call time for somebody who picked up. Right. Putting specific notes if you're not using any auto. I can't see your screen to see if the agent did what they're supposed to in the CRM. Other than that, I'm sorry, the sky's the limit with empathy call control. Hold times, right we can, we can tell the difference between somebody on hold for getting a supervisor to being on hold to be transferred and then the total amount of hold I'm going to be going to be going to be going to be going to

[Gap 11:16–11:57: no reliable transcription]

11:57to put in coaching versus auto summarization versus just scoring. It really doesn't. Now Now, when you get to the accuracy things and So we have a lot of things that you need to do, but again, looking and trying to get all, all of those things that your future QA will want, right? How are you going to get better coaching for your agents? Well, let's put in some things that they're doing well and doing wrong, compared to the form. And then we can build that out later. But our engine is built, right? We talked about CSAT and N.P.S. We want to do a voice of customer down the, let's start Again, we have an unbelievable tool, it's totally free. If you go to- auto QA.com and click on our quality,. Our QA form analyzer, you can upload your quality form, and in about In about 20 seconds, we where most people stop.

13:48And then we go a town with our humans. Really trying to create something that does box but allows you to create the data that you need to to make improvements, right? are you a sales call center, are you a customer service call center. And how can we build, and, I think a lot of these forms have been built a long time It isn't just there to talk about agent behaviors, go to wrong, but it is the richest voice of It's better than, any survey that you're going to get that's again where the CSAT and the NPS we saw could be could be done, but. [QA coverage remark omitted]

15:51Because those post call surveys, they're noisy, they have extremely low response, seven to people who sent them back, and it's really, the only people are the really, really, happy people. So if we could say, and we could use the, I, and I., and we, and we could prompt for certain specific things, the amount of data that we've been able to pull out that you can too, if you really think it through, is really, is, is, is really, it's really crazy. Q, QA is, is kind of like a, it's a CX lens, right? We're trying to just find the truth about what is happening. So, it's the truth let's shift gears here for a second and let's talk about coaching, right? Because most QA systems don't really support coaches, they absolutely, especially they're overwhelming them, right? We have all these, all of these messy dashboards, about coaching, because most QA systems don't really support coaches. They absolutely, especially today, they're overwhelming them, right? We have all t he messy dashboards. We have analytics that are going out through the call where you're looking at, you know, where things things are going up, where things are going down, where things were set, it's very difficult pull up the call and then-., and,.

18:01F I I

[Gap 18:31–19:31: no reliable transcription]

19:31AT&T obviously has voice, has voice, has voice. and just came out with their voice voice

[Gap 19:46–20:01: no reliable transcription]

20:01are the smartest people on the program, they do it every single day, so being able to have the rubric, have that uploaded into the large language model, have the quality form uploaded into the language model, having the quality form not just have as much data as humanly possible with what they possible with what they did and what they could have improved upon. Then and only only then, can I think, we start to have the conversation of, can an, can an agent have a, a supervisory training session, that M In a ton of simulations here. questions, um, some guardrails that have to happen with it. I I think the better use in AI voice avatar is what we're doing with, with, I know there's There's a, again, I'm on a platform called React. I really like React instead of we were calling this Redo Sounds kind of lame to, to be honest.

21:32But not that I guess React is better. But we're turning Q and A into kind of an AI training ground, right? We're turning auto into that, not just having a scoring tool, right? But agents can learn better when they And, we can to pull out And, we can pull out what the agent said. So, again, let's back this up. Let's say an agent has a really poor call. They scored a 52., they All of those different thes. you can go and redo the call. So they will click a button. And what we've done is we've, taken again, that transcript pulled out what the agent said, but the exact things that same customer name. Same tone, we're using male and female voices, same issue, same problems, right? We can kind of be even play around a little bit with their tone of voice, but we try to keep everything exactly the same.

23:11We send that out to the large language model through a ton of proprietary prompting that we've done. We then send that to 11 Labs right now. We might be using OpenAI's kind of whisper in their voice, but anyway, it comes back and we can have a conversation that agent can retake that call with the AI avatar of the person that they just didn't do a really good job with. So this time they can do the crossover. They can have call control. They can show empathy. And then when that call is done, we go re-score it. Then we show the difference on the platform of what they scored before, and then when they redid the call. So again, auto QA is moving very quickly from not just a scoring mechanism, but a coaching tool and a full agent quality tool, right? To have the full circle of not just scoring, but improving the entire agent behavior.

24:12And again, that's what we're trying to do with auto QA. I know a lot of other companies are kind of working on that as well. We're moving very, very fast with this stuff. Cause I know that, you know, a lot of people are doing a lot of really cool things in this space. We should have our redo out, you know, by the end of next month, it's being worked on in R&D here in our internal contact center right here at Expivia. The agents kind of are, they're kind of been into it. some call types for finding work better than others. But I think that that's a really cool thing. So again, I know this is a lot and it was kind of a ramble episode. There's so many things that I wanna talk about that maybe kind of flow together, but if you're creating your own quality assurance tool, if you're looking at an auto QA, the first thing is make sure that you're at 95, 96% accurate with the scoring.

25:09Also, have a little bit of leeway in understanding that your human agents that were scoring calls aren't always going to be right, and sometimes the AI will be more correct. Keep an open mind with that. Ask the quality assurance tools, what information am I getting? How easy is this to use this data to coach? I think all of those things really play in. Is there a CSAT? Is there an NPS that you're starting to build out that we can kind of look for. And again, I think getting deeper into what type of coaching tools do you will you have? How will you make my agents better with this platform? But we're way beyond just being able to score calls at scale. But it's not how we're going to But it's not how we're going to make the agents better. And how will this tool help us? And so again, I think I would love to with a demo of auto QA, OTT OQA.com.