Everything I Have Learned about Ai Powered QA

00:00Hi, guys. Tom Lair. Laird here. You've heard me break down contact center technology, technology on the podcast time and time again., now it's time for me to help you with your challenges. As your contact center technology advisor through our new company Expivia Digital, I, can help you with things

[Gap 00:13–00:30: no reliable transcription]

00:30advice from a cousin, a guest, contact centers, Thank you so much for watching, see you next time. people behind the operations. Welcome back everybody to an advice center geek, the call center and contact center podcast. Give you, you some action items to take back in your actual contact center, improve the overall quality, improve that agent experience, So, I hope everybody's doing well. I know I'm trying to get back on track here with a more consistent podcast schedule. And I think, you know, we're going to do that here hopefully this month, as things have kind of calmed down a little bit work, I want to talk about QA. Again, you know that's what I'm really passionate about, and I think I got some really cool insights for you here. I don't want to talk too much about the technology piece. and helping customers recreate or look at their QA A, everybody just has that Excel spreadsheet

[Gap 01:41–02:03: no reliable transcription]

02:03to this. The first. The first And the reason, is because we want to have a benchmark for calibration, and I, think if you're using any type of AI tool, I think it's really, really important. Right? That you have at least something that you've been. Been using to benchmark off of. And then you, you can kind of see the progression as we change your form. We try not to- unless you Again, really, a really, really important, make sure and calling that engagement, right? Having script like I'm You can define engagement by looking at sentiment, by looking at outcomes of the call, looking at the attempts of your agent and what they really did. Did they really connect? Did the agent listen? they control the call. did the customer feel heard? All of these questions we can start to answer now with AI I, that you don't the have these rigid, really rigid things that before was the only way that a human being could score these things. I think that the I CCaaS is getting rid of kind of these things that we've been doing for the last 20-30 years in a manual scoring sense and really making this AI powered where you're looking at the outcomes of the call, many outcomes during the call.

04:38Interaction with the agent and I'm going to train our agents and educate our agents onB Just actually talking and engaging and not having to give these stupid false Phrases out because again, we call that something we call that empathy or you know I think that's the biggest change that I can see from again as we move even the human customer experience is going to get more difficult. And having that type of non-scripted, non-robotic responses and being able to score it, I think is really, really important. [QA coverage remark omitted] [pricing remark omitted] That, that.

05:54That's a lot of calls. at kind, if you look at anthropic, if you look at sonnet, right? If you look at even the high end

[Gap 06:08–06:27: no reliable transcription]

06:27to the point where you can totally have a very comfortable sense that everything is correct, everything is accurate, and everything, everything is consistent. So, what we say is that, again If you have a score that's in it that's your human beings and let's say it

[Gap 06:56–07:25: no reliable transcription]

07:25team scores, we're all going to be a little bit off. AI -powered QA is going to be more accurate than that, right? [QA coverage remark omitted] It is something that you not only don't need, you shouldn't have right now, until AI gets much cheaper, that it can score more accurately, because they're not using the model. And again, ask them, go on their websites, ask how, accurate those guys are, right? Right. The big QA a about 85% and they kind of getting even closer to as accurate as they would get if they used a percentage and calls at the very high model, that makes sense. We did a study and I found a study online too, right? There We found that about 30 to 40 percent and I've found this online as well from other people that I've talked to 30 to 40 percent of the time human QA evaluators disagree.

09:08To create. To create analysts listen to the same call. They use the same form and 30 to 40 percent almost 50 percent of the time they come back with different scores. Right. So it makes sense. Right. aInitialized. tired on a Friday. They just want to blow through it. Somebody is more strict on an empathy type question. One and interprets call control this way that we have as humans. And I think that's why AI is in more accurate. It's not only more accurate, it's more consistent. Because it just scores the same way, the same standards, if you're doing this the right way. The hardest thing that we found with auto QA was not the the accuracy so-to-speak and I guess you can call this accuracy but it would be scoring

[Gap 09:54–10:24: no reliable transcription]

10:24The real power of AI Power QA is in the aggregate data, individual calls are cool, and individual scoring is fine, but you start to see patterns that humans couldn't find before. you start to see things like 80 or let's say 19% of the time they're not using this rebuttal. This agent on a Monday, 62. percent of time doesn't show empathy, right? Maybe she's just, she doesn't like Monday's or, he doesn't like Monday's, right? You, can start to see things in the aggregate on your agents, and on supervisor teams, and then your overall contact center, that you could never see before on just an instant dashboard. That's real power, right? It's not just, you know, did Jane, Jan. [QA coverage remark omitted]

11:19Insights that, that they could never have and, and again, if you're doing, two calls per agent per week, you have no idea what to train, right? That might've been an outlier and now you're, you're hammering that agent on something they, they you know, calling nitpicking on it, right? It's it. I think that's the step that's extremely powerful. Um, I think how we educate and train and that kind of I want to say a little is is going We're going to be massive as we move forward with AI taking over, again, some of the, quote unquote, easier stuff, right, that everyone's been talking about. That still frustrates me because we've been taking away the easy stuff forever, but again, the harder stuff. I think that's no doubt, that need a We also need agents that don't give scripted responses, that can actually interact, that can really show empathy, that understand how that other person is talking, that we're not check-barking boxes on scoring words and phrases.

12:48We have to... to get rid of that scoring words and phrases, type deal, because we can now educate at a much, much higher, higher level. I think that's really important. The last-thing that we're seeing, That has been really, really, really cool, is looking at what we call redo, and looking at AI in a different way, looking at how do we make avatars of customers, real customers. So again, when you have a call. and you fail a call, you had in 75 and passing 80, to be able to take that same Haul with the same customer. Same customer, same name, same issue, create the AI avatar, go have that conversation again. That's the stuff that we're doing on auto QA. And I think, we're trying to think through this AI thing differently than just voice bots and all the borings stuff, there's so much out there from a QA standpoint that can really be done to educate and make your agents better.

13:50And we're gonna need these tools as we kind, the next two, three, five, seven years, of AI will be taking certain things over and how that human touch kind of really needs to kind of change. So again, some really cool things I think that we have found with AI Power QA. Getting a couple of things on the technology part. You've heard me talk about this a we talked about earlier, right? The accuracy was okay, but at the beginning, the consistency was a struggle. So how do we fix that? There's multiple ways that we've figured out from a prompting standpoint, right? The main one that I always talk about is kind of the reiteration. [QA coverage remark omitted] That's why I think we're the most accurate. Is again, we will have the AI ask, you know, have it read the question and answer it.

14:41So, hey, did the agent use the brand of greeting? Yes, right? And then we say, okay, hold that in your memory and now ask that question again, did the agent use the brand of greeting? if it says yes again. then we score the calls a yes, put the context why. If it does, I mean, if it says, if it says two, two-and-a-half years, that was a struggle at the very beginning, right? And I think we've come such a far way and so have a lot of other companies as well with how accurate. And how good this stuff is. But I think, again, if you start to look at, hey, I want to train my agents, I want to look at trends. I want to look at things in the aggregate, you know, with AI Power QA, you H can use it as an analytics tool as well.

15:44You don't just have to score agent-agent behaviors. You can look at things. And we do this for a lot of customers have zero score sections with maybe 10 or 15 questions about. What are some things that they? stuff you want to know the main things, how do I affect my business? How do I affect my customer support? What aspects can I tell marketing about what we're hearing? What can I tell rales about some of these things that Again, please, come take a demo with me. I want to show you our tool. Even if you and if you don't use it or you don't buy, I want you to think differently about how AI can really power QA and can do things at a whole different level and make you kind of hopefully rethink how QA can be done and then have a quality aspect of your content as well.

16:54Thank you guys very much. I hope that this QA thing, this Q a thing has been an awesome experience and is taking us in so many different ways. So hopefully I can kind of help you kind of think a little bit differently just as I have. Thanks. Talk to everybody next week.