Unlocking the Power of Prompts: Enhancing CX QA with Proven Strategies

00:00Hi, guys, Tom You've heard me break down contact Call Center Geek a weekly podcast with a focus on all things call center We'll cover it all from call-center operations, hiring, culture, technology, and education to give you actionable items to a improve the quality of yours and your experience this is an evolving industry with creative minds and ambitious people not only is his passion call center operations, but the CEO of Expivia Interaction Marketing Group and the call center geek himself, Tom Laird. So I'm going to just kick this bad boy off. We've done a lot, about, the enterprise guys have. I don't think we're the enterprise guys have so many different tools and they're doing so many different things that we saw that there's And, you

[Gap 01:49–02:10: no reliable transcription]

02:10that we thes figured out, especially when it comes to prompting, especially when it comes to how does chat G. GPT utilize transcripts in the best way for listening for the specific things like how do you listen for empathy. How do you try to score things that are unseen

[Gap 02:32–03:02: no reliable transcription]

03:02I have 15 things that I want to talk to you guys about that prompts I want to talk to you work. So just the quick overview of how we do, is we have a S, SaaS product that we basically take a call and as soon as and as soon as we analyze that call, it goes out.

[Gap 03:47–04:13: no reliable transcription]

04:13We've decided that the best outputs at least to start with are the actual scoring of every customer and agent sentiment as well. But let's talk about some of these prompts and some of the things that if you're planning on doing this or some of the things that we have found, so number one is less is more for easy questions. So if you have a greeting or if you want to collect an email address, did the agent collect an email address. Right. That's really all you want to say. You don't want to get. We. We tried to do these things for everything and it just confused it for the shorter type black and white that's pretty easy but let me say this the word explicitly is like ingrained in ChatGPT so if you use the word explicitly sometimes we would use ChatGPT to help us with developing some of the prompting for each of the questions it would be absolutely exact So, so if there was anything relent, one of the things was a explicitly, please make sure the agent is calling customer service. If there was anything off, if there was a pause, that would score it as a zero or no.

05:36So, we have found that if you want to be exact, you don't have to really tell it to be exact. Just give it kind of that general deal and it works much better unless you have something You can't go a T dotted and a T crossed in an I, it's all going to be, be careful about being too explicit when you want to have something exact. Most of the time if you just tell it and give it kind of the rough example, it will work. Now, this is the, the cool stuff, right? So how do you have ChatGPT, talking about empathy, that was big for me and you could just say, well we want the. I'm so sorry to hear that or oh my gosh I can't believe that happened right now. You could do that and it's pretty generic because those kind of conversations can come up in a lot of different instances, but what we have found better is is to kind of use a lot of if-then statements when it comes to the more thinking. Thinking type questions. So you know it will say something and let me actually I'll, pull the actual I'll pull it up in a second, but basically,.

06:55Basically, what we say is, hey, can you look in the transcript, look in the transcript to find out any instances where the customer seems distressed. That was had a negative sentiment that was not positive. And then after you have found that, then- the agent isn't using kind kind of just a basic scripted response, but that they're actually using some words in there that correlate directly to what the customer has said. So we're not looking for specific keywords like the agent must say, I and so sorry for you to hear that, right? We got a little general with what the agent a little bit as long as it kind of correlated, kind of correlate to the actual problem and that the agent was actively listening for that. Again, if you go on my LinkedIn, I think yesterday I posted five core prompts I have the exact I have the exact say, hey, can S in a recording, but how do you take it to the next level, uh, to really kind of use the use, case that we want, so I think that that was interesting. Um, the other thing and I'm just kind of all over the board here. These are all random is, is don't tell ChatGPT to tell you when something is not there. Now, that let me give you an example, but it It would get confused a lot when we would say certain things like, please score this with full points if this is a sales call or a retention call, but score it as an NA if it is a password reset call., right? And chat G.P.T. would consistently confuse with what was what, even though we've done some things with even selecting what different call types can come in.

09:11And for our platform, it doesn't matter.

[Gap 09:14–09:41: no reliable transcription]

09:41So please only score this if it is a sales or retention call and then you kind of. of leave it at the end a-to-the-prompt. Don't tell it to the end if it's not the NA. I got crazy confused and that was super- But, it would get frustrated with that. are there there, but it's more difficult to kind of look for things that aren that aren, why it why it gets confused. The other thing that I think is pretty cool is, is, how do find in the actual transcript, but up, so for scoring for the un- we basically say things and again, again, this what do we know? An agent has to get some specific information screen, certain things like, there's a promptness and providing that a information, right? So if we say, all right, let me I need to pull that up or, or, you know, something along, there's if there's a delay in the in the actual talking, they're probably being that piece of information quickly. Can they transition between topics, like if there's a big change in topic. And again, if the, if the question is, did the agent read the proper disclosure, or Let's say, did the agent find the information confirmation of actions, minimal need for correction they that prompt that basically said how quick did that did that agent really find this Now, things like, did they click the box for, opt out of email?

13:10We actually look for, for a little bit of a delay. So if a customer, if that If that's a question, in the, the, the, the, would you like to opt out of our email and the customer says, yes, I do, if the agent says, okay a second, all right. we've been able to find in all of this data that

[Gap 13:58–14:28: no reliable transcription]

14:28imperative work the best. So we basically said btI, it is imperative we have found and those words and I'm bit for service, please look for that in the first five lines of the transcript. So we have found that that has, I don't want to say significantly, reduced the allotted amount of time that it takes for a QA form to come back, been more accurate because it's not looking at everything. And it has been a little bit quicker. The more that we've implemented things, you can do the same thing for the closing, right? Because you're not going to have a closing at the beginning. So why have it read through the entire transcript for all of those things? know those I really got excited with the thinking questions the black and white binary did the agent do this or that everybody knows that ChatGPT could handle that but I think that the nuance to any of these companies that are going to try to do this is how do they handle the the empathy questions the did the agent do appropriately throughout the call where it's just a black and white but it process of maybe multiple sections of the call and I think that it can be done.

16:42I think we've done a really good job with you know the cool thing about this is being able to test this with our actual customers customers so pretty much every single customer we have on our BPO is is utilizing this now so our QA department haven't got rid of any Q people or yet they are we're basically scoring a call human beings and they're we're just doing that all day long. All of these prompts work. We're now kind of, I don't want to say we're hands-free, but we're at a point now where, you know, I think that the core basic prompts that everybody has, right? Everybody's going to have like an opening, a closing. They're going to have a greeting. They're going to have, you know, did the agent use a proper tone? Did they use proper word choices? Did they not use him at all?

17:29Did they not have diminishing language for the company? like these like 10 things that we know work really well or 15 things, that be a five-stop version and have one prompt that has everything and I think that could be one option. Obviously, we have a slicker version of that and there's. There's a a lot of, of companies that are slicker version. This isn't just us, but that's the thought process that goes into it a from understanding how chatGPT thinks to get the best result and to get the most results. And I would say now again, like I said, all of our QA partners is utilizing this

[Gap 19:02–19:18: no reliable transcription]

19:18Do you guys have any questions? I, I appreciate everybody kind of joining. just trying to think of things like you know, we didn't really struggle we found that I know chatgbt has kind of a and again, I We just prompt for the JSON output in our actual static prompt. And that has worked, and we have a lot of flexibility then for a half hour. So we've utilized her a couple times at the very beginning a couple months ago the really helped us, some of some of the outputs, understanding, just look for certain aspects of the transcript, don't read a whole transcript to every single time time. If you know something's at the beginning at the end, understanding that, you know, the structure, right? of how chat.GPT's quote-un in-quote mind works. I think all that helpful when we're going through our prompting.

[Gap 20:57–21:14: no reliable transcription]

21:14I joined

[Gap 21:44–23:44: no reliable transcription]

23:44If that if that kind of answers your question. Yeah, that's a great thank you, all right All right, let me bring yep Thank you for this session. Same as Jeremy, I apologize. I joined a little late, but All right, Chavi, you're up. How you doing, buddy? I'm good. I'm good. I'm good. Good to talk to you. Yeah, thank you for this session. Same as Jeremy I apologize I joined a Ask the questions that are related to the a form provided the intro and what it is that Use language, do not use negative terms like me or poor or weak It's got all that context before it the answers it in addition to that to Jeremy's point What is it that? Shagpity can do that we have to rely on a quality analyst to to do We've started to So, I want to learn from, the context - point the transcript was garbled you know something like that we also found that we would ask confidence score as we're

[Gap 26:55–27:17: no reliable transcription]

27:17oversee scoring for quality,.

[Gap 27:47–28:17: no reliable transcription]

28:17Derogatory or poor Um comes to the actual summaries We do we're calling it the the rationale right now So if a ChatGPT it does if it scores it as a yes, like it got gave a full points We don't say anything, but if like but after that it's just each question we're calling a context but the context is just the mini prompt to find that question and then that's basically and then we just find the output of how we

[Gap 29:13–29:25: no reliable transcription]

29:25done I don't want to say a thousand of these, but hundreds upon hundreds of these myself K the call summ Aries have been

[Gap 29:31–29:40: no reliable transcription]

29:40pretty much on point with what the call is black and white. Um, in there to please in the call summary, you know, talk about It probably does mean based on the, umm, you know, how complicated, how complex, you know, the calls are. We're talking about BPO, financial services, retail, tech support, The I don't know, say 4 to 10 minute type calls that a lot of them them are extremelyB question that we have one client that has, seven different call types that come in that have seven different, um, types S of different calls that all have different scoring and correlate to different types in calls. We've I mean, I guess, I, I guess, I really haven't seen too much kind of derogatory language or, or those types of things, you know, with the, with the outputs, but I'm going to probably look out for it maybe a

[Gap 30:44–31:17: no reliable transcription]

31:17We'll continue to kind of do this, I think it's been interesting to kind of go go down this path and then I know there's a lot of you who are interested in this stuff too and it's it's a little with you guys