Legal Leaders Exchange - Podcast episode 16

Change management is hard, but even harder with AI

 

For this episode of the podcast, our own Abhishek Mittal and Jeffrey Solomon discuss the challenge of change management, particularly with artificial intelligence technology. While change management has always been difficult, it becomes even more complicated when AI-driven technology is being introduced to users. Our experts offer advice on negotiating the difficulties and increasing transparency to help achieve user comfort with new technology.

Listen to learn more about:

• The value of involving users in early stages, such as proof of concept and pilot deployment
• Advice for getting users engaged in processes involving the new technology that is being introduced
• The value of highlighting the technology’s weaknesses
• Making the process fun with gamification
• Planning for the learning curve associated with an AI solution

Be sure to follow Legal Leaders Exchange on:

Apple Podcast | Spotify | Google Podcast | Audible | iHeart Radio

Transcript

Greg Corombos

Hi, I'm Greg Corombos. Welcome to Legal Leaders Exchange. This podcast series was created to deliver insights on optimizing legal operations for corporate, legal and insurance claims professionals. They say change management is hard, but it becomes even harder if the new variable in this change is AI-enabled legal technology. Listen to our recommendations on how to create awareness with hands-on experience around artificial intelligence. Learn from early adopters and peers who have know-how on the matter, and encourage transparency around AI. For this episode, we brought in our experts Abhishek Mittal, Vice President of Data Analytics and Operational Excellence. And Jeffrey Solomon, Senior Director of Product Management. Abhishek and Jeff, welcome. I'll pass the conversation over to you.

Jeff Solomon

Welcome. We're here to talk about change management with AI today. With lots of AI powered solutions now becoming mainstream and legal in areas such as invoice review, contract management, and prediction technology, among many other things, AI is no longer some new thing that we're afraid of. We've seen that trend proven in recent GC and legal ops industry surveys as well, where a few years ago, the minority had or was looking to bring in an AI-enabled solution. And now it's the majority. We're going to talk about all of that today. I'm Jeff Solomon, I lead our LegalVIEW BillAnalyzer solution here with ELM Solutions. With me today is Abhishek. Abhishek, do you want to introduce yourself and talk a little bit about your role and your team?

Abhishek Mittal

My name is Abhishek Mittal, and I head our global Data and Operational Excellence teams. What that basically means is, we have brought in our data and domain experts together to create AI-enabled solutions for our customers, as well as improve our existing operations using these advanced technologies. So looking forward to talk more about that.

Jeff Solomon

Perfect, well, let's dive in then. Change management is always the topic with new technology, AI is no exception. They talk about how change is hard, but we have a lot of tips and solutions along the way, how we've worked with other customers, and some best practices along with that, for how you can make it a little bit less hard and get more people involved in the process along the way, so they're excited about it and not afraid. So why don't we start talking about a few recommendations around change management that can help ease our unfamiliarity. Abhishek, I know you've worked with a lot of customers on this, what are some of the things that you've seen work out well?

Abhishek Mittal

I really like your statement, I know, change is hard. And it's even more hard when there's an AI-enabled solution because AI-enabled solutions are not deterministic, right? Like software solutions, they had a very expected outcome, and you can expect that that will be repeated every time you use that product. With AI-enabled software solutions, because of the technology being probabilistic, the change management becomes even harder, especially if you are working with professionals who are used to very strong black and white areas, or expectations at least from the software side.

So some of the things that have worked as we have deployed these solutions internally, and also for our customers, is - first and foremost - awareness, with hands-on experience. So what I mean by that is doing a proof of concept, doing a pilot with the users, making sure that before they go into the real production, that end users have had a dry run with the solutions over a period of time. And they also have seen the benefits of that technology, and that will help earn the trust. The second thing is, it's always easier to learn from a peer, right? So creating some change agents, who can then work with data science people to create that, like be the early adopters. And then they are the ones who teach other experts. And that has worked really well in many of the cases, as well.

Two more points: the third one is about transparency. That's a big deal, right? Usually, we only want to talk about the good things, and the features and functionality of the software. My recommendation with AI-enabled solutions is to show the weaknesses as well. Because it's, again, it's not a deterministic solution, showcasing examples of where it's not meeting or where it's low-confidence recommendations. And how do you handle those scenarios as an operator, as a domain expert? That will give them more confidence how to work with the solution. So I think having that transparency in the weaknesses is actually better for the overall program.

And last point, which I think has really worked -- and I can share some examples as well -- is deployment models. Usually what are typical processes of deploying software is we look at the existing process, we either try to deploy the new technology to automate or eliminate that process. And we want to sort of have a champion/challenger model, right? That is one way, where you have a champion and challenger model, that existing process is the champion and this new AI and process as the challenger, that's one way to do it.

The other way to do it is use it as a quality assurance as a second layer, as a competition, make it as a productivity enhancer, to the existing process. And then when it becomes, the crust is there, and then maybe you can have like the champion challenger model. Or you can even have, sort of a segmentation model where part of the volume is being handled by your AI-enabled solutions, and over a period of time you increase the volume going towards that. So those would be my recommendations to make change management easier for the AI-enabled solutions.

Jeff Solomon

Great, thanks for those, those are all great points, I just have a couple of from my experience that I'll add to that. The first is as you're looking at, and ultimately onboarding a new solution, bring your users into the process, having them onboard and engaged in the discussions. And you know, as I said, well trained on what the goals are, and what the benefits are. It helps make them allies who are proponents of, you know, the support that's being added to them, rather than somebody who's a little bit scared, and put off by the fact that there's additional support available for them.

Also, we're still relatively early in the AI journey, but we're not as early as we were three or four years ago. The likelihood is you're looking, with solutions, is, you know somebody who's done this before one of your peers at another company, a friend that you've worked with a past life, talk to them about what's worked for them, what lessons they've learned coming out of their process, maybe if they didn't kind of follow all the steps along the way in order to build that trust and deploy the right solution and everything. And, you know, what did they run into? How did they get around that? That's a tool that you can have out there as well. Moving on, real world examples always can help. Abhishek, you have a couple of real-world examples of where we've done this in the past that you can raise here.

Abhishek Mittal

Yeah, one of the areas that I really like, is gamifying things. Because when you're playing sports, your competitive spirit comes out, right, even in professional settings. So there was this, even in our LegalVIEW BillAnalyzer in the early days, when we were trying to create adoption internally for our teams. What we did was we gamified it in the way that we gave a set of invoices, a large set of invoices to one team with AI and one team without AI. And then the idea was who can... there was a prize at the end of the day who has better efficiency and better savings. And overall, you know, there was a overall score. And then we gamified that experience. And I think initially, obviously, there was a lot of spirit, and everybody was like, we can beat the AI. But as the number of turns happened, people were like, Okay, I want to be part of the AI plus expert team. So, I think that creating that... and that became fun, right? That became fun, that I have like this competitive spirit where I have to win something. And that way everybody started getting exposure to how to make AI work for them. So that that's one example where we sort of gamified the experience to get the adoption.

The other example is, this is for one of our larger legal operating centers where we handle a lot of legal matters. This is similar to what I was saying earlier, that there was a very strong distrust by the experts that we know this process, we have been using doing this for decades. And any new piece of technology, it has errors, it has now the new word "hallucinations," it is not very reliable. And the errors that the AI technologies were showing people were just pointing to that and the ball was not moving forward.

So we said okay, agreed, the point taken, that it's not a deterministic solution. So what we did was we said, okay, this will run behind the scenes. And it will actually, instead of humans checking the work of AI, we said, Let the AI check the work of humans, and see whether we are able to find any opportunities. And we ran that in a silent mode, if you want to call it that, behind the scenes. And then for the process owner, we shared the results where AI is adding value on top of the expert review. And that gained an appreciation of the technology because they now could see that okay, this can help me make my outcome to my customer better versus more of like a threat. And I think that really helps in this legal process matters. So now it's being used significantly in not just the review process but, once they know how to make the tool work, the experts actually have come up and added into other parts of their processes well. So I think that's another great example of sort of trying to make it a tool rather than a competition.

Jeff Solomon

Yeah, definitely those are, those are good examples that we definitely run into. Now, kind of moving on from how we prepare for change and how we manage through change, let's move on to talking about some recommendations around actually selecting an AI-based solution, the software involved. Obviously, I'll kind of start with know the problem you're trying to solve. Don't bring an AI for the sake of bringing an AI. But know the problem you're trying to solve, make sure it is the top of your list of priorities, so that by solving this problem, you will either make it much better or completely solve the that specific goal, not just kind of add something else to the process, but keep everything else the same. So you have any recommendations around selection, vetting of any software.

Abhishek Mittal

Yeah, I obviously create software. But as part of that, I also buy a lot of software to enable our machine learning teams and our operation teams. So this is an area where I spend a lot of time, because you bring in a piece of technology, you want to make sure there's a whole change management journey on it, and you don't want to get surprised, and then pivot again, because it requires a lot of effort, and you lose credibility as a leader as well. So this is an area which I really, really, personally have a lot of worry as well as I have focused a lot on to make this a good experience for employees, because I lead large operations teams, and I want to make sure that they don't go through this change multiple times. So the first thing is to have a very clear baseline, or KPIs. Whether you're trying to improve your cycle time, whether you're trying to improve your compliance rate, whether you're trying to improve your quality. Whatever is the KPI that you're focused on improving with the new technology, having a very strong baseline on that is super important. That's the number one thing we do.

Now there are various ways to measure it, obviously. The reporting teams... one of the other things that I've tried to do is actually visualize the way people spend time today, before a new piece of technology comes in. So now there are various tools, which are process mining software, which is available, we use some of them, which sort of looks at how much time people are spending on the core, sort of operational platform, how much they're spending, maybe on Excel or something else, or maybe outside the system, as part of their daily work. I know it might seem intrusive, but it's super important to do it at least on a sample with their, obviously, awareness so that we can then identify, both pre- and post- implementation, how is the work of your operations changing. So I think that's again, having a good KPI or having a good process mining kind of visualization done of the how people are spending time on their daily tasks, both before and after, will really help and that will show you whether the software that you're trying to invest in, is it working or not.

And obviously, I've missed the part saying that, before you buy it, having a POC done right away with this software so that you can see the impact on that KPI that would be part of that evaluation process as well. Then, the couple of other points I would say is look at the support model, especially with an AI-based solution. Support is usually, may not think of that much because software is now everywhere. But for an AI-enabled solution, models need to keep on updating. And it's very important to understand what is the support being provided to you by the vendor? Is it a combination of just customer support experts? Is it data and domain experts behind the solution? So look at the support model of an AI-enabled solution, because you're going to learn how to use the software, the software is also going to learn from your behavior, and it needs to be calibrated, at least, if not updated. So that support model will be critical for you to have long term success.

And lastly, for the AI-enabled solutions, we just should not look at automating the process. But it should be what can we do with this technology which we are not able to do today. So obviously I know you're we are looking to improve efficiencies, but also see if we can create some additional value for the process. I'll give you an example of this in the legal bill review space. Let's say you have 100,000 invoices to review on an annual basis. And today with your existing team, you can, let's say, review only 30% of those. The objective of an AI-enabled solution should not be to just review and get more savings out of that 30% you review, but it should be can I review 100% with this AI-enabled solution, or maybe a larger percent of that, and get the savings and prioritize it. Prioritize the ones where there are more dollars available. So using the technology to offer the strengths rather than just replicating how the process is right now would be the last thing that I would say when you're looking at AI enabled solutions, that maybe it's not just a replacement, but it's more of an addition and taking away part of the workflow. So that would be my four things on this space.

Jeff Solomon

Yeah, thanks, Abhishek, I'll just add a couple to that, from my experience. I mean first, building on your first point about baseline KPIs that's critical there. Identifying your goals, it could be to save money, improve accuracy of whatever the solution entails and reduce risk, gain efficiency, or in some cases, do something you may not be doing now, because you don't have time. I would also say, ask for references. Any reputable company with a with a strong solution that aligns with all the things Abhishek just talked about, they'll be happy to provide you, at the appropriate time in your discussion, a couple of references of customers like yourself who are already using the solution, getting benefits from the solution. And ask for that discussion to be one-on-one. The vendor doesn't need to facilitate you having that discussion. They shouldn't be afraid of anything, as we're not.

And I think the final point is, and this encompasses both the change management and the software selection piece, but have buy in at the top. Anytime you make change, some people just don't like change, there's always going to be detractors. But having support at the top ensures that any of those initial complaints, the noise that comes up during the process, that it doesn't derail your project. You made this decision, you did your due diligence, you got to the right place, you picked the right solution. A couple of noisy people don't necessarily need to put you off the rails. So when you have the right buy in on that and the right support across your org, you can work with them, train with them and nip that in the bud pretty quickly. I think we got through most of what we wanted to cover today. I know we talked about a lot. Any final comments before we close?

Abhishek Mittal

Yeah, don't just treat AI-enabled solutions as a piece of software, think of that as a way of changing your process and to make it a success. I'll repeat something that you said, look at the metrics that it's generating and use that to create awareness in your leadership team, as well. Because initially, AI-enabled solutions, the journey is the harder part, right? Because the model might need clear calibration, your user needs trust. And it actually might be an investment. So, plan for that, whenever you launch a new AI-enabled solution. I think there should be a plan for initial overhead, that it will take that learning curve, and then you will get the x-scale benefits from it. So rather than thinking of it as a failure, that oh, I'm not getting the immediate benefits, think of it as a lack of planning there.

So I would say in the planning process for launching a new software in the first, depending on the problem area and the maturity of the vendor and the solution, but whatever it is plan for that overhead in the beginning months, and consider that part of the change management cycle. That would be the last thing I would say on chain management, if you're planning for that overhead in the beginning, and iterating with the solutions and the support model team, AI-enabled solutions, especially the way that technology is advancing, I know we can talk about the favorite topic of everybody right now, which are the large language models. But it's the same thing at the end of the day, technology will keep on improving. But these things on change management, I think will be applicable even for these advanced technologies.

Jeff Solomon

Thanks. Yeah, I couldn't agree more. I'll just close with, actually building on Abhishek's point, which is setting expectations appropriately. So, anticipate some sort of ramp as you bring in a solution and you do all your work along the way and you identify what your goal is and what the outcome should be. Set the expectation internally that the check-in of whether you've met those goals or not, isn't a week after you implement it. There's a ramp, there's a curve, as Abhishek said. So just make sure you're trending along that curve, to ultimately get to the, you know, to the point in time where you are at the full expected productivity. Depending on the solution, whether that's, you know, a couple of weeks, a couple months, could be a year depending on what the solution is, but expect progress along the way as you ramp up to being at that steady state where you're looking to be. Abhishek, thanks for joining me today. It was great talking to you. And thanks, everybody, for your time. Hope you enjoyed it.

Greg Corombos

That was Wolters Kluwer ELM Solutions' Abhishek Mittal, Vice President of Data Analytics and Operational Excellence, and Jeffrey Solomon, Senior Director of Product Management. This podcast is hosted by Wolters Kluwer ELM Solutions, the market leading provider of enterprise legal spend and matter management, contract lifecycle management, and legal analytics solutions. For more information and additional guidance, please visit wolterskluwer.com or call 713-572-3282. Please join us for future podcasts on optimizing legal operations and achieving your legal and business goals.