GENERATIVA AI IN YOUR BUSINESS

0:00 TODD GLICKMAN: But now, I'm very pleased to introduce Dr. George 0:08 Westerman, Principal Research Scientist at the MIT Sloan School of Management. Dr. Westerman is a world-recognized thought leader 0:16 in leading transformation and competitive advantage through technological innovation. 0:22 He works at the really exciting intersection of executive leadership and digital innovation 0:28 to provide continuous, valuable insights to leaders in all industries. George, welcome. 0:33 GEORGE WESTERMAN: Thanks, Todd. 0:39 It's really great to be here. And it's especially great because at home, we 0:44 got 5 inches of snow yesterday. And there was no snow on the ground here. So that's a good thing. 0:50 So yeah, I teach in the management school. And so what I focus on-- he said technology, innovation, 0:56 and leadership. I try to take technologies and make them-- to demystify them, to help leaders understand what's happening there 1:03 and really think about how do you design your organization and how do you lead transformation? So I did some of the earliest research 1:10 on digital transformation 15 years ago. To tell you how long ago that was, nobody used the word digital transformation back then. 1:18 And so I wrote some book on that. I had done a work with IT organizations before that. I've done work in skills and careers. 1:26 And now, of course, AI, all of those things come together. AI is affecting all of that. 1:32 And so what I want to do today is talk to you a little bit about how do you think about generative AI and how do you work that in your strategy? 1:40 Now, Eric has been doing AI for almost as long as I've been alive. 1:46 And I want to try to tell you about AI from a management 1:52 standpoint. And that means it will-- I'll try to make it crystal clear for you in trying to make 1:57 decisions in your organization. But it also means that if Eric and I disagree, he's correct. 2:05 So that's where we want to go. My job is to be as clear as possible, help you figure out the questions you can ask, 2:11 the instincts to develop, as you are making decisions, because all of you, probably, are having to make decisions 2:17 with AI. How can AI help you understand that a little bit better? Or if I could say it a different way, 2:23 how can I make it a little bit less scary and make you feel more competent in dealing with this stuff? So are you ready to go? 2:31 Three pieces here-- number one is, what is AI? Now, Eric already talked about what is AI. 2:36 I'm going to give you my version of it. And think about these elements of the different kinds of AI 2:42 and how to think about it. Number two is GenAI because that's really where we're spending a lot of time now, 2:47 a lot of attention on ChatGPT, and Gemini, and Midjourney, and all these, Sora, and all these others. 2:52 How do you think about making that work in your organization? Because it's tremendously powerful, but also very risky. 2:59 And how do you think about that? And then the last is, what does that mean? How are companies innovating with this right now? 3:06 So first, what is AI? And like I said, it's a little bit scary, after having followed an AI professor. 3:12 But like I said, it's mostly correct and hopefully very clear when we talk about it. What is AI? 3:17 Well, the thing about what is AI is, it's really hard. The definitions Eric gave were from 2003. 3:24 Now, Google does not track Google Trends back in 2003. They start in 2004. 3:30 And look at generative AI. Nobody was talking about it because it didn't exist back 3:35 then. But then deep learning, he talked about that, and you could see how that came on 10 years ago or something 3:42 like that. That was really cool. We had a thing called the Work of the Future Initiative. 3:48 And the question there was, with deep learning doing what it is doing, what jobs will be left 3:54 after the robots eat the jobs? And we found that, actually, they don't eat the jobs. They eat pieces of the jobs. 4:01 And also, there's a lot of extra effort that goes in. It's not just the technology. But that was the scariest, most interesting AI. 4:09 And now, what do we call that? We call that traditional AI. That's how fast these things move. 4:14 Go back a little bit farther, this is machine learning. And you can see, why is it bigger? 4:22 Well, it's actually a superset of these other two and then artificial intelligence. And the interesting thing about this, you see, 4:28 artificial intelligence was more common. And now, it's less common because artificial intelligence 4:34 back then was a different kind of thing than it is now, just like Eric was talking about. So with these elements, how do you make sense out of this mess 4:43 when it keeps moving so quickly? I'm going to really first give you my important law 4:49 that if you remember nothing else, remember this. You know Moore's law, Moore's law and the exponential. 4:57 These things keep growing at an exponential rate. This is Westerman's law. 5:04 Westerman's law is different. Technology changes quickly. Organizations change much more slowly. 5:11 I know this to be true. I should not have to say this is true. But I know it to be true. And you know it to be true. 5:16 But our technology people often forget this. And your technology vendors often forget this. 5:23 The hard part is not adopting the technology. It's changing the way you do business. 5:29 It's not the digital. It's the transformation, is the hard part. And so what does that mean? 5:35 Technology is not the problem. Transformation is. So it's not really a technical problem. It's also very much a leadership problem. 5:44 We need to get both of those things in there. And so, for example, in talking about AI, 5:49 we've talked to two-- we've talked to a lot of executives. But here's two that we can quote. Matthew Evans, who worked on AI for the manufacturing 5:57 process at Airbus, he says, "Strictly speaking, we don't invest in AI or natural language processing or image. 6:04 We are always investing in a business problem." Or if we go on to the Home Depot, Fahim Siddiqui-- 6:11 another thing I do is I run the MIT Sloan CIO Leadership Award every year. 6:16 And Fahim was one of our finalists last year. And Home Depot is a home goods store. 6:22 If you need hardware, or saws, or wood, or anything, you go to Home Depot. 6:27 I'm renovating a house right now, so I spent a lot of time in Home Depot. And Fahim and his team have created a really good experience 6:34 for a really, really complicated store. He says, we should always be looking for extraordinary experiences We want to bring joy to the user. 6:43 We want to delight. The technology-- that's secondary. And once again, we forget that all the time. 6:50 But we can't forget that because technology provides zero value to your company. 6:56 What you do with the technology is what creates that value, how you change your business or your products to make it work better. 7:04 So what's the first thing from a management side? What's the first thing you need to know about AI? 7:12 Artificial intelligence is not intelligent. And Eric mentioned this, and we should continue that. 7:22 Basically, it's a program that executes, but it doesn't have that context knowledge. 7:28 And we have to remember that. It's basically executing a formula. And that formula is what you've programmed it to do, 7:34 what it's learned how to do. But it may not be the right thing. 7:40 Aude Oliva, who has created AI that can actually read what you are thinking, with brain scans, 7:47 one of the smartest people I know, she says, artificial intelligence should be artificial idiots. 7:54 That's one way to think about this. It's not intelligent. But the thing is, it can act very intelligently. 8:00 And that's very useful, as long as we are careful, as long as we do it in the right way. 8:07 So the digital transformation research-- this is an update. We started this in 2010. 8:13 This is an update from 2021. Where can you look for opportunities not from AI, not just from AI, but from mobile, and from Internet of Things, 8:22 and from other technologies you're facing? Four good areas you can look-- one is creating an emotionally engaging, targeted, personalized 8:33 customer experience, number one. Number two is operations, not just automation, 8:39 but the ability to adapt and adjust as things go on. 8:45 That's industry 4.0. Business models-- not just being the Alibaba or the Amazon 8:51 of your industry, but what can you do with information to make more low-hanging fruit, turning 8:57 your products into services, these kinds of things? And what we rediscovered-- we should have noted before-- 9:02 employee experience. Employee experience is a critical part. We know that satisfied employees lead to satisfied customers. 9:10 We also know that if you have a bad employee experience, it's a very good indication that your systems and your processes 9:18 are not running the way they should or your incentives. So when you're looking for what to do with AI, 9:23 it's the same way we think about any other digital transformation work. You can look in these areas, and not only in these areas, 9:31 but the opportunity to go across, like Home Depot has done, like Airbus has done. You'll see also that this rides on top 9:38 of your systems, your data, your systems. And if that's a mess, it's hard to do this. 9:44 AI actually helps a little bit with that messy data. But the better your data is, the cleaner your processes are, 9:51 the better you will be. So I really think of AI as the next stage 9:57 of digital transformation. The same principles apply. But there's more that you can do. 10:02 And there are more powerful opportunities. And so we still need to lead this, but we need to think more and more about what we can do. 10:09 So here are some things we could do with generative AI. I have designed courses over time. 10:16 We're doing one in the Philippines right now, a set of courses. And instead of me standing up, and putting a suit, on and going 10:23 in front of a camera, now we just type words in, and she will say it instead. And if you don't like how she looks, 10:29 somebody else that looks different will do it. And they can do it in a different language. And frankly, I can do it with a deepfake very easily. 10:36 I don't need to go to a studio anymore. Also, any corporate literature you have 10:43 can be in any language you want, instantly, with a push of a button. 10:48 Over here, certainly coding-- many people are doing coding right now. How many of you are your programmers? 10:54 Are they using Copilots and things like that? Well, if they are not, it's time to start. 11:00 This really does help. Now, there are questions about whether it's better for the senior people or the junior people, 11:06 but there's a learning process. And it is better. It's not only better for coding, but also enforcing standards 11:12 and making documentation. People hate doing that. The computer can help them. 11:17 What else is there? Well, Cresta. Cresta is a call center tool, especially for sales. 11:23 And you heard some really good startups. In a randomized trial at MIT, they found that everybody using this tool got better. 11:33 The most senior people got 14% better. The most junior people got 34% better. 11:41 If you have the senior and the junior people, they raise the bar for everybody, and it moves up. What does it do? 11:47 It listens to you talk and it gives you hints while you are talking. That person is getting confused. 11:54 Explain the product better. That person is getting angry. Try this to calm them down. 12:01 And then at the end, it comes back and says, looking across what you've been doing, you don't close fast enough. 12:07 Try to get to the sale faster, these kinds of things. It becomes what a good supervisor would be. 12:12 But it's with you all the time. And I'm working on a project right now with the media lab, where we are creating a personalized 12:18 tutor for the programming class, the first Python class, for minority institutions, minority learning institutions. 12:31 The idea is, if we can get people through that first class, with an individualized tutor, they are on a set for a career 12:38 that they never would have had possible. But people drop out of that first class because they do not have a tutor. So we're developing that right now. 12:44 And then last but not least, those of you who are using any products, whether it's SAP, or Workday, or Adobe, this is being integrated 12:51 into all of your products. So this is happening all over the place. But once again, it's not the technology, 12:57 it's what do you do with the technology that makes it better. The other thing that's really important here is we talk about generative AI solutions. 13:05 But most of the best solutions are a combination of generative AI, traditional AI, boring old IT, 13:14 and also people and process in there. So Lemonade, they have a very specific niche 13:21 in the insurance market. They write 98% of their policies, 98% of first claim notices, and they handle 50% of claims automatically. 13:30 Through this combination of different AIs and also traditional systems, what about that 50%? 13:36 Well, the easy stuff, the computer does it. And the hard stuff, they give it to a person to figure out what to do. 13:42 That's how their model works. 13:47 And this is Sysco. This is not the technology company Cisco. This is Sysco, the food service delivery. 13:54 They have one of the largest truck fleets in the world. Restaurants around the world get supplied by these. 14:01 And here are all the different ways that Sysco is applying AI in what they do. 14:06 Now, this is not a high tech company. They deliver cans of stuff to restaurants. But you can see all the different things 14:12 they have on the customer experience side over there and also on the back office. 14:17 And when Tom Peck, when I was talking with him about what he does, all those little brains, 14:24 those are opportunities where generative AI can help. We think about generative AI in helping a salesperson figure 14:33 out their call planning. But how about generative AI in helping you route things around a warehouse or a-- 14:42 we don't have this thing they want. We don't have this kind of mushroom. But this kind of mushroom will work, so all kinds 14:49 of opportunities there. So if you ask three AI experts, what are the categories of AI, 14:58 you will get five different answers. Nobody can really agree. So I'm going to give you my four categories. 15:05 And I'm not going to tell you this is right. I'm going to tell you it's mine, if that helps, 15:11 so four categories. And this should look familiar because Eric talked about a lot of this. And what I want to do is walk you through these, 15:18 just to give you a little bit more instinct on what to do about these things. So number one, rule-based systems-- Eric called it expert systems. 15:25 Put a lot of if/then statements together. 1984, when I first started programming, 15:30 one of my first jobs was to create an engine for a rule-based system. And like Eric said, these things don't work beyond the simplest 15:38 problems. But now, rules engines are better and if you use them in the right environments, prescriptions, 15:45 loan making these kinds of things. Within the limits in which they are good, they are useful. 15:51 But don't try to get outside that context. Then there's econometrics, which is statistics. 15:58 Most of us learned in school deep learning, generative AI. Let's walk through these things. So expert systems, we talked about this. 16:05 How do you program them? You talk to an expert. And the expert tries to figure it out. 16:11 But there's a nice paradox, which is that we know more than we can tell. Or as Eric said, your two-year-old 16:17 understands physics but couldn't tell you how that works. And we have that problem all the time. 16:23 But you have to talk to an expert. You don't need any data, the expert. The good thing is, they will give you a precise answer, 16:31 and they will give you the same answer every time. But they do not adapt. 16:38 You add a rule, things get a little tricky. So that's one thing. Econometrics, that's the next thing. 16:43 That's statistics. Many of you have probably learned statistics. You've probably forgotten much of it. 16:48 But we still use this all the time. If you have structured data, meaning data that you can put into a spreadsheet, usually 16:56 numeric data, these things work. And they work really well. They're pretty cheap to program and especially 17:04 the two-click things. How do we create a formula for those two dimensions 17:09 that will help us to figure out, is it a blue thing or a green thing? And they tend to be-- there are false positives. 17:16 There are false negatives. But they tend to be pretty good. The other is looking at the trend line, looking at the regression 17:22 These things actually work remarkably well with numeric data and also things 17:27 you can turn into numeric data, like is he going to buy my product based on what I do to him? 17:33 You can also do multiple dimensions. Finding that trend line with two dimensions is easy. 17:39 Try it with 100 dimensions. I have a team right now currently looking 17:44 at 100 million CVs, resumes, and tracking people's careers 17:50 through that 100 million resumes to try to figure out what is a more or less productive path. 17:56 You're not going to do that by just looking at two dimensions. And then, of course, you don't have to just look at lines. 18:03 But you do have to have an idea of what the functional form is that you're looking at. So it's often numeric. 18:08 There are some things, but you can do this. It's relatively cheap. It works really well. You get precise answers. 18:13 You get the same answer every time. But it has to be numeric data. Then you have deep learning. 18:19 And Eric talked a lot about deep learning. This was the coolest thing ever 15 years ago. And it's still pretty darn cool. 18:26 Now, the thing is, Eric also showed you there are 50 different versions of this. So I'm just going to give you one archetype. 18:33 This picture-- do you know what that picture is? It's a neural net. But what this picture mostly is? 18:40 It's a thing that people use to scare you. It's a complicated-looking thing. 18:45 And there's a little bit of magic involved in there. So Eric explained this, but let me explain it also a little bit more. 18:50 You're taking inputs. You're running them through a set of weighted averages and coming out with a prediction of multiple different states 18:58 at the end. And it sounds very easy, but it gets beyond human capability 19:06 really fast. You train it with labeled data. You need to know the truth. 19:11 That's why every time you log in, it says prove you are human. Tell us where there is a car. 19:17 Tell us where there's a bicycle. You are labeling data to train these things. Every time you get on social media and they do the de-aging 19:23 challenge-- show us you and show us you 20 years ago-- you're creating labeled data of old and young people. 19:31 You need that data. And then you do the-- the outputs are repeatable. But they are in no way explainable. 19:37 Our smartest people are just starting to figure out how to make this explainable. And it's going to be a little while. 19:42 But I'm going to give you an example now. At MIT-- I'm going to get a little complicated on you, 19:49 but I promise, not too complicated. The good thing is, I'm going to try to take this scary picture and make it a little bit more real. 19:55 Eric started. I'll try it. And after the two of us, hopefully, it will make sense. 20:01 What are those? Anybody? They're numbers. 20:07 Now, try to tell me, if we were to say features, loops 20:13 and lines and these kinds of things, tell me the formula to look at the number 2. 20:20 How would you create the rules to decide what is a number 2? 20:27 Well, it has a little loop at the top. And it has a loop down here. And then the number goes down. 20:32 That's how a two looks, except no loop there. 20:38 And there the line is going across instead of going down. If you were to try to program this with a bunch of features, 20:46 you would probably not get very far. But 35 lines of code, 25 to 35 lines of code 20:51 can do this no problem at all by doing this kind of thing. What do you do? 20:56 Well, first of all, you take these two-dimensional objects and you turn them into a one-dimensional set of numbers. 21:03 So 28 by 28 image becomes 784 pixels. Now, what you have just done is you've 21:09 thrown away all the relationships between what is on top of each other. You've thrown it away. And the algorithm doesn't care. 21:16 And then you put the numbers in the left hand side, and you have random numbers here. And it spits out 10 estimates, 0 through 9, 21:25 how likely is it that that is the number. And so you come through, and you get this. 21:31 Well, we think maybe it's a 2, a 1. We think maybe it's a 2. 21:41 Maybe it's a 7 and some stronger or less strong estimates. And then you go back and say, hey, it's really a 9. 21:48 And you go back the other way, and you turn all the knobs to make it a little bit better. 21:57 Now, if that bothers you, how much do you turn each knob, it should bother you because nobody knows 22:05 how much you turn each knob. But we do know, if you do this 10,000 times, the knobs end up in the right place. 22:12 So if it sounds like magic to you, it is magic. It's the magic of large numbers and good algorithms. 22:18 And so if you're wildly uncomfortable, then you are like the rest of us. We are all wildly uncomfortable with this. 22:24 But it does work. So you do that 10,000 times, and it gets there. 22:29 Now, what does that mean? Well, that means you've got to have labeled data. You need to know what that number really is. 22:35 You need to do the reinforcement over and over again. And you need to back chain to adjust those numbers. 22:42 If your data is biased-- let's say we have only data on men, but not women. 22:48 Well, then what happens when a woman comes in? Amazon found this out. They were reviewing resumes to try 22:54 to figure out what resumes look like their best engineers. And they were routinely rejecting women. 23:03 Why was that? None of their engineers were women. And women talk differently. 23:09 By the way, if you were the captain of the women's swimming team, you would never get a job. None of the men that they trained it on 23:16 were ever captain of the women's swimming team and also different use. So they had to go and fix that bias, same thing 23:23 if you train on men versus women, children versus adults, different parts of the world. 23:28 So you got to get rid of that. You got to test that bias. You also need to make sure you actually have some kind of accuracy in your data. 23:35 So that's it, no more on the neural nets. Do you understand that picture? I could test you, and you would know how to say that now? 23:45 Well, at least nobody can scare you anymore. So then generative AI continues. And as Eric said, what they do now, 23:52 not only they're using a lot of these similar algorithms, but they're also inventing. And they're saying, given what you told me and everything 23:59 I know, what's the next best word or pair of words to say? And then it says, once I have that, 24:06 given everything you told me and everything I know, plus everything I just invented, what's the next best word? 24:12 What's the next best word? And it can get things very right, as you've seen. It can also get them very wrong. 24:19 I first did my bio with ChatGPT. And the bio was really good, expert on digital transformation 24:25 and innovation strategy, author of four award winning books, blah, blah, blah. I loved it. 24:31 I've only written three books. I don't know what that fourth book is. 24:37 And it wouldn't tell me. So you want to be careful. But the interesting thing about that is-- well, 24:42 I'll come back to that in a minute. It randomly generates next. So why do you get a different answer every time? 24:48 Because it is random. It's meant to be that way. And you can use that. 24:53 It creates new things. It doesn't just classify. It creates new. It can be used for good as bad. 25:00 You can get some boring stuff. You can also get creative stuff. But you also get this, the hallucinations. 25:06 My favorite is this. This lawyer prepared all of his court documents using ChatGPT. 25:12 And unfortunately, some of the cases that he cited as precedent for what he's doing were not real. 25:20 That made the judge very angry. So not only did he get in trouble with the judge, 25:25 but he ended up in all the business press as being the laughingstock. Don't do that. But here's the thing. 25:31 We are so worried about these hallucinations. But how many of us know a perfect employee? 25:40 People make mistakes too. And so the idea is not to expect a perfect answer, maybe, except if your car is driving. 25:47 But the answer is to get these things to work well and put the right controls in, in case they are wrong, 25:54 the same thing we do with people. So why do we expect the computers to be better than people in every case? 26:00 Why don't we just put the right processes in place to account for the fact that these are sometimes not right? 26:05 The other things, of course, huge training data this thing 26:11 needs and huge amounts of energy. And so these are other topics we can talk about over time. 26:18 So the problem is, you got to start with the problem 26:23 and figure out the right technique. So one way to think about it is there are questions, you can ask. How accurate do I have to be? 26:29 And what's the cost of being wrong? Getting in a traffic accident or making a wrong medical decision, 26:36 there's a big cost. Sending the wrong marketing message to somebody, no cost at all. 26:43 Do you need the answer to be explainable? Because if it does, then those first two are probably useful. 26:49 Those last two, they're not explainable. How do you do that? Do you need the answers to be the same every time? 26:57 You're not going to get that with generative AI. And confidentiality-- and confidentiality is less of a problem right now. 27:03 You can get these generative things that are going to keep you, or at least the vendors promise they will keep you safe. 27:09 But these are things to think about, right? Then on the other side is the data. Do you have a source of truth, and how true is that data? 27:18 And number two, is it generalizable? Or are you going to reject all women that 27:23 apply for your company? So as you're thinking about how to do it, these are questions you can ask about this problem I'm 27:31 trying to solve right now. And the questions, either you can make the decision 27:37 or you can ask these questions of your technical people. And first of all, they will now be a little bit afraid of you. 27:42 But second of all, you will have a better conversation because you are asking these questions. So first bit, what is AI? 27:52 Four things. Here's how to think about them. Here, if you like traffic lights, here's a nice little chart you can use 27:58 to think about these things. Where are they good and bad? But that's only part of the problem. 28:06 That's the part saying, I have a specific problem I want to solve. How do I think about what the right technique is? 28:12 The real problem is how do I make this work in my organization? 28:19 So I have more questions for you, maybe some more answers. Are you ready? 28:28 How do I make it work in the organization? Remember, come back to this technology is not the problem. 28:34 Transformation is. How do we put the processes, the policies, 28:39 and the capabilities in place to do this the right way? Because doing it once is hard. 28:45 But you need to do it over and over, and you need to make it mix in with the rest of the stuff that you do. 28:50 Here are three challenges to think about when you try to make this stuff work in your organization. 28:57 And I'd like to say, this is only GenAI, but this is basically anything. How do we work this in? 29:03 But with generative AI, how do we prioritize what to do and what not to do? What do we do first? 29:10 What do we do second? And what do we never do? Number two, risk management. 29:16 What if we are wrong? What if we have a privacy problem? These kinds of things. And last is the capabilities, so four sets of questions 29:23 you can ask for that. And then after this, I'll have one more little section for you. 29:30 How do you ensure the safety and the value of what you're doing? And one of the big other ones is learning. 29:37 How do you not only learn about this innovation, but how do you learn across innovations so you can choose the best way to do things, 29:44 and you don't end up doing the same thing 10 times? 29:50 One of the big questions to think about here is, how does your governance process look? 29:58 And what we've seen in generative AI is multiple approaches, but they tend to fall into top down, 30:05 this is a very risky, very expensive thing. Let's control everything. 30:10 And the good thing about that is, we're not going to make mistakes. We're not going to throw money away. 30:15 We're probably not going to kill a lot of people. But we're also not going to innovate very well 30:21 because the central part of large organizations, it doesn't always know the best opportunities. 30:27 The other side is the decentralized way. Hey, go out and innovate. 30:32 Do whatever you want. Now, here are some rules. Don't steal money. 30:37 Don't be bad to people. But here are some rules. Innovate within these rules, and tell us about what you're doing. 30:45 It's a tremendously good way to discover really interesting ideas at the edges. But it's a great way to waste money also. 30:52 And also, depending on how careful they're being, you can also break some laws when you do that because where the central might try to be very 30:59 careful, the edges might not. And so what we saw in generative AI is the very top down, safe, but slow, or the very decentralized, 31:10 very fast, but more risky and more costly. We saw both of those coming out. So at Societe Generale, one of the big French banks, 31:19 they did it in a very centralized way. But they asked everybody to say, if you want to do this, 31:26 what would you do? And they got 700 use cases. And what they decided is they did a few of those right away. 31:34 But the rest they said, wait, because all these cases, an intelligent agent is important. 31:40 A chatbot to talk to people is important. Programming is important. Let's do this the right way, and we will build everything on top. 31:47 So they did a few things fast. They told other things to wait until they 31:53 had these other capabilities set up. They tried to bridge the centralized and decentralized. 31:58 Sysco that I told you about, they did a different way. They said you know what? This is just another technology. 32:06 We have a way we think about this. And the first thing we think about is, can we buy this instead of building it? 32:14 And if we can, we will do that. The second thing they say is, if we think it needs to be programmed, can we do it cheaper and easier 32:21 with expert systems and statistics? Or do we do the fancy, complicated stuff? 32:27 And only when it answers all those questions do they do the leading edge, the generative AI. 32:32 So you see these governance rules, they fit in there. You got to figure out what works with your company, 32:38 and how does it fit the risk profile that you're thinking about? 32:43 Some other questions you want to ask, though, are also just as important. Is our culture ready for this? 32:51 What happens when the computer is just as smart as our expert person? 32:57 15 years ago, this started happening in banks, where the computers were as good at judging a high quality or a low quality loan 33:05 than people were-- or insurance. The computers were just as good at underwriting some things 33:11 as people. The people felt very threatened by that. And there was a whole process. 33:17 It's happening in all kinds of jobs. Now, if you've seen the movie Moneyball, have you seen-- 33:22 I don't know whether-- In America, baseball is a big deal. I don't know if it is here. But there's a big question. 33:28 How do I tell who is a good potentially new employee for this company? 33:34 Maybe the computer is better, and people don't like that. So do we have the humility to work with these? 33:40 Or do we fight against them? Do we have ethics to do the right thing and not 33:46 the wrong thing? And also in the culture, how good are we 33:51 at experimenting and trying things and failing fast, rather than trying to have the first answer, 33:57 the right answer, before we start? The culture's got to be ready for these things. 34:04 And the way you do that, you have to drive that. You don't just say plug in the model. You got to help people get comfortable, 34:09 with innovation units, other things. And another one also with people being comfortable, 34:15 do we have the skills? But what does this mean for careers? 34:22 A friend of mine, Daniel Rock, looked at GPTs, and he has calculated through a very rigorous process 34:28 that 46% of all jobs are likely to have 50% of their tasks replaced by AI over time. 34:38 Now, some jobs, it'll be 100%. These are just-- 46% of jobs will have half of their tasks 34:46 go. What if you do that job? How do you feel about that? And am I going to-- am I going to go along with this? 34:53 Or am I going to reject this because I think it's going to hurt my job? So what do you what do you do about this? 34:59 Certainly, I've done some writing on this. And certainly after lunch, you're going to see some really interesting stuff on that. 35:06 But AI doesn't have to replace you. How can it make your job easier, reduce your cognitive load? 35:12 How can it do-- one of the things I do for all the time, I'm a pretty good writer. But I don't do those sexy, creative titles. 35:19 So I let AI do it now. Give me five potential titles for this. 35:25 Or I can give it my text and say, make this more exciting. And the other thing we did not expect 35:32 is the ability to learn with AI. AI is a tremendous teaching tool. 35:37 And I'm finding I'm learning all the time, just by having it help me think, having it ask questions about what I'm doing. 35:44 And certainly, I talked about this tutor we are doing. And many places are doing these tutors, including at Cresta 35:50 and at one of the startups we talked about. So it does not have to replace us. 35:55 Are you having that conversation with your people about how this-- you're not just saying, we're not going to hurt you because nobody believes that. 36:02 But can you have the conversation about how we can help you introduce this? 36:08 So, for example, Dentsu Creative, an advertising firm, that is a firm that you might think should be a little bit nervous about these things. 36:15 And what they're finding is that these creative people love this. 36:23 But they're being very systematic in the company about how they introduced this. First of all, they are saying all the boring stuff. 36:29 Your planning, your proposals, those kinds of things, it will do that for you, take a few words 36:36 and turn it into a proposal. That's great. What they didn't expect and that they love now is, let's draw a picture. 36:43 And instead of saying, go away, and in a week, we will have a design for you, now they 36:48 say, go get a cup of coffee, and in five minutes, we'll have a design for you. And now, we can iterate that and iterate it. 36:54 You can do it with the client. It's better for both. They didn't expect that. The other thing they're doing, though, is they're not saying, 37:01 you use it. You use it. What they're saying is, everybody use it. We will help you. And let's get together and share your ideas. 37:08 What is working for you? What's not working for you? These office hours where people share their tricks. So people are investing as a group in getting better. 37:15 And you see what happens. It frees up the time that creatives need to be creative. 37:22 All the stuff they hate, they don't have to do. They can't see themselves going back to the old way. 37:28 So what they did, they were very careful about helping people understand how this can be good for them, not just bad for them. 37:34 They involved them in that process. And that's how they avoided the rejection, and they get better with this. 37:41 In coding, in your marketing people, in many places, with customer service, you will have this thing going on, 37:48 a lot of fear. We've done a thing called the Global Opportunity Forum, where 37:54 we are getting companies, talking to companies about career questions. How are you developing your people? 38:01 What are the skills that you need? What's the best way that you can do to help your people grow with you, instead 38:06 of being afraid as you grow? And so if you're interested, contact me about how we are getting companies together 38:12 to think through this challenge. So that's that. You ready for the last part of this talk? 38:20 We talked first of all about types of AI, four types. And if you have a particular problem, 38:26 how do you decide on the right type? And I gave you questions you can ask. Then we talked about having your company have the capability 38:35 to use this well to transform in a better way and questions you can ask there about culture and helping 38:41 your people be ready. So we just completed a study-- it's coming out tomorrow 38:47 in Sloan Management Review-- about how our company is looking at transformation with generative AI. 38:53 And we were looking for these giant transformations. Let's change the entire underwriting process in a firm. 39:00 Or let's turn over all of our sales and customer service to a computer. 39:06 We didn't find those. But we did find a lot of smaller transformations happening 39:13 that people are starting to get a lot of value from this and setting themselves up for the bigger transformation 39:19 over time. So what we do is, if you think about the big transformations as being transformation with a big capital T, 39:27 what we found is a company are doing a lot of transformation with a little t, smaller transformations. 39:33 But they're doing it in a systematic way that is helping them get ready. And it looks like this. 39:41 Level 1 is, most companies already are thinking about individual productivity. 39:48 Here is a version of the LLM that is safe to use. 39:53 Or here's our own private version. McKinsey, for example, now has an LLM 39:58 that looks across all of their slide decks. And it's not perfect. But if you need to know about energy generation in Southeast 40:06 Asia, McKinsey probably did this once before. And it can help to answer some questions for you. 40:12 So not only the public things, but the private things they're starting to do from an individual productivity 40:17 standpoint, very low risk. They're just informing people. It's a good way to get started. 40:24 Number two is the specialized roles and tasks. Let's start to transform that. So we're seeing it in call centers a lot. 40:30 We're seeing it in coding. We're seeing it in some other things. Typically, a human still stays in the loop. 40:36 But sometimes for low risk things, they're letting the computer take over, like at Lemonade. 40:42 Lemonade is 50% of things get processed automatically, 50%. The more risky things they take on. 40:49 And so that's starting to happen. And then what we're not seeing as much of, except in technology companies, is actually 40:56 doing the direct customer impact. So how do we think about this? In the first level, individual productivity, 41:03 summarize these documents. Many banks I know that the minute a company releases its financials, a minute later, 41:12 five minutes later, all the spreadsheets are updated. You had to have a lot of interns that did that before. 41:19 Now, it just happens. These kinds of things can happen. Sum-- well, sorry. That's the next piece. But summarizing documents, hey, what 41:25 just happened at my meeting? And in the company specific LLMs, the opportunities in the tasks are like that one. 41:32 Can we make sense of the latest quarterly report for financials? Can we think about customer support? 41:38 That's human-in-the-loop. They tend to be lower risk. And you keep the human in the loop 41:44 to help mediate that risk even more. And last but not least, we're seeing this farther thing, 41:50 the direct engagement. And we're seeing this in online right now. The people that make Coach and Kate Spade things, 41:58 things that only this big, but they cost a lot of money, they're personalizing a very conversational approach 42:06 to help you. Just as if you went in a store and that person would make you buy that handbag that you charge 42:11 too much money for, now online, they can start to do the same thing. Relatively low risk, those kinds of things 42:18 are starting to happen and also the idea of doing the first or second tier of customer service. 42:24 And these are happening. The product companies are already doing it because they have to. 42:29 What we're seeing less of is this bottom thing, transformation of large processes. 42:36 What will happen, from what we're hearing, is that you're not going to get GenAI replacing the whole. 42:41 But combinations will do it. GenAI will have a part. So many companies, we already have the forms that you give. 42:49 Generative AI will translate that into data that the rest of the processes can use. 42:55 And then it will translate it back into conversational later. You'll see these combinations happening more and more. 43:02 So you want to think about the risks and the capabilities. What's happening here, we call this the risk slope, 43:08 how as you grow, you grow your risk management capability, but you also grow your capability 43:14 to do more and more bigger things over time. Here's the reason why this is so important. 43:23 The proofs of concept, it's easy to do them. But making it work in a large group of users or customers, 43:29 getting from the lab to reality is really hard. And in fact, in a large bank, she said, 43:35 "The more stuff you do, the more stuff you find to do." She did not mean more features we can build. She meant more errors we have to solve. 43:41 And one way to think about it is H&M, the fashion company, 43:48 "It's almost like putting a tire on a car." How many of you have changed a tire on your car? 43:54 Not many people. Well, if you've ever changed the tire, you know that you do not put a bolt on really hard and another one, because you will bend the tire. 44:01 You'll bend the rim. Put a little bit, a little bit, a little bit, a little bit. That's how you want to think about your AI capability 44:08 in your companies. Do it here. Learn from there for the next thing, the next thing 44:13 So what's the conclusion? I just gave you a lot of stuff. 44:19 But that's OK. This is MIT. You're smart people AI can seem to be intelligent, but be 44:28 intelligent in how you use it. But remember, just because it's not perfect, that doesn't mean it's bad. 44:36 Your people also make errors. Let's put the right processes, right controls, to deal with the fact that people are sometimes wrong. 44:44 Number two, start with the problem, not the technology. And many times, the answer will be combinations, not just 44:50 one of these. You look at the task that needs to be done. Number three, get started now because if you're 44:57 going to put that tire on your car a little bit, you work up the growth slope, the risk slope. 45:02 And every time you do something, you learn more to be better at it. 45:08 You got to help your people be ready, because if they're not ready, they will fight you. 45:13 And they will fight you either very actively, or more likely, they'll just say, oh, that's hard. I don't know what to do. 45:19 Those of you who have teenagers, you know they do both. And last but not least, continuously improve. 45:27 What can you do with small t transformations that will help the large T transformations later on? 45:33 So I have exactly ten seconds left. So I'm going to ask for questions, 45:39 even though I'm not allowed to. So let's take a few questions. But I won't go too far over. Can we do some questions? 45:46 So by the way, lots of good reading if you want it, including the new article, so thank you very much. 45:53 What should we do with questions? They're up here. 46:00 I don't get to choose the question. This is scary. Our agency is pushing everyone to use AI, even the blue collar 46:07 workers. Is this the right move? Is it necessary for everybody? The interesting question thing that you said in there 46:12 is, they're pushing everyone. How can you encourage everyone? 46:17 Do you see the difference? If I say, you have to do this, half the people are going to say, you can't tell me what to do. 46:25 If you say, hey, this is a fun set of tools, let's try to help you think through it, they might be a little bit more interested. 46:31 So what I suggest is, encourage everybody to do it, and you might learn something. 46:36 And if you encourage everybody to do it, they might actually suggest some innovations and fight this a little bit more. 46:43 What's another question? There we go. Many of our works revolve around achieving efficiency and optimization. 46:52 When AI can do that better, what would be the focus of human's work? So certainly, we had the Work of the Future Initiative. 46:59 Ben's going to talk about this more also after lunch, some really fascinating things. But I'm convinced there will be plenty for people to do. 47:08 And in an ideal world, you take all the boring stuff, let the computers do that, and you 47:14 let the people do more interesting stuff. Now, that's an ideal world. But can we get closer? 47:20 If you think of the way automation has worked all over time, it has always taken the routine work. 47:26 But there's still been a role for humans. And when you look at the way advanced manufacturing works, you still have people in there. 47:32 They're just not doing the really boring stuff anymore. So I think there will be a role for humans. 47:37 And the tricky part is trying to figure out how to introduce the right things in the right way 47:42 and to engage your people in the process. The Global Opportunity Forum we're talking about 47:50 is another opportunity there. Next up, how do you balance creativity with and without AI? 47:57 What are the advantages, disadvantages of flexing ideas? 48:02 I don't see any disadvantage. Actually, I do see a disadvantage. I do it all the time. 48:07 What are some creative ways to think about this? You can always ignore it. But what would I do without the computer? 48:13 I would go to a couple of friends. What's a creative way to think about this? So I don't see a problem with helping 48:20 the computer be more helping. The only challenge is, sometimes you start down a route, 48:26 and you get stuck in that route. So just as if you're talking to people, when you're talking with the computer, 48:33 be ready to back out of that route and try a different thing. You can get uncreative because you get stuck in a certain path, 48:42 but absolutely, flex the ideas all you can. You don't have to accept those ideas. How can we prepare young people for a job 48:49 15 years from now that might not exist due to AI anymore? Hopefully, if you are training-- oh, sorry. 48:54 Let me say that differently. Certain people you train to do a job. We have vocational training all over the place. 49:01 You train them for that job because nobody should expect that you start a job now, and it will 49:06 be the same job in 15 years. Now, there are some-- my mom was a school teacher. 49:13 15 years later, she was still a school teacher. But she was teaching differently. My uncle was a fireman. 49:19 15 years later, he was still a fireman, but there were new tools he had to learn to use. 49:25 So you train people for the job, if you're training them for a job. But they should also get the ability to learn. 49:31 They should gain the comfort, the growth mindset to get things. And you should always train that. 49:36 As far as training people for what will the good things be, creativity, working with people, critical thinking. 49:47 These don't seem to be going out of style. This generative AI stuff-- those don't seem to be going out of style for a while. 49:53 Actually, writing those emails, maybe. But knowing how to tune those emails to be the right kind of interaction with people, AI will help, 50:00 but those things will happen. So what I call these human skills-- and we have a framework we call the MIT-- 50:07 when I was in Open Learning, the human skills framework. In that framework, we think about how 50:12 I think, how I work with others, how I manage myself, how I lead others. 50:19 Those skills are probably not going to go away. And so how can you get those in college? How can you get those in high school? 50:25 Teamwork, writing and communicating, leading things, even when you're 12 years old. 50:33 I think that's the end of my time. So I want to thank you. And I'll be around at lunch if you want to talk. Bye. 50:38 [APPLAUSE] [MUSIC PLAYING]