ChatGPT: How To Use It In A Business School Classroom

If there is a single topic that has captivated business school academics in this past year, it is Generative AI. The popular use of Chat-GPT has caused business school faculty all over the world to consider not only how it might change the world of work and play, but also how to teach MBA and other business students how to use it.

While best practice examples are still emerging, there is one clear consensus: It cannot be banned nor ignored. It is a technology that needs to be embraced given the already widespread impact AI is having on every aspect of the business world.

McKinsey & Co. has predicted that banking, high tech, and life sciences are among the industries that could see the biggest impact as a percentage of their revenues from generative AI. Across the banking industry, the technology could deliver value equal to an additional $200 billion to $340 billion annually if the use cases were fully implemented, according to the McKinsey study. In retail and consumer packaged goods, the potential impact is also significant at $400 billion to $660 billion a year.

In other words, AI will become so integral to business that it must be a critical part of nearly every course in the curriculum. At Harvard Business School, Professor Mitchell Weiss, chair of HBS’s first-year required curriculum, has done a webinar on how new generative AI tools invite more of what case-method teachers do best: ask probing questions. Weiss argues that ChatGPT and other AI tools offer teachers and students ways to accelerate their preparation for case-based sessions and deepen the person-to-person discussion that’s the hallmark of the case method.

We recently sat down with three business school academics at the University of Cambridge to explore how they are thinking about the new technology and incorporating it into their classes at the Judge Business School. It was an excellent place to get answers because for 800 years Cambridge has used the essay as its core assessment tool for students.

Jaideep Prabhu, professor of marketing at Judge Business School:

I think it’s a plus for educators and it’s complementary to what we are already using. For business students, it is an amazing tool. You can’t rely on it but it can be a huge aid to a business you are running. We have to get our students to use it with caution and to sharpen their judgment about it. It is relatively easy to check if it is feeding you BS but more difficult to go behind generic concepts and answers. That’s why the prompts and interrogation you use for ChatGPT are just as important as what it tells you. And if it tells you something you are unhappy with, you can ask in a different way and see how that impacts the answer.

I see it as a research and learning tool. I have studied previous breakthrough innovations and how that might change behavior. And whenever you have a breakthrough technology, people are going to have three types of expectations on what it can do to current practice. One set is that it will completely substitute for something we currently do and replace it. Another view is that it is complementary. And the third view is that it doesn’t change anything. It has no impact whatsoever.

What we have found in a study on Internet banking is that it was complementary, at least in the short term. In the long-term, it is probably a substitute. With ChatGPT, it is currently complementary. We teach not just MBAs but students in an MPhil in Technology Policy. These are students who want to be policymakers in government. The whole one-year course explores what technology can do and how should governments regulate it. So the students are very aware of AI and ChatGPT, and they are already thinking about what governments should be doing in smart regulation. In one of the courses I teach in that program, they are assessed entirely by exam.. This is a very 19th Century way of assessing but there are some merits to it. They take it seriously because it is a closed-book exam where they have to hand write their answers in essay form.

For that, I send them a mock test along with mock answers so they can prepare for the exam. I hate doing this so this year I said, ‘Why don’t I use ChatGPT to write the mock answers?’ ChatGPT gave a brilliant answer to the first question about frugal innovation. We asked it to define frugal innovation and provide the pros and cons to it. I just had to tweak it a bit. The second question was, ‘Give me an example of a frugal innovation in government that has made a transformative difference.’ And the ChatGPT answer was very generic so I wrote my own answer. It didn’t pass the BS test. I shared the questions and answers with my students and then asked them to use ChatGPT themselves to compare the answers.

In the other course on technology policy, they are assessed entirely on an essay that they have one month to write on a forthcoming technology of their choice in any country. Many of them chose to focus on what AI is going to do to education. They could have used ChatGPT but it would have been very hard because we want them to do primary research and ultimately take a stand. But they could have used ChatGPA to get a structure for the essay and that is what ChatGPT does very well. I didn’t see any essay that was so generic that it was done by AI.

On the MBA, I teach an elective on emerging markets which is assessed entirely through a group presentation. So I don’t think there was much impact there. Another course I teach is called the Marketing Concentration. That is assessed by a live case that they do for the Financial Times. They are given a marketing challenge and they have to do primary research for a board-level presentation. Again, they could use ChatGPT for the structure but a lot of it has to be based on primary research that demonstrates how much they know about the topic.

An interesting issue with ChatGPT are the implications for academics like myself. Even when students are asked to write an essay, my advice to them is try out ChatGPT and see what it gives you in terms of structure. I have used that myself. I have seriously considered writing a handbook on frugal innovation for healthcare and I tried using Chat-GPT for that because it gives you a great structure and lots of examples that I can cross-check. I have tried writing a course on sustainable marketing. Ten of us were sitting around the table trying to develop the course and I tried ChatGPT and it gave me a pretty good equivalent of what we would come up with as a committee of experts.

I have a colleague who was in global media and took early retirement and set up his own company. He runs his company with himself and ChatGPT which is his colleague. When he wanted to create an ad campaign for his new consulting company, he used ChatGPT to do it. So I want to get him to come into class and do a session for marketing managers on how they can use ChatGPT to develop marketing campaigns. From what he has said to me, you can go from having a team of four people, taking a week to do an ad campaign, to have one person do it with you in a few hours.

I want our students to know that there is this capability and that they can use it but to use their judgment as well because sometimes it totally creates BS. One time I had this idea for a series of podcasts on the Indian Institutes of Technology in India. Then, I thought maybe there are podcasts on this topic already and so I asked ChatGPT. It came up with a list of podcasts but none of them existed. There was no trace of any of them on the Internet. It was a total hallucination. So if you are not careful in using ChatGPT it can be embarrassing or cost you your career.

 

Thomas Roulet, professor in organization theory and deputy director of the MBA program at Judge Business School:

I teach leadership in the MBA program and it’s all about trusting yourself and your ability to manage complex situations. It would be a mistake to ban AI because it is with us now. People will have access to it in organizations all over the world, and it will be integrated with their approach to work. So we have to teach our students how to use AI as a resource for knowledge and decision-making.

It is clear that students are already using ChatGPT. If you ask it what is the definition of psychological safety, ChatGPT will give you a better definition than Wikipedia. In the workplace, students can use ChatGPT to help them decide how to deal with a certain situation. But in the end, what will matter is their own judgment and the cues they get from their interaction with others.

We need to make them use it and make them use it in a critical way where they can derive nuggets from ChatGPT, assess them and use them in the workplace with a human and social touch which makes the difference between a good and a bad manager. If you were to ask ChatGPT to manage people, it would be quite bad at it unless it becomes very good at reading emotions and making instinctive decisions.

We are thinking about asking students to actively use ChatGPT. We used to ask them to tell us about a situation you have faced in the workplace that is related to the leadership issues we have discussed and use the concepts learned in class to inform how you would approach it. Now we will ask them to use ChatGPT to give them some answers on what you could do and then assess the answers with the knowledge and the critical skills they have developed during the course. This way, they don’t only apply the knowledge to their own situation; they apply the knowledge to either criticize or complement what ChatGPT tells them to do. They need to assess and criticize the information they are getting from AI because a lot of what they will get from ChatGPT is bull crap.

Stella Pachidi, assistant professor in information systems at Judge Business School:

We want people to think critically. We want them to develop the skills to become able to assess the situations they will face in their organizations and that is not at all impacted by ChatGPT.

Still, we used it in the classroom this year for the first time. Students were asked to engage in an exercise planned to explore how organizations can leverage AI so we were already discussing a healthcare case and I gave them an exercise on around sensor technology and I asked the students to use ChatGPT to identify the big players in sensor technology. The first company ChatGPT named doesn’t exist anymore because it had to close down. So they immediately started figuring out the limitations.

What is really important is making people aware of how you have to play around with the prompt to make ChatGPT useful. You can tell if someone has used it to write an essay or a response to a question. It reads like Chat-gpt. You figure it out. You cannot obviously prove it so you cannot punish the person but it reads as such. Because it reads bland, those essays had already been marked very low because they don’t show that the person has a deep understanding of the issues we discussed in class. 

In one course I teach, the biggest part of the students’ mark comes from a group exercise where they have to come up with a big idea. They have to do a final presentation to pitch their idea but they have to justify all the choices they made throughout. It is a fairly practical exercise.

Can they use ChatGPT for brainstorming? Yes and they are encouraged to do so. But that doesn’t detract from their ability to think critically. So I see this as not much of a threat but more as an additional tool, a more advanced Google search, and we are already trying to think about how to integrate it. For example, we want students to write a reflection and if they have used it, give us the search file with all the prompts put in an appendix. And write a reflection of how it was useful to you and how you cross-checked the information to make sure it was accurate. You have to embrace it. You cannot ignore it.