Michael Olenick: Better Prompts Or Better Judgment? Columbia Bets On The Latter

Latest in an ongoing series exploring how business schools are integrating AI into teaching and research, one school and one faculty member at a time. See also At Duke Fuqua, Using AI To Strengthen The Bonds Between Humans, At Indiana Kelley, “AI Is,” Not “AI Will,” Training Students To Leverage Increased Cognition (Michigan Ross), Rebuilding The Role Play (Kellogg), Learning Requires Friction (Wharton), The Deck Is Dead (Stanford), and We Teach MBAs To Apologize About The Best Tool They Have (INSEAD).

One tool keeps coming up as I survey how AI is being used in business school teaching: CAiSY, from Columbia University.

CAiSY began as a research project. Professor Dan Wang of Columbia Business School was working on something else, saw the potential, and built the first version of CAiSY. He eventually teamed with former student Jill Cohen, who focuses on software.

AI-based teaching materials, like GenAI itself, are experimental and evolving. There aren’t many and most keep the DNA of a research project.

There’s the Avris teaching system, and more recently we’ve talked with Harvard about its Foundry bootcamp, with Kellogg about its simulations, and with more schools than we can count. (If you’re building an AI teaching tool, I’d like to hear.) Some are chatbots. Some are simulations. Some are interactive cases, VR optional. My own VSTRAT.ai belongs on that list too. It focuses on the formulation and analysis of interactive strategy frameworks.

What they all share is they’re still experiments. We’re all working the same frontier.

I’d be remiss not to mention the most common system of all: frontier models. Every school now has ChatGPT, Claude, Gemini, or similar. Are these horizontal platforms the ultimate end state? Or are they, like SQL databases back in the day, the foundation that specialized vertical systems get built on top of? It’s too early to say. What we do know from early feedback is that students and executive participants are underwhelmed when handed a $20-a-month chatbot and a lesson on “how to prompt.”

Here’s Wang, on how a scramble in November 2022 became the tool half the schools I talk to bring up unprompted.

---

P&Q: What’s the background of CAiSY? I’ve heard about it from many different schools and it seems to be going well. How did you dream it up?

Dan Wang: I’m a professor at Columbia Business School, where this is my 14th year. I teach strategy and a class on technology strategy. The birth of CAiSY was November 2022, when ChatGPT became publicly available. I was about to teach in a month and thought, this is not good, because pretty much all my assignments are written. Rather than doing nothing, I encouraged all my students to use ChatGPT and made them systematically report what they did with it.

What I learned was that students were thoughtful. Rather than using it to shortcut their learning, they used it as a thinking partner. So the very next semester I worked with a former MBA student to build a very rudimentary, untrained chatbot. Instead of a passive writing assignment, students would talk through the next session’s case with the bot. One of the most interesting things I could document was that students changed their minds during the conversation. That got me thinking about different layers of critical thinking.

In 2024 I had a sabbatical and a small grant from Columbia to explore how technology could be integrated into the classroom. That’s when I reconnected with Jill.

Jill Cohen: I graduated from Columbia in 2020 and had Dan as a professor. My background is software engineering at startups. I worked at Snapchat before business school, then on a lot of zero-to-one projects in mobile apps. When we reconnected, Dan said he was noodling around with AI during his sabbatical and I said, count me in. I had no idea CAiSY would get to this point, but I knew it would be a lot of fun and discovery.

Wang: The magical moment was October 2024, when OpenAI made its real-time API available. That’s the engine behind CAiSY, a voice-based foundation model. The release wasn’t publicized at all, so finding it was almost accidental. We built a quick and dirty prototype, swapping the text chatbot for a voice chatbot, and immediately we knew this was the direction.

I deployed it in my class in spring 2025. They used their voice to chat with CAiSY about the cases we’d be discussing. I anticipated some positive feedback from the novelty. Instead I was completely overwhelmed, and I think Jill was too, by the sheer detail of what students told us. We were learning things about their experience I couldn’t have anticipated. A lot of those things have since become rigorous research studies. There are a couple of working papers now, with a couple more coming.

CAiSY’s purpose is not to shortcut pedagogy or make it more scalable. It’s the only AI tool I know of that is an intervention outside the classroom that enhances interaction inside the classroom. That started as anecdote from instructors. Now we have field experiments to back it up.

P&Q: Somebody mentioned this returns to some principles of older education, the Oxford and Cambridge tradition.

Cohen: Pre-COVID, pre-AI, the way I engaged with cases as a student was: read it, answer a poll question before class, strategy A or strategy B, give a few sentences why. You’re trying to get things done as fast as possible and fire it off. Then I went through the debate experience where it actually talked back to me. To get real, personalized pushback on exactly what I said was a light bulb: oh, I hadn’t considered it that way. CAiSY pushes back, pokes holes, brings up viewpoints I hadn’t considered, so students walk into class thinking far more comprehensively about the case. What was one-way has become two-way.

That’s the connection to the Oxford or Cambridge tutorial system. Each student gets a one-on-one tutor who gives that same pushback. It’s not offered at scale at most universities because it’s way too cost prohibitive.

Wang: The model is elegant. I didn’t go to Oxford or Cambridge, but my friends who did say it’s the best part of the experience. What’s fascinating about CAiSY is that you get that experience, and then it pairs with the in-class experience and motivates the student. Talking with CAiSY offers a layer of psychological safety, and the pushback makes you inch up a little in your seat. That happens to me every time I use it. I’m like, “Oh, that was good, I’m into it now.” You remember the conversation more and understand the topic more deeply because you’re forced to articulate your thoughts clearly. That motivation spills over into the whole class.

Cohen: Anecdotally, professors across many universities say class participation has declined steeply post-COVID. Cultural, generational, people are less comfortable speaking in front of peers, maybe afraid to be cringe. Going through it once alone, you and your computer, students report significantly increased confidence. Professors report class discussions are much better and more engaging.

P&Q: My graduate degree is a law degree. A young man I know just started law school and said, every professor is Socratic, they’re getting students used to opposing counsel and judges, in front of everybody. It sounds like CAiSY is doing the same thing in a business school context.

Wang: There’s overlap for sure.

P&Q: Most of the systems I’m seeing seem to be research projects that flew off the shelf. Am I wrong? What’s the origin?

Wang: You’re not wrong. My initial intent was a short-term collaboration with Jill that would result in a paper, and I’d have been happy with that. The shift came during the first pilot in my class, when we started hearing from other institutions and from professors I didn’t know, third-degree connections. We realized there was a lot more to be learned if we thought about distribution. Distribution produces a feedback loop into innovation, and innovation produces a feedback loop into knowledge.

We don’t talk about this much, but we think of CAiSY as an organizational innovation. It sits inside Columbia Business School but operates with the agility of a venture, and it has to. When you’re talking about AI, you can’t be a research project. The constraints and goals of a research project are far too restrictive. If the goal of a research institution is to learn, the form ought to fit the function. The result is that we’ve been able to deploy field experiments, observational studies, and interview studies at scale because we built for such a diverse set of use cases. I anticipated having the data for my original paper a year and a half ago. By focusing on scaling and distribution instead, we ended up with far more data and many more studies than we could have done otherwise.

P&Q: Have you spoken with the law school or medical school at Columbia?

Wang: There are a couple of pilots at law schools, and Jill is working on a pilot with the med school. Those schools deal with similar organizational, managerial, group-based topics, so the practice is germane.

P&Q: A lot of business schools are just using raw LLMs. Turn on Claude or ChatGPT and go. I watched a marketing professor try to build a customer journey map in ChatGPT and he typed and typed and typed. No student could ever do that. What I’m seeing is that structure is far more effective than raw LLMs that go on and on.

Cohen: You hit the nail on the head. Universities, and I hear this from friends in corporate America too, feel they need to have an AI strategy, and then it just means giving everyone a Claude or ChatGPT subscription. That’s a start. But most students were already exposed. There’s an interplay between not being too top-down and encouraging experimentation. Research projects are experimentation by nature: poke around, hit a dead end, pivot. That’s product development. So it makes sense that the ed tech tools that are working came out of universities but are very structured and opinionated about a method.

Wang: Having an institution impose an AI strategy on its members asks a lot of its key members, the faculty, who have spent decades honing a pedagogical point of view. Asking them to do something different requires training and space. Handing them a general-purpose tool creates a lot of chaos, and I’ll count Columbia in that. There’s not much direction in a general-purpose tool.

As much as we know where CAiSY creates value, we also know where it doesn’t. That’s really important. If you claim a tool is there for everything, it’s probably not useful for anything. We have a clear point of view on where this fits into a learning journey, and that’s earned. You can only know it through iteration.

P&Q: So what is CAiSY best for, and what isn’t it best for?

Wang: CAiSY is really good at exercising students’ practice for decisions that involve judgment. Where it brings the most value is activating critical thinking: the ability to question your own assumptions, take different perspectives, look at a statement or opinion and trace where it came from and see its ambiguities. That’s the real world. So CAiSY exercises students’ tolerance of ambiguity.

Where it hasn’t been effective is as a didactic teaching bot. You don’t need an LLM for that. There are very good ways to communicate objective facts, laws, formulas, and have students absorb them. That should be the work of a tutor, or plain old reading and writing and problem sets. Not the best use case for CAiSY.

P&Q: Stanford told me the deck is dead: build a prototype, not a deck, refine it, and know when it’s time to call the computer science department to build a real piece of software.

Wang: It’s true. Last year I taught students with no written assignments whatsoever. No decks, nothing. The final project was a prototype. Most of these are MBA students with zero technical expertise who can barely spell GitHub. The goal wasn’t to make them software engineers. It was to make them software designers, product managers, and to give them a different means of expressing their ideas. Every single one of them got there.

---

Michael Olenick, JD, is a former INSEAD research fellow and founder of VSTRAT.ai. He has spent four decades building AI and expert systems; his research has been used at INSEAD and, before then, cited by Congress and media on how to resolve the 2008 era financial crisis. Michael currently lives with his partner, Anastassia, in Austin, Texas.

© Copyright 2026 Poets & Quants. All rights reserved. This article may not be republished, rewritten or otherwise distributed without written permission. To reprint or license this article or any content from Poets & Quants, please submit your request HERE.