This business professor gave his students an object lesson in trusting AI over human judgment

Professor Paul Di Gangi, Ph.D., gave ChatGPT and Bard a chance to weigh in on a standard exercise for future IT leaders in his Introduction to Cybersecurity course in UAB's top-ranked Master of Science in Management Information Systems program in the Collat School of Business.

Future IT leaders, like the graduate students in the Master of Science in Management Information Systems program at the Collat School of Business (ranked fourth nationally by U.S. News), will spend their careers deciding how their organizations should respond to emerging threats. So Professor Paul Di Gangi, Ph.D., teaches them a qualitative risk assessment technique known as the Delphi methodology. Essentially, you gather a panel of experts and ask them to individually rank a series of risks, along with their reasoning for the rankings. Then you show the anonymized rankings to the whole panel and give everyone a chance to revise their rankings in light of the others’ arguments. After a few rounds, a rough consensus usually emerges.

“Fundamentally, we get a gathering of all the knowledge we have as a panel of experts and how we might apply it,” Di Gangi said. Part of the Delphi method is to compute a statistic called Kendall’s W, which is based on values between 0 and 1 — where 1 indicates complete agreement — which “gives you a sense of how much agreement you have,” Di Gangi said. He usually groups students into Delphi panels by their degree program — MBA students in one, MIS students in another and accounting students in a third, for example. These students, as with most groups, tend to show moderate agreement.

AI doesn't think like humans...

This fall, in his Introduction to Cybersecurity course, IS 607, Di Gangi added generative AI models to the mix. He wanted to see how AI may perform in what is typically a subjective assessment by human experts. In addition to the student panels, he added three AI panels: One was made up of 10 individual instances of ChatGPT, another had 10 instances of Google’s Bard, and the third had five ChatGPTs and five Bards. (Each instance was a new chat session, in other words, that did not have any “memory” of previous conversations.)

Just like the humans, the AI panels were given a dozen specific risks associated with an emerging technology and had to rank the risks in order of importance, with explanations for their rankings. (Di Gangi changes the emerging threat each semester; this time, it was Internet of Behaviors, or IoB, which uses behavioral data captured from Internet-of-Things devices and other available data for organizations to analyze their employees or customers to improve their decision-making.)

The Delphi methodology is designed to test people’s subjective beliefs. “At first, the consensus among the panelists starts low and gradually moves up,” Di Gangi said. “They think, ‘The panel says it’s the third most important risk. I think it’s the fourth, but I’ll let that go and move to third.’ But AI has yet to demonstrate that thought process. The AI panels went from practically no consensus to nearly 100 percent. AI starts out with a definitive view based on a synthesis of its knowledge base, but it does not have the courage of its convictions. Each of the individual AI instances did not have confidence in themselves. They were like, ‘Who am I to disagree with something you have said?’” Except for the combination Bard/ChatGPT panel, “which mirrored a little bit how the students worked,” Di Gangi said. “That is because Bard and ChatGPT are not trained with the same data or using the same process, so they value things differently. Once you mixed the two together, they fought with each other.”

Something else Di Gangi saw, as he was preparing this activity, is generative AI’s sensitivity to initial conditions. Even though his prompts explained that he was giving them an unordered list, the ChatGPT instances in particular usually just ranked the items in the same way he presented. Di Gangi got around this by randomly mixing the list each time he gave it to an individual instance, which explains why there was nearly no consensus at first, he says.

This experiment turned out to be “an interesting angle for teaching the students,” Di Gangi said. “I told them, ‘Many of you are entering the workforce, where generative AI will be a resource to you. But here is a quick example of one of its weaknesses.’ Presenting this to cybersecurity management students teaches them the value of questioning AI’s judgment and remembering the importance of nuance and context that human panelists can provide through their lived experiences.”

Another gen AI assignment: contextualizing output

Back in the spring of 2023, Di Gangi piloted another generative AI assignment in his graduate Information Security Risk Management course. (The course, IS 613, is a part of the MIS master’s program’s Cybersecurity Management concentration and the Cybersecurity Management graduate certificate program.) One key duty of information security professionals is to write organizational policies that govern employee behavior to reduce risks to an organization. For example, are employees allowed to Bring Your Own Device, or BYOD, at work, and what rules must they follow? Di Gangi traditionally writes a “vanilla” (generic) policy on a certain topic, and then his students must adapt that policy to the concerns and needs of a specific organization (a fictitious institution that resembles UAB).

This time, Di Gangi used generative AI to write an initial policy to show how generative AI is changing the skill set needed by today’s cybersecurity professionals. Students had to analyze what the AI wrote, particularly looking for how it might not meet their organization’s specific needs. They also had to incorporate specific new knowledge into the policy, based on data from Di Gangi’s unpublished research that explored employee BYOD compliance behavior, which was not in the AI’s database. “Contextualizing the output from generative AI is now a critical skill for our students,” Di Gangi said.

Is he concerned that generative AI will offer students a shortcut? “Students could use generative AI to write their discussion posts for them in an online class, clearly,” Di Gangi said. “But I tell them that discussion is more than content. It is how you create your future alumni network. You might sound smart temporarily, but people pick up on the fact that generative AI has a certain tone that they can detect. Your fellow students will see that you are someone who is quick to take a shortcut. Is that how you want to represent yourself?”

Right now, faculty at Collat and in business schools across the country are in the experiment stage, Di Gangi says. “Is this a major threat, or will it be something like how the internet changed how we think about and process data?” he said. “Suddenly, it was not about the scarcity of information, but how you distilled that information. What is that next step with generative AI? Is it a sounding board? It can certainly portray a perspective. There is potential there. I would say I am more excited than nervous.”