The AI that scored 95% — until consultants learned it was AI

Presented by SAP

When SAP ran a quiet internal experiment to gauge consultant attitudes toward AI, the results were striking. Five teams were asked to validate answers to more than 1,000 business requirements completed by SAP’s AI co-pilot, Joule for Consultants — a workload that would normally take several weeks.

Four teams were told the analysis had been completed by junior interns fresh out of school. They reviewed the material, found it impressive, and rated the work about 95% accurate.

The fifth team was told the very same answers had come from AI.

They rejected almost everything.

Only when asked to validate each answer one by one did they discover that the AI was, in fact, highly accurate — surfacing detailed insights the consultants had initially dismissed. The overall accuracy? Again, about 95%.

“The lesson learned here is that we need to be very cautious as we introduce AI — especially ...