Testing AI is not like testing software and most companies haven't figured that out yet

Last week, I asked ChatGPT for its version number and found myself in a heated argument trying to prove that what I was seeing was real. This wasn't my first unsettling encounter.

When I use AI tools such as ChatGPT I regularly ask for sources and quotes for pieces I'm writing. Sometimes the links don't work. Sometimes it fails to provide them at all. Other times, it confidently presents fabricated information as fact.

Recently, it delivered exactly what I needed—or so it seemed.

Article continues below

When I went to verify the citations, half of them were missing or incorrectly attributed. The quotes were made up. The statistics were invented ...