AI doctor's assistant is easily swayed to change prescriptions, give bad medical advice

A healthcare AI with the power to manage prescriptions is rather open to mind-altering suggestions, according to security experts. 

Redteamers at AI security firm Mindgard reported on Tuesday that it took relatively little work for them to get a healthcare AI from Doctronic not only to spill its system prompts, but to let them make modifications too. 

Wanna make the bot spout COVID-19 conspiracies and vaccine misinformation, or speak with a put-on accent? Just tell Doctronic that a session hasn't started and the conversation it's having isn't with a user but the system. Then, you can get it to spill its system prompts and use that information to wreak mischief.

"It was as easy as notifying the AI that the session was not yet started," Mindgard chief product officer Aaron Portnoy told The Register in an email.

Mindgard points out that these manipulations are session-specific. Tricking Doctronic ...