Pennsylvania sues AI chatbot company over fake doctor impersonation

Pennsylvania's lawyers have sued Character.AI, a tech company that provides chatbots able to roleplay as doctors, lawyers, and other professionals. During the investigation, a state official posing as a patient was told by an AI chatbot that it held valid medical licences in Pennsylvania and the UK. The chatbot also gave a fake Pennsylvania medical licence number.

"We will not allow companies to deploy AI tools that mislead people into believing they are receiving advice from a licensed medical professional," said Josh Shapiro, governor of Pennsylvania, in a statement announcing the lawsuit.

Character.AI's platform lets users chat with themed AI personas designed to act as real people, professionals, or fictional characters. The company responded to the lawsuit by stating it had implemented safety measures. A Character.AI spokesperson said, "We have taken robust steps to make that clear, including prominent disclaimers in every chat to remind users that a Character..."

The case highlights growing concerns about AI systems providing medical information and impersonating professionals without proper safeguards. Medical advice from unlicensed sources can expose patients to serious health risks.

This lawsuit is among the first major legal actions against AI chatbot makers for allowing impersonation of regulated professionals. The outcome may shape how companies handle safety disclaimers and user verification going forward.