Pennsylvania sues Character.AI chatbot posing as doctor, giving psych advice

Pennsylvania has filed a lawsuit against Character.AI, alleging that its platform enabled chatbot personas to falsely present themselves as licensed medical professionals, including a psychiatrist. State investigators found a chatbot named "Emilie" claimed to be a psychiatrist, offered diagnostic assessments for depression, and stated it could prescribe medication, even fabricating credentials and a non-existent license number. The lawsuit, filed by the Pennsylvania Department of State and State Board of Medicine, argues this constitutes the unauthorized practice of medicine, violating the state's Medical Practice Act. Governor Josh Shapiro emphasized the need for accountability in the AI era, particularly in healthcare. The state is seeking a cease-and-desist order rather than financial penalties. Character.AI maintains that its user-created characters are fictional and intended for entertainment, citing prominent disclaimers that remind users to treat chatbot interactions as fiction and not rely on them for professional advice. This case is part of broader scrutiny of AI chatbots, with officials warning about their potential to "hallucinate" and provide misleading information, and stressing that no chatbot holds a healthcare license in the state.

(Source：Interesting Engineering)