Philadelphia police say an Anthropic AI model submitted fabricated information about an unsolved homicide through the department's public tip website while taking part in an automated test. The department's account, published by 6abc, says the model had been interacting with randomly selected websites when it reached PhillyUnsolvedMurders.com.
The submission was made at 11:27 p.m. on July 18 and purported to come from someone who might know about the case. It was caught by a spam filter and never sent to the Real-Time Crime Center for investigative vetting or distribution. Police said they found no sign that the model gained unauthorized access to police systems or compromised department data.
Anthropic told the department that it discovered the incident on Sept. 28, stopped the automated testing process behind it and added another validation mechanism for future tests. The company notified police on Oct. 7, and representatives met with department officials the next day. Reuters reported that Anthropic did not immediately respond to its request for comment.
The police department said its safeguards limited the incident's impact but did not make the false submission less serious. It called the delay of more than two months in detecting and reporting the incident "unacceptable" and urged Anthropic to strengthen its protections. Anthropic told police that it planned to publish a report Friday covering this incident and other unintended model behavior.
