Lawsuit: “Police let an error-prone AI system stand in for an investigation.”
A man suing Florida police alleges that cops relied on a faulty facial recognition match and concealed exculpatory evidence when they arrested him on a charge of attempting to lure a child in August 2024. The plaintiff, Robert Dillon, was arrested after a facial recognition system flagged him as a 93 percent match to a suspect filmed by a McDonald’s surveillance camera.
“This case is about what happens when police let an error-prone artificial intelligence system stand in for an investigation,” said the lawsuit filed today. “A facial recognition algorithm flagged Robert Dillon as the man who tried to lure or entice a child under twelve years old at a Jacksonville Beach McDonald’s. It was wrong. Mr. Dillon, a fifty-two-year-old resident of Fort Myers, had never set foot in Jacksonville Beach. But rather than test the machine’s answer against the evidence that would have cleared him, the officers built a case to confirm it. Mr. Dillon was arrested and prosecuted for one of the most stigmatizing crimes a person can face.”
Dillon lives more than 300 miles from Jacksonville Beach, and a police search of a license plate reader database found no evidence he was in the area when the alleged crime was committed, the lawsuit said. Dillon was flagged as the suspect based on a low-quality image, specifically a photo taken of a McDonald’s computer screen that was displaying video surveillance footage, the lawsuit said.
The lawsuit was filed today in US District Court for the Middle District of Florida. The defendants are the City of Jacksonville Beach, Jacksonville Beach Police Corporal Scott O’Connell, Jacksonville Sheriff T.K. Waters, Pinellas County Sheriff Bob Gualtieri, and Sergeant James Walters of the Jacksonville Sheriff’s Office.
The wrong facial identification match came from the Face Analysis Comparison and Examination System (FACES), “the centralized facial recognition database maintained by the Pinellas County Sheriff’s Office,” the lawsuit said. The Jacksonville Sheriff’s Office has access to the system and uses it to conduct searches for itself and “partner agencies, including the Jacksonville Beach PD,” the lawsuit said. Walters was responsible for conducting or overseeing those searches and for transmitting results to other agencies, the lawsuit said.
The prosecution against Dillon was pending for over two months before “the State Attorney’s Office finally dropped all charges,” the lawsuit said. Alleging that Dillon was maliciously prosecuted, the lawsuit demands financial damages and changes in how the police and sheriffs’ offices use facial recognition technology.
Dillon is represented by the American Civil Liberties Union and the Hoguet Newman Regal & Kenney law firm. “The police relied on an incorrect result from facial recognition technology to get an arrest warrant, while concealing evidence that showed he could not have committed the crime he was being accused of,” the ACLU said. “He is one of 15 known people to have this happen to them in the United States.”
The lawsuit said that “Dillon was arrested at his home in front of his wife” and “accused of attempting to lure a child—a charge carrying devastating social stigma and permanent reputational destruction. He was held overnight in jail, forced to borrow money and pledge the title to his truck to post bond, subjected to months of criminal prosecution, and publicly branded with a mugshot that remains accessible online, long after the charges were dropped.”
The arrest was carried out by the Lee County Sheriff’s Office and recorded by a deputy’s body camera. Dillon’s wife told the deputy that her husband had never been to Jacksonville Beach, and Dillon told the deputy that he hadn’t left Fort Myers in two years, the lawsuit said.
Shortly before midnight on November 2, 2023, Jacksonville Beach Police responded to a report of a man trying to lure or entice a minor inside a McDonald’s, the lawsuit said. A girl under the age of 12 told police “that a man had repeatedly asked her if she wanted to leave the restaurant with him. She told him no. The suspect approached her a second time and asked, ‘Are you sure?’ The victim called her mother, and the suspect left.” The victim’s parents were next door, at a hotel.
O’Connell had access to exculpatory and material information that he did not disclose in an affidavit used to obtain an arrest warrant from a magistrate, the lawsuit said. For example, “O’Connell requested a search of automated license plate readers for Mr. Dillon’s two vehicles for the period November 1 through November 3, 2023. The search confirmed that neither vehicle was detected anywhere in Duval County during that period. He omitted this from the warrant,” the lawsuit said.
O’Connell had called Dillon months before obtaining the warrant. Dillon “denied any involvement, stated he had never been to Jacksonville Beach, and described a distinctive scar from his hairline to his nose. O’Connell omitted this call and Mr. Dillon’s exculpatory statements entirely,” the lawsuit said. The affidavit also failed to disclose that facial recognition “results cannot constitute a positive identification, are inherently unreliable, and do not constitute probable cause under Jacksonville Beach PD’s own policy,” the lawsuit said.
O’Connell’s warrant affidavit said a McDonald’s manager provided “a similar suspect description as the victim,” giving the impression that she witnessed the crime, according to the lawsuit. “In fact, she did not because she was occupied with her work duties,” the lawsuit said.
The complaint continued:
O’Connell did not merely fail to investigate. He instead affirmatively chose not to pursue readily available investigative avenues that would have confirmed or excluded Mr. Dillon, including: mobile ordering records, payment data, McDonald’s app account records from the night of the incident, older surveillance footage showing the “regular customer’s” prior visits, a comparison of Mr. Dillon’s physical features (including his scar) to the surveillance footage, cell phone location records, time-stamped photographs, and travel or financial records.
The man in surveillance footage apparently placed an order in advance, likely doing so online or with the mobile app, the lawsuit said, adding that police did not request mobile ordering records or payment and account data from McDonald’s. The lawsuit faulted Jacksonville Beach Police for hiring and putting O’Connell on a sensitive case despite his own legal history.
“O’Connell is an officer with a documented history of volatility and poor judgment, having previously been terminated from the St. Johns County Sheriff’s Office for threatening to ‘blow up’ the agency, later reinstated, then arrested for domestic battery before resigning under the weight of those charges,” the lawsuit said. “Jacksonville Beach PD hired him anyway, assigned him as lead investigator on a sensitive child-luring case, and later promoted him to corporal after his investigation resulted in the wrongful arrest and prosecution of an innocent man.”
FACES contains over 38.5 million images, such as mugshots and driver’s license photos, and at least 196 law enforcement agencies had access to FACES as of 2022, the lawsuit said. Dillon’s face was said to be a 93 percent match with photos of a suspect from the McDonald’s restaurant’s surveillance footage.
These were the photos taken of a McDonald’s computer screen that displayed surveillance footage and not a digital extraction from the video file, the lawsuit said. “These photos add another layer of degradation to the poor quality of surveillance footage, including screen glare, reduced resolution, and color distortion inherent in photographing a screen rather than exporting a digital file from the original video,” the lawsuit said.
“Says it’s 93 percent accurate. Far as I’m concerned, it’s 100 percent inaccurate,” Dillon told Gulf Coast News for an article about the wrongful arrest. The man in surveillance footage has “black wavy hair. I don’t have wavy hair,” he also said.
After the FACES search, “O’Connell requested a photo array from the General Investigations Unit including Mr. Dillon as the person of interest,” the lawsuit said. He presented the photos to a McDonald’s restaurant manager, who identified Dillon as the man “wearing a black trench coat on the night of the incident,” the lawsuit said. The manager had previously described the person in the surveillance footage as a regular customer.
In the photo array, Dillon’s face was “surrounded by five fillers—chosen to resemble Mr. Dillon, not the suspect,” so “Dillon became, almost by definition, the person in the array who most closely resembled the suspect” recalled by the McDonald’s manager, the lawsuit said. O’Connell did not display the photo array to the victim, the lawsuit said.
The 93 percent figure is a confidence score, which “is a measurement of digital proximity between two mathematical templates” and “not a measurement of a probability that the two images depict the same person,” the lawsuit said. Facial recognition algorithms vary based on how they were designed and trained, making it difficult to determine what the score means, the lawsuit said.
“An officer presented with a ‘93% match’ from an AI-powered system has no way to evaluate the basis for that score, no way to assess whether the system’s confidence is warranted, and no frame of reference for understanding what ‘93%’ actually means in probabilistic terms,” the lawsuit said.
Dillon was self-employed as a commercial crabber and was arrested during a particularly lucrative time of the year for his occupation, the lawsuit said. He didn’t work for about a month because he “was unable to concentrate on anything other than the pending charges and the continued public availability of his mugshot” and “did not want to be in public for fear of being confronted as a suspected child abductor,” the lawsuit said.
Dillon fell behind on his monthly rent and returned to work when faced with the possibility of losing his home, the lawsuit said. “Community members still approach him in public to ask about the case,” it said. “He no longer feels comfortable being friendly to children. No law enforcement agency has ever apologized or acknowledged the error.”
The ACLU press release quoted Dillon as saying that he “will never get over how terrified and worried I was, wondering if I’d ever go home to my wife and daughter again.” Dillon said police “relied on this dangerous technology instead of doing their jobs and actually investigating.”
The Jacksonville Sheriff’s Office declined to comment when contacted by Ars today. We contacted the Jacksonville Beach Police and Pinellas County Sheriff’s Office and will update this article if we receive any response.
Source: Ars Technica
