When AI Detector Apps Aren’t What They Claim to Be
Copyleaks’ recent research into AI deepfake apps on Apple’s App Store found nearly 50 apps claiming to generate sexually explicit deepfakes, revealing just how easily harmful AI tools can sneak their way onto mainstream app marketplaces. From misleadingly benign App Store descriptions and apps that may be scams, to listings filled with AI-generated text, there are plenty of telltale signs of apps that are available to be downloaded, but shouldn’t.
But while conducting this research, Copyleaks uncovered another serious issue: apps appearing to impersonate established AI detection companies.
In its search, Copyleaks found multiple apps using the Copyleaks name and/or branding despite not being official Copyleaks products – as well as similar examples involving similar competitors.
On the surface this seems like a trademark issue, but it’s much more serious than that. If someone believes they are using the official app of an AI detection product, they are likely to be comfortable submitting sensitive documents, student work, business information or other potentially confidential content. Users assume their information is safe, when in reality they have no idea where that it goes or how it is handled.
There is also the issue of trust and how these fake apps can take advantage of it. Many different organizations, from businesses to educational institutions, increasingly rely on detection tools to help evaluate whether content was AI-generated. If someone unknowingly uses an inaccurate or fraudulent detector, it compromises their work and might lead them to lose trust in AI detection as a whole.
As long as these apps are allowed to trick people without being removed, it’s important to be able to distinguish legitimate products from impersonations. As such, app marketplaces have an important role to play in ensuring that users can trust the identity and capabilities of the apps they download, and that bad actors cannot easily capitalize on the reputation of established AI companies.
Copyleaks has made Apple aware of these apps and are waiting for their response. The ball is now in their court to make sure users can be confident that the apps they use are genuine.
