San Francisco's City Attorney's Office issued cease-and-desist letters to Apple and Google this week, demanding the immediate removal of 13 artificial intelligence applications that generate non-consensual nude images. The letters accuse both tech giants of profiting from apps that use face-swapping technology to create explicit deepfake content, with women and girls being the primary targets of this abuse.
These so-called "nudify" apps allow users to upload photos of clothed individuals and use AI algorithms to generate realistic nude images without the subject's knowledge or consent. The technology has become increasingly accessible through mobile app stores, making it easier for bad actors to create and distribute non-consensual intimate imagery. San Francisco's legal action represents one of the first major municipal efforts to hold app store operators accountable for hosting such applications.
The cease-and-desist letters specifically identify 13 face-swap applications available through Apple's App Store and Google's Play Store. While the exact names of the apps were not disclosed in available reporting, the City Attorney's Office argues that both companies are directly profiting from the distribution of these tools through their standard revenue-sharing arrangements with app developers. The technology behind these apps has advanced rapidly, making it difficult for victims to prove images are fabricated.
The proliferation of AI-generated non-consensual intimate imagery has created serious privacy and safety concerns, particularly for women and minors. Victims often face harassment, reputational damage, and psychological harm when these images are created and shared. Current legal frameworks struggle to address this emerging form of abuse, as many jurisdictions lack specific laws criminalizing the creation or distribution of AI-generated intimate content.
San Francisco's City Attorney is demanding that Apple and Google immediately remove the identified applications from their respective stores and implement stronger review processes to prevent similar apps from being published in the future. Organizations and individuals concerned about non-consensual intimate imagery should document any incidents, report abusive apps directly to platform operators, and contact local law enforcement. Several states have begun introducing legislation specifically targeting deepfake pornography, though federal protections remain limited.
Source: https://www.wired.com/story/san-francisco-demands-apple-and-google-delete-ai-nudify-apps-from-app-stores/