San Francisco City Attorney Demands Apple and Google Remove AI Nudify Apps From Digital Stores to Combat Nonconsensual Deepfake Imagery

In a significant escalation of the legal battle against nonconsensual AI-generated content, the San Francisco City Attorney’s Office has issued formal cease-and-desist letters to Apple and Google. The notices demand the immediate removal of 13 specific artificial intelligence applications that facilitate the creation of "nudified" images, a practice involving the digital removal of clothing from photos of real individuals without their consent. The letters, sent on Thursday, represent a targeted strike against the infrastructure that allows these tools to be distributed and monetized on a global scale.

The San Francisco City Attorney has identified eight applications currently hosted on the Apple App Store and five on the Google Play Store. While many of these applications market themselves as benign "face-swap" or photo-editing tools, the legal office argues that their primary function or heavily marketed secondary function is to generate sexually explicit deepfakes. The City Attorney’s office has characterized these apps as "illegal, harmful, and completely unacceptable," calling on the tech giants to not only remove the software but also to terminate developer accounts and cease the collection of commission fees derived from their sales.

The Allegations of Aiding and Abetting

Central to the City Attorney’s argument is the financial relationship between the platform providers and the app developers. Under current App Store and Play Store models, Apple and Google typically claim a commission of 15% to 30% on in-app purchases and subscriptions. The San Francisco legal team contends that by facilitating these transactions, the tech companies are effectively "aiding and abetting" the creation and distribution of nonconsensual pornography.

The office estimates that Apple and Google have collectively garnered millions of dollars in fees from these specific applications. By providing the hosting environment, the payment processing, and the global distribution network, the City Attorney argues that the platforms are inextricable links in a chain of digital abuse. The demand for removal is predicated on the idea that these apps violate not only common decency but also existing consumer protection and harassment laws.

A Growing Crisis: The Rise of AI-Generated Nonconsensual Imagery

The emergence of "nudify" apps is part of a broader, more alarming trend in the evolution of generative artificial intelligence. Unlike early deepfakes, which required significant technical expertise and high-end computing power, modern AI models—often based on diffusion architectures—can be packaged into user-friendly mobile apps. These tools allow even novice users to upload a photograph of a colleague, acquaintance, or public figure and generate a highly realistic nude version of that person in seconds.

Data from cybersecurity firms and online safety researchers suggest that the volume of deepfake pornography has exploded over the last three years. According to industry reports, nearly 98% of all deepfake videos found online are pornographic in nature, and the vast majority of these target women without their consent. The proliferation of these apps on mainstream platforms like the App Store and Google Play Store provides an air of legitimacy to the tools, making them easily accessible to a wider audience, including minors.

Chronology of Platform Action and Policy Shifts

The battle over "nudify" apps has been a cat-and-mouse game between developers and platform moderators for several years.

Early 2020s: As generative AI began to gain traction, the first wave of "nudify" websites appeared. These were largely browser-based and frequently shut down by hosting providers, only to reappear under different domains.

2023-2024: Developers began migrating these capabilities into mobile applications, often disguising them as "AI Art" or "Face Swap" utilities to bypass initial automated reviews by Apple and Google.

June 2026: Reports emerged that Apple had significantly tightened its App Store Review Guidelines. The update specifically targeted developer responsibility for pornographic content, placing a higher burden of proof on developers to ensure their generative AI tools could not be easily repurposed for creating nonconsensual sexual imagery.

July 2026: Following a series of investigations by media outlets and digital rights researchers, the San Francisco City Attorney’s Office compiled a list of the most egregious offenders still active on the platforms. This led to the formal cease-and-desist letters sent on July 16, 2026.

Official Responses from Apple and Google

Both Apple and Google have long maintained strict policies against the hosting of pornography and apps that facilitate harassment. However, the enforcement of these policies against generative AI tools has proven complex, as developers often hide "nudification" features behind paywalls or "pro" modes that are not immediately visible during the initial app review process.

In response to the City Attorney’s demands, a Google spokesperson confirmed to media outlets that the company has already taken action against the five apps named in the letter. Google stated that it has deleted "hundreds" of similar apps over the past year for policy violations. The company emphasized its use of both automated systems and human reviewers to flag and remove content that violates its "Sexual Content" and "Harassment" policies.

Apple provided a more detailed statement, emphasizing the App Store’s mission as a "safe and trusted place." The company noted that it has proactively rejected thousands of apps that attempt to introduce nudification features. Regarding the eight apps identified by San Francisco, Apple confirmed it has already removed three of them and is in the process of terminating the associated developer accounts. For the remaining four apps, Apple stated it is in communication with the developers, who must address specific policy violations or face immediate expulsion from the platform.

Apple’s response also highlighted the role of user reporting. The company encouraged individuals who encounter offensive or abusive content to utilize the "Report a Problem" portal at reportaproblem.apple.com. This feedback loop is often the primary way the company identifies apps that have successfully circumvented the initial review process.

Legal and Social Implications

The intervention by the San Francisco City Attorney highlights a shifting legal landscape where local and state officials are increasingly willing to challenge the "platform immunity" traditionally granted to tech giants. While Section 230 of the Communications Decency Act has historically protected platforms from liability for content posted by third parties, the argument that Apple and Google are "aiding and abetting" illegal activity by profiting from it represents a potential pivot point in tech litigation.

The social ramifications of these apps are profound. Victims of nonconsensual deepfakes often suffer from severe psychological distress, loss of employment, and damage to their personal reputations. In many cases, the images are used as tools for extortion or "revenge porn." Because the images are synthetic, they often exist in a legal gray area where traditional defamation or privacy laws are difficult to apply.

Furthermore, the presence of these apps on the App Store and Google Play Store creates a "normalization" effect. When a tool is available for download on a major platform, users may mistakenly believe its use is legal or socially acceptable. The City Attorney’s office argues that by removing these apps, the platforms are not just enforcing their own rules, but are fulfilling a moral and legal obligation to protect the public from a new and virulent form of digital violence.

The Path Forward for App Store Governance

The demand for the removal of these 13 apps is likely just the beginning of a broader crackdown. As AI technology continues to advance, the ability to generate realistic imagery will become even more accessible, necessitating more sophisticated moderation tools.

Experts suggest that platforms may need to implement "AI-aware" review processes. This could include:

1. Stress-Testing Generative Models: Requiring developers of AI photo apps to demonstrate that their models have built-in safeguards (such as "negative prompts" or blocked keywords) that prevent the generation of nudity.
2. Financial Audits: Monitoring the marketing materials of apps that see high revenue growth to ensure they are not using "nudification" as a hidden selling point on social media platforms like X (formerly Twitter) or Telegram.
3. Collaborative Blacklisting: Better information sharing between Apple, Google, and law enforcement to ensure that a developer banned on one platform cannot simply migrate to the other under a different name.

As the legal pressure from the San Francisco City Attorney’s Office continues to mount, the tech industry faces a critical moment of reckoning. The balance between fostering AI innovation and preventing the automation of sexual harassment remains one of the most pressing challenges for the digital age. For now, the removal of these apps serves as a vital, if reactive, step toward securing the digital safety of millions of users.