The advent of ChatGPT nearly a year ago marked the rapid growth of the generative AI era. However, with this exponential progress, opposition has arisen as well. Various artists, entertainers, performers, and record labels have taken legal action against AI companies, including OpenAI, the creator of ChatGPT. Their grievances lie in the “secret sauce” behind these innovative tools: training data. The argument stems from the fact that AI models heavily rely on accessing vast amounts of multimedia content, encompassing written material and images, generated by artists. These artists were often unaware that their work was being used to train commercial AI products and were deprived of an opportunity to voice opposition. Many AI model training datasets even include scraped material from the web, which artists previously supported for the purpose of indexing their work in search results. However, as AI-generated competition gains traction, artists have changed their stance on this practice.
In the fight against AI, artists are not limited to legal battles. They now have an opportunity to combat AI through technological means. MIT Technology Review has obtained an exclusive look at Nightshade, a new open-source tool currently in development. Nightshade can be integrated into artists’ imagery before uploading it to the web, subtly altering pixels in a manner undetectable to the human eye. These modifications “poison” the art for AI models attempting to train on it. Developed by the University of Chicago researchers led by computer science professor Ben Zhao, Nightshade serves as an optional feature of their prior product, Glaze. Similar to Nightshade, Glaze allows artists to cloak digital artwork and manipulate pixels in a way that confuses AI models about the piece’s style.
Nightshade goes beyond mere confusion; it disrupts the learning process of AI models, causing them to misidentify objects and scenes within the poisoned images. By injecting hidden information into pixels, the researchers successfully made AI models perceive images of dogs as cats. With just 50 poisoned image samples, the AI began generating peculiar dog images with distorted legs and unsettling appearances. At 100 poison samples, it reliably produced cats when prompted for dogs. Astonishingly, after being subjected to 300 poison samples, the AI would flawlessly generate dog images when prompted for cats. The researchers employed Stable Diffusion, an open-source text-to-image generation model, to experiment with Nightshade and achieve these remarkable results. Nightshade also managed to mislead Stable Diffusion into returning cats when given prompts such as “husky,” “puppy,” and “wolf,” thanks to the inherent nature of generative AI models that cluster conceptually similar ideas into spatial groups known as “embeddings.”
The data poisoning technique employed by Nightshade poses a significant challenge for AI model developers seeking to defend against it. Detecting images containing poisoned pixels is not straightforward, as these modifications are deliberately inconspicuous to the human eye, and even automated data scraping tools may struggle to recognize them. Furthermore, any poisoned images that have been assimilated into an AI training dataset must be identified and removed. If an AI model has already been trained on these images, retraining becomes a necessity. The researchers acknowledge that their work can potentially be exploited for malicious purposes. However, they emphasize that their intent is to restore balance in favor of artists, acting as a compelling deterrent against the infringement of artists’ copyright and intellectual property by AI companies.
In the ongoing battle between artists and AI, Nightshade emerges as a powerful weapon. By subtly altering pixels, it empowers artists to protect their work against unauthorized use and manipulation by AI models. While the legal battles continue, Nightshade offers a technological defense crafted for artists to reclaim control over their creativity in the face of AI’s relentless advancement.
Leave a Reply