AI Fiction in the Wild

To classify the fiction in WildChat, we used an LLM. We showed the model — specifically, GPT-o4-mini — the first three exchanges in each human-ChatGPT conversation and asked whether the user’s prompts contain a request for fiction (as well as fanfiction and/or explicit material). We provide a detailed explanation, specifying that fiction can be any “content that is imaginative, speculative, or not grounded in real-world facts,” including “original stories, speculative scenarios, or alternate histories.” We also provided one example input and a desired output—an approach known as one-shot prompting or in-context learning—using a real WildChat user prompt and the “correct’’ answer according to our schema. All three authors indepenently labeled 300 randomly sampled conversations, and we compared our consensus classifications to the model’s, finding high agreement. You can learn more in our paper.

Because the WildChat data is anonymized, it is difficult to know exactly how many unique users are represented in the dataset. While the original research team recorded hashed IP addresses for every conversation, if someone used WildChat at home, a coffee shop, and the library, that would show up as multiple IP addresses, overcounting users in that case. To address this issue, we bundle together IP addresses that posted at least one near-duplicate fiction prompt (longer than 30 characters) from the same geographic region. We embed the first prompt of all convesations, and we cluster with an algorithm called DBSCAN. If two IPs from the same state ever submitted prompts in the same DBSCAN cluster, we treat the as the same estimated user. We acknowledge that this method is not perfect and may overly merge on templatized prompts.

We determine clusters of story permutations by using the same clustering algorithm, DBSCAN. Each first prompt is encoded with the all-MiniLM-L6-v2 sentence transformer, and we cluster the resulting vectors with DBSCAN. Then we map clusters to estimated users.

The users featured in the WildChat data (Zhao, et al.) consented to have their data used for research purposes. You can read their paper for more details. However, ethical and privacy considerations remain. We might wonder how many of these users actually read the agreement or fully understood the implications of their decision.

Additionally, though the conversations are anonymized — including only a hashed IP address and, based on this IP, an estimated geographic state or region — and though users agreed to share their chats publicly, some users still shared personal and sensitive information. The WildChat authors implemented automated procedures to anonymize personally identifiable information (e.g., personal names, email addresses), but subsequent studies showed that sensitive data remained (Antoniak et al.). These authors notified the WildChat creators, who took steps to scrub this additional information.

In choosing to study and release this data, we acknowledge a trade-off of concerns. While we believe the potential risks and harms for the WildChat users have been sufficiently mitigated, some may still exist. We believe this analysis is justified by users’ explicit consent and by the urgency of helping researchers and the public understand what people are really doing with these tools.

Because the WildChat users are anonymous, we also do not have a clear sense of their demographic breakdown or representativeness. The chatbots were hosted on Hugging Face — a community for sharing machine learning models and datasets — so we might guess, as the original creators do, that these individuals skew more technically literate and “online” than your average user. These instances of ChatGPT were also free, did not require a login or account, and did not have the same rate limits as a regular account. This might mean that users are from lower-income backgrounds, from countries where ChatGPT is banned, or that they were more invested in pushing the boundaries of the model with explicit or prohibited content. WildChat is not a representative sample of all ChatGPT users, with recent research showing a particular dominance of power users compared to a random sample of Microsoft Bing Copilot user chat logs (Hicke and Tomlinson).