TikTok reportedly tested a safety feature by withholding it from millions of users in the United States, leaving a significant portion of its younger audience exposed to repetitive streams of content involving suicide and self-harm. Internal documents reveal that in early 2022, the social media giant excluded approximately 10 percent of its American user base—roughly 15 million people—from an algorithmic update specifically designed to disrupt "filter bubbles" of depressive and dangerous material. The decision was part of an A/B testing strategy intended to measure how safety interventions might impact user engagement and daily active usage metrics.
The revelation underscores a growing tension between the growth-oriented engineering of social media platforms and the mental health of their most vulnerable users. According to confidential internal records, the safety feature was meant to diversify the "For You" feed, preventing users from being bombarded with a singular, often dark, topic. However, for those in the control group, the algorithm continued to function without these safeguards, sometimes with devastating real-world consequences.
The Mechanics of the Algorithmic Filter Bubble
The "For You" page, TikTok’s primary content discovery engine, relies on a sophisticated recommendation system that prioritizes videos based on watch time, likes, and shares. While this system is designed to keep users entertained, it has long been criticized for its tendency to create "rabbit holes." If a user interacts with a video about sadness or loneliness, the algorithm often responds by serving more of the same, potentially trapping the individual in a loop of reinforcing negative emotions.
By late 2021, TikTok leadership publicly acknowledged that its algorithm could inadvertently show users too much of the same troubling material. The company announced it was developing ways to break up these repetitive recommendations, focusing on sensitive subjects like extreme dieting, breakups, and mental health struggles. The goal was to ensure that even if a single video did not violate community guidelines, the cumulative effect of seeing dozens of similar videos would not create a harmful viewing experience.
However, the implementation of this solution was not universal. To validate the effectiveness of the update, TikTok engineers opted for a "backtest," a standard industry practice where a new feature is withheld from a segment of the population to compare behavior against the majority. In this instance, the experiment involved a high-stakes trade-off: testing a safety feature by withholding it from millions of users to see if their engagement levels differed from those protected by the new code.
The Case of Chase Nasca and the Control Group
The human cost of this experimental design became a focal point of internal scrutiny following the death of Chase Nasca, a 16-year-old from Long Island who died by suicide in February 2022. Nasca was among the 10 percent of users excluded from the safety update. An internal review conducted by TikTok more than a year after his death found that his account had become severely "trapped" in a filter bubble.
The company’s reconstruction of Nasca’s feed revealed a staggering volume of content focused on suicide, hopelessness, and self-harm. Out of 7,563 videos served to the teenager in the final two weeks of his life, approximately 73 percent were related to mental health struggles, loneliness, or sadness. The internal report noted that because Nasca was part of the control group, the strategies meant to disrupt these repetitive recommendations "did not take effect on this user by design."
Even more concerning was the finding that nearly 10 percent of the videos served to Nasca actually violated TikTok’s existing policies against promoting or normalizing suicide. Despite these violations, the algorithm continued to push the content to his feed, even during his final hours. The internal review highlighted a critical failure: the platform’s primary defense against harmful content—its safety update—was intentionally disabled for Nasca as part of a corporate experiment.
Balancing Safety Metrics Against User Engagement
The decision to maintain a 10 percent control group for a safety-critical feature highlights the "delicate balance" mentioned in TikTok’s internal documents. Technology companies frequently use A/B testing to refine user interfaces or test new ad placements, but applying these methods to life-saving safety features remains a controversial practice. In this case, TikTok sought to determine whether breaking up filter bubbles would negatively impact "daily active users" (DAUs) or the total time spent on the app.
Internal teams eventually recommended shrinking the control group from 10 percent to less than 1 percent. While a reduction, this still would have left at least 1.8 million U.S. users without the safeguard. The report also suggested limiting the duration of such experiments to seven days, a sharp contrast to the previous testing cycles that lasted anywhere from six months to a full year.
In statements regarding the reporting, TikTok maintained that the experiment was a necessary step to confirm that its safety changes were functioning as intended. The company stated it has since refined its testing protocols, though it declined to provide specific details on how those protocols have changed or what ethical guardrails have been implemented to protect users during future algorithmic trials.
Transparency and Congressional Scrutiny
The timing of TikTok’s internal review has also raised questions about the company’s transparency with federal regulators. The review was completed just weeks before TikTok CEO Shou Chew appeared before the House Energy and Commerce Committee in March 2023. During that high-profile hearing, Chew was questioned extensively about the platform’s impact on children and its data privacy practices.
Nasca’s parents were present in the gallery as lawmakers played clips of the somber and dangerous content found on their son’s account. While Chew told Congress that "safety and wellness—in particular for teens—is a core priority," and emphasized a "commitment to transparency," the internal findings regarding the Nasca account and the 10 percent control group were not disclosed at that time.
In subsequent written responses to Congress in May 2023, TikTok acknowledged using A/B tests to evaluate its "recommendation diversification" strategies. However, when specifically asked to list instances where algorithmic testing had harmed users, the company did not mention the Nasca case or the specific risks identified in its internal audit. Instead, the company reiterated its devotion to community safety and stated it did not believe the platform caused users to commit suicide.
Legal Challenges and Industry Implications
The disclosure of these internal documents arrives as the social media industry faces a wave of litigation. Thousands of lawsuits filed by families and school districts allege that platforms like TikTok, Meta, and Snapchat are designed to be addictive and fail to protect minors from harmful content. Courts are currently navigating the complexities of Section 230 of the Communications Decency Act, which has historically shielded platforms from liability for third-party content.
However, recent legal theories focus not on the content itself, but on the "product design"—specifically the algorithms that push content to users. The argument posits that if a platform’s design intentionally directs a vulnerable minor toward self-harm material, the company may be liable for a defective product. The fact that TikTok reportedly tested a safety feature by withholding it from millions could provide significant ammunition for plaintiffs arguing that the company prioritized engagement metrics over known safety risks.
While TikTok recently settled three high-profile lawsuits involving teen mental health shortly before they were set for trial, the Nasca family is currently appealing the dismissal of their wrongful death suit. The newly revealed internal report remains under court seal in many jurisdictions, but its contents are likely to influence the broader legislative push for the Kids Online Safety Act (KOSA) and other regulations aimed at forcing platforms to exercise a "duty of care."
The Ongoing Mental Health Crisis
The debate over TikTok’s algorithmic experiments occurs against the backdrop of a national youth mental health crisis. The Centers for Disease Control and Prevention (CDC) has reported rising rates of depression and suicidal ideation among teenagers over the last decade, a trend that many experts link to the ubiquity of social media.
For many families, the issue is not just about the presence of "bad" content, but the lack of friction in how that content is delivered. When a platform identifies a user is in a state of distress and responds by amplifying that distress through an automated loop, the technology moves from being a neutral tool to an active participant in the user’s mental state.
As TikTok continues to face pressure from both sides of the political aisle regarding its ownership and safety record, the revelation of its internal testing practices serves as a case study for the tech industry. It raises fundamental questions about whether safety features should ever be used as variables in engagement experiments and how much the public has a right to know about the algorithms that shape their daily reality.
If you’re feeling suicidal or experiencing a mental health crisis, please talk to somebody. You can call or text the 988 Suicide & Crisis Lifeline at 988, or chat at 988lifeline.org. You can reach the Trans Lifeline by calling 877-565-8860 or the Trevor Project at 866-488-7386. Text "START" to Crisis Text Line at 741-741. Contact the NAMI HelpLine at 1-800-950-NAMI, Monday through Friday from 10:00 a.m. – 10:00 p.m. ET.
