The AI Bug Bounty Crisis: How Fake Reports Are Overwhelming Security Teams in 2025

Something interesting is happening in the world of bug bounty programs, and it's causing significant challenges for software maintainers. Daniel Stenberg, the core maintainer of curl (the widely used command-line utility for web requests), has documented a troubling trend in an article titled "Death by a thousand slops" that highlights how AI is disrupting vulnerability reporting processes.

The Rising Problem of AI-Generated Bug Reports

Curl is listed as an application on HackerOne, a platform where companies pay security researchers to find vulnerabilities in their software. However, Stenberg and his team have been facing an increasing flood of what he calls "AI slop" - bug reports generated by artificial intelligence that report vulnerabilities that don't actually exist.

According to Stenberg, approximately 20% of all submissions they receive are now AI-generated bug reports with no basis in reality. These reports often appear legitimate at first glance, containing technical jargon and detailed explanations, but upon investigation, they describe code paths or functions that don't even exist in the codebase.

Anatomy of an AI Slop Bug Report

One particularly egregious example highlighted in Stenberg's article was a report submitted in July 2025 claiming to have found a use-after-free vulnerability in libcurl. Use-after-free vulnerabilities are serious memory corruption issues that could potentially allow remote attackers to execute code on a victim's machine.

However, when Stenberg investigated the report, he discovered that the function mentioned in the report, SSL_get_X_data, is used exactly zero times in the libcurl codebase. The AI had completely fabricated both the vulnerability and the code it supposedly existed in.

This is a perfect example of what makes AI-generated bug reports so problematic - they often include plausible-sounding technical details that require significant time and expertise to verify, only to discover they're entirely fictional.

The Real Cost of Fake Bug Reports

Why is this a big deal? Some might argue that triaging bug reports is simply part of a maintainer's job. However, Stenberg explains that each report submitted to the curl security team engages three to four team members for anywhere from 30 minutes to several hours each.

With a security team consisting of only seven volunteers, this represents a significant drain on limited resources. When 20% of reports are completely fabricated by AI, it creates what Stenberg aptly calls a "death by a thousand slops" - essentially a denial-of-service attack on the human resources of the project.

The Legitimate Role of AI in Security Research

It's important to note that AI can play a valuable role in security research when used responsibly. For example, security researcher Sean He successfully used GPT-4 to identify a legitimate zero-day vulnerability in Linux's SMB implementation.

The critical difference is that professional researchers like Sean don't submit every AI-generated finding directly to maintainers. Instead, they carefully verify each potential vulnerability before reporting it, acting as a human filter for AI's output.

This verification step is crucial because AI models have been shown to have an extremely high false positive rate in vulnerability detection. In Sean's research, only about 2% of AI-flagged issues turned out to be actual vulnerabilities - meaning 98% were false alarms that would have wasted maintainers' time.

Proposed Solutions to the AI Bug Report Crisis

To address this growing problem, several potential solutions have been proposed:

• Mandatory disclosure of AI assistance: Requiring researchers to indicate when bug reports were generated or assisted by AI
• Reputation requirements: Limiting bug submissions to researchers with established track records on platforms like HackerOne
• Submission fees: Implementing a small fee (around $15) for each bug report submission, refundable if the report is legitimate, to discourage mass submissions of unverified AI-generated reports

Each approach has trade-offs. While reputation requirements and submission fees would likely reduce the volume of low-quality reports, they could also create barriers for new security researchers trying to enter the field.

Best Practices for Responsible AI Use in Security Research

For security researchers and bug bounty hunters looking to incorporate AI into their workflows, there are several best practices to follow:

1. Always verify AI findings manually before submission
2. Disclose when AI tools were used as part of your research process
3. Understand the limitations of AI in security contexts, particularly its tendency to hallucinate code paths and vulnerabilities
4. Use AI as an assistant rather than relying on it to generate complete reports
5. Take responsibility for all submissions under your name, regardless of how they were generated

Finding the Balance Between AI Assistance and Human Expertise

AI tools can be valuable assistants in security research, helping to identify patterns or potential issues that might be missed by human analysis alone. However, they cannot replace the critical thinking, contextual understanding, and ethical responsibility that human security researchers bring to the table.

The key is to use AI as one tool in a comprehensive security workflow, where human expertise remains the final arbiter of what constitutes a legitimate vulnerability worth reporting to software maintainers.

Conclusion: The Human Element Remains Essential

As we navigate this new landscape where AI can generate seemingly plausible security reports at scale, the human element in security research becomes more important than ever. Security researchers must act as responsible gatekeepers, ensuring that only verified, legitimate vulnerabilities reach the already-stretched resources of open source maintainers.

For the security community to thrive, we need to establish norms and practices that harness the benefits of AI while mitigating its potential to overwhelm the vulnerability reporting ecosystem with false positives. By doing so, we can ensure that critical security resources remain focused on addressing real threats rather than chasing AI hallucinations.