back
264 comments
An entry fee that is reimbursed if the bug turns out to matter would stop this, real quick.

Then again, I once submitted a bug report to my bank, because the login method could be switched from password+pin to pin only, when not logged in, and they closed it as "works as intended", because they had decided that an optional password was more convenient than a required password. (And that's not even getting into the difference between real two-factor authentication the some-factor one-and-a-half-times they had implemented by adding a PIN to a password login.) I've since learned that anything heavily regulated like hospitals and banks will have security procedures catering to compliance, not actual security.

Assuming the host of the bug bounty program is operating in good faith, adding some kind of barrier to entry or punishment for untested entries will weed out submitters acting in bad faith.

Bug bounties often involve a lot of risk for submitters. Often the person reading the report doesn't know that much and misinterprets it. Often rules are unclear about what sort of reports are wanted. A pay to enter would increase that risk.

Honestly bug bounties are kind of miserable for both sides. I've worked on the recieving side of bug bounty programs. You wouldnt believe the shit that is submitted. This was before AI and it was significant work to sort through, i can only imagine what its like now. On the other hand for a submitter, you are essentially working on spec with no garuntee your work is going to be evaluated fairly. Even if it is, you are rolling the dice that your report is not a duplicate of an issue reported 10 years ago that the company just doesn't feel like fixing.

> I've since learned that anything heavily regulated like hospitals and banks will have security procedures catering to compliance, not actual security.

Sadly, yeah. And will do anything only if they believe they can actually be caught.

An EU-wide bank I used to be customer of until recently, supported login with Qualified Electronic Signatures, but only if your dongle supports... SHA-1. Mine didn't. It's been deprecated at least a decade ago.

A government-certified identity provider made software that supposedly allowed you to have multiple such electronic signatures plugged in, presenting them in a list, but if one of them happened to be a YubiKey... crash. YubiKey conforms to the same standard as the PIV modules they sold, but the developers made some assumptions beyond the standard. I just wanted their software not to crash while my YubiKey is plugged in. I reported it, and they replied that it's not their problem.

A problem with this approach is that one of the key functions of a bug bounty program is to encourage people to report vulnerabilities to the developers, rather than selling them elsewhere.

If I have to pay money to submit a vulnerability to the developers with no guarantee that I'll even get refunded for a high quality and good faith report, let alone any actual payout, there's much less incentive for me to do so compared to selling them to someone else who won't charge me money for the privilege.

> An entry fee that is reimbursed if the bug turns out to matter would stop this, real quick.

That adds an extra layer of complexity to the cURL maintainers, handling other people’s money and whatnot. It was considered.

Daniel (cURL’s lead) has been discussing this for months. Whatever “quick and easy” solution you think of, it’s already been suggested and thought about and rejected for some reason.

> I've since learned that anything heavily regulated like hospitals and banks will have security procedures catering to compliance, not actual security.

I personally came to that conclusion thanks to the GrapheneOS situation regarding device attestation. Insecure devices get full features from some apps because they are certified, although they cite security, while GrapheneOS get half featured apps because it's "insecure" (read, doesn't have the Google certification, but are actually the most secure devices you can get, worldwide)

> An entry fee that is reimbursed if the bug turns out to matter would stop this, real quick.

I refer to this as the Notion-to-Confluence cost border.

When Notion first came out, it was snappy and easy to use. Creating a page being essentially free of effort, you very quickly had thousands of them, mostly useless.

Confluence, at least in west EU, is offensively slow. The thought of adding a page is sufficiently demoralizing that it's easier to update an existing page and save yourself minutes of request time outs. Consequently, there's some ~20 pages even in large companies.

I'm not saying that sleep(15 * SECOND) is the way to counter, but once something becomes very easy to do at scale, it explodes to the point where the original utility is now lost in a sea of noise.

I found that banks are one of the worst organizations when it comes to authentication. They are regulated but the requirements are completely outdated and irrelevant in a risk context.

And then you have banks such as Boursobank (a French online bank) that has weak traditional authentication (and a faulty app, but they do not care) and out of the blue also provides passkeys. Making it at the same time horribly bad and wonderfully good.

The worst part is that they hide behind regulations when in fact there are only few of them.

Other instiytutions such as SWIFT are as bad and equally arrogant.

For weak bank logins, my guess is that reimbursing all account takeovers is cheaper than having a complex login process that would scare away non-technical customers. Or, well, I could see myself making that decision if I were more versed in finance than in computer science and I had a reasonable risk assessment in front of me to tell me how many account takeovers happen.
That anecdote is hilarious and scary in equal measures. Optional passwords are certainly more convenient than required ones, but so are optional PINs. The most convenient UX would be never needing to log in at all! Unless you find it inconvenient for others to have access to your bank account of course
cURL would operate such a program in good faith, and quickly earn the trust of the people who submit the kind of bug reports cURL values.

Your bank would not. Nor would mine, or most retail banks.

If the upfront cost would genuinely put off potential submitters, a cottage industry would spring up of hackers who would front you the money in return for a cut if your bug looked good. If that seems gross, it's really not - they end up doing bug triage for the project, which is something any software company would be happy to pay people for.

There's also the issue of what happens to my money as a researcher. Is it paid to the company, or is someone holding it in escrow? What if it takes the developer months to respond, or they never do? Do they just get to keep my money indefinitely? What if the vendor pulls out of the scheme? What if I do a chargeback on the payment I made? Etc, etc

I wonder if a better model would be to make the platform pay to entry, but not the specific bugs? So you have to pay a fee to gain access to a platform like HackerOne, and if your signal:noise ratio gets too bad then your account gets revoked? That would make it feel like less of a gamble than having to pay for every individual bug - but still has the same problem that it's putting a big barrier in front of legitimate good-faith researchers.

I've been active in the bug bounty community for almost 7 years now. The problem is that the majority of companies don't act in good faith.

Even when you have something fully exploitable and valid, they will many times find some way to not pay you or lower the severity to pay you very little.

The catch-all excuse is something along the lines: "although this is vulnerable, it doesn't impact the business".

I've gotten this excuse, even when I could prove it was a production server with customer information that I could access.

Sites like Hackerone can help, but in the end, it comes down to the company running the bug bounty program.

Agreed, although the reimbursement should be based on whether a reasonable person could consider that to be a vulnerability. Often it’s tricky for outsiders to tell whether a behaviour is expected or a vulnerability
Are bug reports a 100% sure black and white thing?

Could people who think they found a bug but not sure be turned off by the up front cost / risk of finding out they are wrong or not technically finding a bug?

> An entry fee that is reimbursed if the bug turns out to matter would stop this, real quick.

It would also stop a lot of genuine submissions unfortunately, as some literally can't pay not just won't pay (for both technical or financial reasons), and adds complexity¹. Each project working this way will need to process a bunch of payments and refunds on top of the actual bounty payments, which is not admin free nor potential financially cost free.

I can't think of an easy answer that would work for more than a very short amount of time. As soon as there is money involved and an easy way to use tooling rather than actual effort/understanding to be involved, many will try to game the system ruining it for those genuine participants. Heck, even if the reward is just credit² rather than money, that will happen. Many individual people are honest and useful, people as a whole are a bunch of untrustworthy arseholes who will innocence you and the rest of the world for a penny or just for shits & giggles.

> Assuming the host of the bug bounty program is operating in good faith

This is a significant assumption. One that is it harder to not be paranoid about when you are putting money down.

> they closed it as "works as intended", because they had decided that an optional password was more convenient than a required password

This does not surprise me. My primary bank (FirstDirect, UK) switched the way I authenticate from “between 5 and 9 alphanumeric characters”³ to a 5-digit pin, and all their messages about it assured me (like hell!) that this was “just as secure as before”…⁴

--------

[1] Needing a payment processing option that is compatible with both the reporter and reportee, at the point of submission. At the moment that can be arranged after the bounty is awarded rather than something a project like curl needs to have internationally setup and supported before accepting submissions.

[2] ref: people submitting several simple documentation fixes, one misplaced comma or 'postrophe per pull request, to game some “pull requests accepted” metric somewhere.

[3] which wasn't ideal to start with

[4] I would accept the description “no less secure than before” if they admitted that the previous auth requirements were also lax.

PIN only isn't too uncommon for online banking these days.

You still need to complete a SMS auth to do anything other than view records though, like transfer money.

It may stop it but it may also hinder real people contributing. How many people want to pay a free in order to "contribute"?
> An entry fee that is reimbursed if the bug turns out to matter would stop this, real quick.

The problem is that bug bounty slop works. A lot of companies with second-tier bug bounties outsource triage to contractors (there's an entire industry built around that). If a report looks plausible, the contractor files a bug. The engineers who receive the report are often not qualified to debate exploitability, so they just make the suggested fix and move on. The reporter gets credit or a token payout. Everyone is happy.

Unless you have a top-notch security team with a lot of time on their hands, pushing back is not in your interest. If you keep getting into fights with reporters, you'll eventually get it wrong and you're gonna get derided on HN and get headlines about how you don't take security seriously.

In this model, it doesn't matter if you require a deposit, because on average, bogus reports still pay off. You also create an interesting problem that a sketchy vendor can hold the reporter's money hostage if the reporter doesn't agree to unreasonable terms.

Github can use youtube strikes like system. PRs are tied to people. Someone reported for submitting slop should get a badge or something similar.

If a PR is submitted by someone who is then known to submit slops, they can be easily ignored by the maintainers.

EDIT: Or may be something like SponsorBlock for youtube. There could be a browser extension that will collectively tag sloppers the sameway and can help identify sloppers.

> I've since learned that anything heavily regulated like hospitals and banks will have security procedures catering to compliance, not actual security.

This is the key insight. Nobody cares at all about actual security. It is all about checklists and compliance.

It seems open source loses the most from AI. Open source code trained the models, the models are being used to spam open source projects anywhere there's incentive, they can be used to chip away at open source business models by implementing paid features and providing the support, and eventually perhaps AI simply replaces most open source code
It has also really accelerated the army of hustlers who can't actually code but want to get a contributor badge on major repos to put on their resume.

It's not like this sort of hustling didn't exist prior to LLMs but the volume has ballooned massively.

Outside of direct monetary gain like bounties are efforts to just stand out, in terms of being able to show contributions to a large project or getting say a CVE.

Stenberg has actually written about invalid/wildly overrated vulnerabilities that get assigned CVEs on their blog a few times and those were made by humans. I often get the sense some of these aren't just misguided reporters but deliberate attempts to make mountains out of molehills for reputation reasons. Things like this seem harder to account for as an incentive.

> “Not much. The real incentive for finding a vulnerability in cURL is the fame ('brand is priceless'), not the hundred or few thousand dollars. $10,000 (maximum cURL bounty) is not a lot of money in the grand scheme of things, for somebody capable of finding a critical vulnerability in curl.”

That's the choice as seen from the perspective of a white-hat hacker. But for an exploitable vulnerability, the real choice is to sell it to malware producers (I'm including state-sponsored spyware companies like the makers of Pegasus in this category) for a lot of money, or do the more moral thing and earn at least a little bit of money via a bug bounty program.

Hackerone (where cURL hosted their bounty program) tracks the reputation of bounty hunters. I don't understand why they are not taking advantage of this. Make a private program, invite only hackers who have proved themselves by submitting relevant reports.
A video showing some of the gems which most likely led to this frustration: https://youtu.be/8w6r4MKSe4I?si=7nfRd0VmX8tvXAnY
The company I work for has a pretty bad bounty system (basically a security@corp email). We have a demo system and a public API with docs. We get around 100 or more emails a day now. Most of it is slop, scams, or my new favourite AI security companies sending us an AI generated pentest un prompted filled with false positives, untrue things, etc. It has become completely useless so no one looks at it.

I had a sales rep even call me up basically trying to book a 3 hour session to review the AI findings unprompted. When I looked at the nearly 250 page report, and saw a critical IIS bug for Windows server (doesn't exist) existing at a scanned IP address of 5xx.x.x.x (yes an impossible IP) publically available in AWS (we exclusively use gcp) I said some very choice words.

It makes sense. This process of searching for bugs was slow and time-consuming so it needed to be incentivized. This is no longer the case. Now the hard part is in identifying which ones are real.

To paraphrase a famous quote: AI-equipped bug hunters find 100 out of every 3 serious vulnerabilities.

A list of the slop if anyone is interested:

https://gist.github.com/bagder/07f7581f6e3d78ef37dfbfc81fd1d...

I just read one of the slop submissions and it's baffling how anyone could submit these with a straight face.

https://hackerone.com/reports/3293884

Not even understanding the expected behaviour and then throwing as much slop as possible to see what sticks is the problem with generative AI.

It's really hard to comprehend how entitled "reward which would be the cost of lunch in Sweden can be massive for those low socio-economic-located people" is.

A lunch in central Stockholm is well within 200 kronor. I can't imagine a country where an person with a computer and skills necessary to claim a bounty in cURL would consider that amount massive.

Get real.

Smart, bug bounties are a huge PITA.
I'm not sure how effective this will be. A lot of AI-generated bug bounty reports are pure spam, but a significant fraction are well-meaning humans who genuinely believe the nonsense an LLM has given them. The former category do not read the rules in the first place and will not be deterred - spray and pray is their MO. The second category will not believe that any "no slop" rules apply to them, because they genuinely think their bug is real.
related: cURL stopped HackerOne bug bounty program due to excessive slop reports https://news.ycombinator.com/item?id=46678710
What I wonder is if this will actually reduce the amount of slop.

Bounties are a motivation, but there's also promotional purposes. Show that you submitted thousands of security reports to major open source software and you're suddenly a security expert.

Remember the little iot thing that got on here because of a security report complaining, among other things, that the linux on it did not use systemd?

So AI slop is used to attempt to degrade the quality of cURL.
Why did “AI slop” disappear from the title?
Free work eventually turns out not to be free at all.
This is silly, people don't need AI to send you garbage. If your project is getting lots of junk reports, you should take it as a good sign, that people are looking at it a lot now. You don't remove the incentive, you ask for help to triage the junk.

Curl is a popular and well supported tool, if it needs help in this area, there will be a long line of competent people not volunteering their time and/or money. If you need help, get more help. don't use "AI slop" as an excuse to remove the one incentive people have to not sell exploits or just hoard them.

Alternate headline: AI discovering so many exploits that cybersecurity can't keep up

Am I doing this right?

Funny how we are now sensitivized to these AI slops, at first I fixated on the En dashes in the lead of the article, made me doubt of the article's author for a few seconds.
Just use an LLM to weed them out. What’s so hard about that?
The solution for this, IMO, is flags. Just like with CTFs, host an instance of your software with a flag that can only be retrieved after a successful exploit. If someone submits the flag to you, there is no argueing about wether or not they found a valid vulnerability.

Yes, this does not work for all vulnerability classes, but it is the best compromise in my mind.