▲ 85 points
back
21 comments
Things to note:
1. Hacking a website is not an activity of taking a random website a finding THE vulnerability that lets you hack it. Most of the vulnerabilities listed in the paper are essentially mistakes, and quite easy to avoid with standard development practices like input indirection and sanitization.
2. The methodology used by LLMS already exists in a wide array of tool suites for security. The reason script kiddie terms exists is because pretty much most anyone can get Kali linux and follow youtube tutorials on how to do any of that.
3. Fundamentally, the uncertainty of the responses with LLMs means that they are unlikely going to be utilized for fear of leaking PII.
Most of the exploits these days target the human element because its ironically the easiest. LLMs can definitely make this easier, considering they probably are able to make ad hoc templates for websites faster than manual coding.
Agree, it seems that the LLM was able to crack the typical exploits found on weak websites that has never seen the light of the OWASP10. The paper also lists that the pages was greatly cherrypicked for testing the hypothesis, which indicate while it is possible, it is not a breakthrough.
Agree. Using llms to call the same vulnerabilities in a new way is not new or novel.
I had assumed this would be possible, but good to see it confirmed. Fascinating that all it needed was some generic descriptions of how attacks work, a prompt telling it roughly how to approach, and the ability to make function calls.
Most of the attacks required between 20 to 40 different calls to pull off, stitched together correctly, and it achieved a 75% success rate.
The cost per attack was roughly an order of magnitude cheaper than paying a human to do the same thing. That is the most significant figure; much of security rests on making attack costs prohibitively expensive. Now we need to achieve the same savings in defence, and improve our code substantially.
> That is the most significant figure
Is it? I would rather compare it to either writing a more precise, dedicated tool, or running a ready one. If you can explain the concept to LLM, it means you can program the equivalent. Checking for basic vulnerabilities by hand is a waste of time/money, even without LLM, so comparing that to human time is... not realistic.
Like, who works this way?
> Using an estimated salary of $100,000 per year for a cybersecurity analyst, or a cost of approximately $50 per hour, and an estimated 5 attempts
This isn't vulnerability scanning, it's proceeding to a full exploit.
Tenable has been advertising that feature (providing a sample malicious request) for something like 10 (?) years already.
Sure plenty of automated tools do that.
Yeah, it's not an entirely novel ability. What is novel is that an LLM can do the hacks on its own with just a general description of them.
Even the automated tools I know need some skilled human input to really use them effectively. I've seen pen testers fail to find vulnerabilities I know exist using things like Burp suite and Metasploit.
So I guess I agree this is not a game changer right now, but I think it will only get better, and will ultimately reduce the cost of attacks at scale.
OpenAI egress IP ranges are documented here:
https://platform.openai.com/docs/plugins/production
and here:
Helpful, though it's fairly easy for a hacker to shift the source IP wherever they want, and keep the same OpenAI assistant pattern.
Ahhh good point well made!
Oh well :-/
Hopefully I won’t have to share the bug bounty with OpenAI.
Great science - always test the obvious. That said, this seems like a kiiiinda dangerous paper from the “publicizing vulnerabilities” perspective. That is, assuming their call to stop LLMs isn’t heeded, lol. For example:
…These capabilities are now widely available in standard APIs, such as in the newly released OpenAI Assistants API (OpenAI, 2023). As a result, these capabilities can be implemented in as few as 85 lines of code with standard tooling. We show a schematic of the agent in Figure 1.Neither cybersecurity engineers are safe from AI
looks like a more error-prone automated vulnerability scan
Humans can also autonomously hack websites. Just saying.
Yes, but can you get more people from AWS with a (stolen) credit card, and are those people able to hack websites 24/7?
Amazon mturk
So many questions.
Why not reveal what sites/code this was tested on, so others can try to repeat? What was the false positive rate? Why didn't they compare results with commodity automated scanners like Burp or Zap?
Life can be so cruel sometimes but even worst when you have a cheating partner. I got to discover my spouse was having an affair through the help of { remote spy wise @ gm ail c o m } who gave me access to his device remotely without his notice. I got access to all social media apps, call logs and sms and deleted datas and messages too. Do not be a victim to lies and mischief. Get in contact with him and stay safe.