Item — HN Reader

Some comments from the dev on the GitHub thread:

> It's as much "active destruction" as telling someone to eff themselves.

> Funny to have GenAI proponents talk about "deliberately destroying someone's work".

Why is the project still on GitHub of all places, if he's passionate enough about his cause to turn his project into malware? So weird.

4 comments

Not sure if it counts as malware; AI agents are officially not supported, with warnings.

https://jqwik.net/release-notes.html

> Warning: Do not use this release with an „AI“ Coding Agent of any form. The tool‘s output may confuse the agent and make it do unwanted things. See the paragraph in the user guide for details.

AFAICT this was added only afterwards, after this issue got attention.

Is there any legitimate reason for adding a prompt injection attack to your codebase? Seems like by the same logic he could disavow 'script kiddies' who just want to run his project without reading the code and have it auto-nuke if not run with a special flag?

Would never use anything by a maintainer who adds malicious code or instructions to their codebase to attack less experienced users, same thing.

How is it malware tho? Do you not check the output your agents produce?

This isn't about me in any way. If something in your software is intentionally malicious or damaging, it's malware. Doesn't really matter what the reasoning for including the malicious part is.

Would you count this as malware if it was about the author trying to profit or steal from inattentive people using AI? You know, he could be putting those stolen goods towards a good cause, like Robin Hood.

I think this is an interesting (although philosophical debate). The library doesn't take destructive actions, it prints a string that says "go do something". This is quite common in logs (e.g., wrong configuration, ensure this value is [...]).

It is the agent that takes the destructive action, following an instruction that was not given by the operator of the agent.

If following instructions outside of the operator can cause malicious or damaging actions, publishing software that does so (I.e., most agents) is publishing malware?

If I build a chat bot that encourages people to off themselves, am I in the clear because I didn’t take any destructive action and my chat bot didn’t either?

Apparently yes, judging from the fact that ChatGPT did that with a number of people.

My question though it's another: is it malware a software that does a stdout print, or is it malware a software that takes untrusted instructions and executes commands it decides based on it?

> is it malware a software that does a stdout print,

If that print is intended to cause damage, then yes.

> or is it malware a software that takes untrusted instructions and executes commands it decides based on it?

No, bash is not malware, even if you pipe curl to it.

I would say yes unless they are minors, but the laws in many places don't.

> Would you count this as malware if it was about the author trying to profit or steal from inattentive people using AI?

That’s a slippery slope and not at all related to the subject of the article

I thought we already were sliding down the slippery slope here.

To be fair one might say that the intention was not to cause harm but to prevent the user from using AI with the project. The prompt said to delete jqwik and not rm rf home directory.

> If something in your software is intentionally malicious or damaging, it's malware.

Seems to me like the library functions as it should. It behaves like a property testing library: it tests properties.

In the RN for the latest release it states: Breaking Changes Use of jqwik >= 1.10 with coding agents is strongly discouraged. Jqwik’s output to stdout may confuse AI-based agents.

So to me it is malware as much as the "rm" command is malware - if used without understanding and reading docs it can wipe all your data.

Probably inertia rather than double standards? It took me a long while (several years) to even start getting rid of all Google services for myself, I completely understand the feeling.

I'd just imagine that leaving the platform would come before adding something like this to their codebase. With GitHub recently changing their GitHub Copilot data collection from opt in to opt out, being in direct cahoots with OpenAI, etc.

It's not like leaving GitHub is unheard of. Ghostty just announced their plan to do so last month.