back
191 comments
> I went through a phase a while back of holding down keys to see what they did.

Back when I was a gamedev at EA, one of the things QA would do is button-mash test the games. Just smash as many buttons as they could at the same time at all sorts of random points in the game. This was a constant source of bugs. It was surprisingly easy to get the game into a state where it was totally hung because of this.

One of the main culprits was transitions between screens in the UI. So much of the UI code assumed that the initial state of a screen is that no buttons are currently pressed. But if you mash a bunch down in the middle of a transition, the screen can end up receiving a button up event that did not precede any button down event. If the screen's code assumed every up has a preceding down, it could get into a broken state.

I never did see any clean systematic solution to this problem. I still think about it a lot when I do UI programming. In the back of my head, I'm always wondering, "what will happen if the user presses X in the middle of this animation?"

Programmers are particularly prone to these bugs because we have unconsciously trained ourselves to baby our own software. We're careful to wait for transitions to complete and only send input when the app is in a known state.

No good generalized solution for this it seems. The best you can do is to specialize per situation and leverage all the assumptions you can while being cautious not to leverage the assumptions you can’t.

This problem contains lots of incidental complexity and it doesn’t help that folks do the opposite of the above and add a ton of accidental complexity on top. For example, for a situation with a crossfade or morphing set of 2 buttons you should:

- leverage the assumption that there are only 2, and static, buttons. - not assume that there are only 1 button present at any time.

Some end up inclined to create some “reusable” abstraction on top and end up doing the opposite: - generalize to handling n dynamic buttons. Without realizing that some important properties of that specific situation would have been lost. - (usually due to the lack of experience/focus/interest on the problem) oversimplify and assume there’s only 1 button present at any time.

If it was up to me, there were no transitions. I am happy I can disable them on Android... Everything that does or manipulates animation is terrible. Scroll hijackers, image carousels, icon transitions, they are all wasting cpu cycles to please UX/designers!
Sounds right, I've even had a lot of modern games crash or hang when the action gets hot and I'm spamming inputs (Path of Exile was frequent, but even Hades crashed the other day like this).
Couldn't you just stop receiving keypresses when a UI transition begins and then re-enable input once the new UI state has been established?
>While I’m not a QA or security professional, I have developed a knack for doing “stupid” things with software which causes it to malfunction.

A person after my own heart.

I've had many a dev go "why would you do that"

In which I answer "it doesn't matter, but if you accept my input it's your job to ensure the app doesn't crash"

We have an excellent (and big) QA department, but 13 years ago when I started at this company we were only just beginning to hire dedicated testers. We had a mature product which was a communication handset and it worked well and was stable. Our software engineers had pressed every button they could think of in every menu and there weren't any problems.

Then we hired Kevin.

Kevin had the handset for 40 minutes before piping up "crashed it". The lead comes over to have the sequence explained to her, and says "huh, nice edge case". Half an hour later "crashed it again" (in a completely different way). Explains the sequence to the lead again. An hour later this happens again and he explains the sequence and she finally bursts out "Why would you even do that?! How did you think of pressing those buttons like that with that timing?!!".

Good testers just think differently than software engineers.

For a long time I've had my full name as user name on my machine, which meant that my user profile path contains both a space (evil) and a non-ASCII character (even more evil, although it's in Latin 1 at least). A lot of things break on one or both of those things and at times it's a bit annoying to deal with. Some bug reports have also been closed as "Won't fix, just don't do that. Who needs spaces in paths, anyway?". I haven't tried to use non-Latin Unicode in my user name and profile directory, which would break everything that uses the old ANSI APIs on Windows instead of the Unicode ones, and that's probably way too much breakage. But broken nonetheless, and that includes a lot of new, recent, and still-maintained software :-/

In a similar vein, I've also used U+2212 as minus sign in my regional settings. There's a lot of software that refuses to parse numbers it previously happily emitted.

I've given up on that too, by now, though. The only thing I still do is using English as UI language (so I don't have to deal with bad translations of software), but German as my regional settings (with ISO 8601 dates). There's a lot of software out there (I think GNU gettext is broken in that way on Windows) that assumes that the way I want my dates and numbers formatted has any bearing on the language I want to see in an application. Many others don't care about the regional settings and use the UI language to also format dates, times, and numbers. That's annoying, but at least nothing breaks, so that's the only deviation from the standard user I still use, to still be able to work.

Mine too.

I write a lot of javascript and the string "null" is pretty harmless in most code. But there's all sorts of fun bugs (and often security vulnerabilities) you can find if you make an identifier "__proto__". (If code ever uses that as the key in an object, you're off to the races!)

Why do you require software to be more resilient than other things?

If I pour water in the gastank of my car, it will also fail to drive. Or gas in the sprinkler tank. So the car should somehow prevent the enduser putting the wrong thing in the tank?

A favourite technique of one of my colleague's was just to mash the keyboard randomly to see if the app breaks.

It's very crude and not at all foolproof. For the lack of sophistication it's shockingly effective at highlighting a huge amount of assumptions we make about how software is / can be used.

>In which I answer "it doesn't matter, but if you accept my input it's your job to ensure the app doesn't crash"

This is how try/catch alls get added :(

There was and still is a website called The Daily WTF about discussing especially funny bugs and programmer's mistakes. In 2012 I registered on it as a user "undefined" to make a comment about JavaScript oddities under one of their articles and almost forgot about its existence, then they migrated their comments and forums to Discourse and in 2015 I got a bunch of email notifications about people mentioning me as suddenly all "likes" in the forum were linked to my profile:

https://what.thedailywtf.com/topic/17637/undefined-liked-thi...

I worked on an API that regularly got requests from the mobile app for GET /users/(null). I think that's Swift, or Obj-C's way of to-string'ing a null?

I have a generational suffix on my name. I often include it, and quite often as the proper Unicode character, e.g., "Ⅲ". (Assuming HN displays it after I post this, try to select it; that's one character.) That wreaks a fair bit of havoc.

When I was in high-school, I took physics. I was assigned to room, say, 309, to a teacher whose name I didn't recognize. But I knew the teacher in room 309, and she even taught physics. So, I approached her, and asked, "I've been assigned 'Ms. Stewart', but it lists her as being in your room. Do you know what the correct room number is, Ms. Cook?" Right room; it was her maiden name, of course.

In my company's HR system, we have to note some contacts, for things like life insurance payouts. My fiancée is one. Then we transitioned to a new system, and the data from the old system was migrated over. Now she's my "fiancée". (And in a separate system, she's a he, because there was no option for "fiancée", only "fiancé".) Similarly (and a long time ago) I had to fix a contact/directory system when it escaped a '. E.g., it would emit "Marie O\'Conner". PHP magic quotes… shudders

(Character encodings and anything outside of ASCII, in particular, are an unending fountain of bugs.)

Just today, Azure's support system can't handle (among many things) the outlandish characters of "<" or ">". Which is great fun, since it's not like anyone would file a highly-technical support request with Azure… right?

The missing hour in the DST spring-forward and the duplicate one on the fall-back are great hunting grounds for bugs, too. E.g., Google Calendar has issues with them.

We have a git branch prefix at work that triggers a special CI action. Let's call it "branchprefix/". Every now and then a dev will make a branch with "BranchPrefix/" and the OS X machines all start having issues since OS X's file hierarchy isn't case sensitive. (We've also had issues w/ two files, same name different case. git supports it, but OS X can't cope.)

(All the names in this post are changed from their originals, of course. But you get the idea.)

> Just today, Azure's support system can't handle (among many things) the outlandish characters of "<" or ">". Which is great fun, since it's not like anyone would file a highly-technical support request with Azure… right?

Someone in Azure is definitely using the Windows reserved characters for filenames.

> Every now and then a dev will make a branch with "BranchPrefix/" and the OS X machines all start having issues since OS X's file hierarchy isn't case sensitive. (We've also had issues w/ two files, same name different case. git supports it, but OS X can't cope.)

FWIW macOS is perfectly fine with it. The FS (both HFS+ and APFS) can be configured to work in CI or CS modes. The default is CI. Since git uses the FS for part of its data storage, things break.

That’s more of an issue with Git not supporting CI FS, really.

Just as an aside, you can reformat the file system to be case-sensitive on MacOS. I think at $dayjob it's more or less policy to do so.

The only downside I have seen so far is that some software only runs on case-insensitive file systems. For example Photoshop did this last I checked.

I still see \’ appearing on large well trafficked websites, like espn.com and cnn.com.
That’s how C usually prints it. Swift will print “nil”.
'Just today, Azure's support system can't handle (among many things) the outlandish characters of "<" or ">".'

My favorite way of breaking things is to go the other way... oh, you won't allow < or >? Well, how about ＜ and ＞? That's ok then? Great!

One I've done several times is encounter a field that "can't be left empty", and is smart enough to filter out the ASCII whitespace before the check... but isn't smart enough to filter out the Unicode zero-width space. "A computer wizard never says too much or too little, he says precisely what he means to."

> When I was in high-school, I took physics. I was assigned to room, say, 309, to a teacher whose name I didn't recognize. But I knew the teacher in room 309, and she even taught physics. So, I approached her, and asked, "I've been assigned 'Ms. Stewart', but it lists her as being in your room. Do you know what the correct room number is, Ms. Cook?" Right room; it was her maiden name, of course.

I'm reasonably certain one of those two 'Ms.' instances should be 'Mrs.'.

When I lived in the US I was amazed how many systems couldn't handle my (English) surname, which has a dash in it.
People like that is the reason why this list was created

https://github.com/minimaxir/big-list-of-naughty-strings/blo...

My personal favorite is this one though

  "If you're reading this, you've been in a coma for almost 20 years now. We're trying a new technique. We don't know where this message will end up in your dream, but we hope it works. Please wake up, we miss you.",
what a great github repo.

I enjoyed:

    #   Strings that may occur on IRC clients that make security products freak out
    DCC SEND STARTKEYLOGGER 0 0 0
and everything under:

    # Innocuous strings which may be blocked by profanity filters (https://en.wikipedia.org/wiki/Scunthorpe_problem)
Reminds me of the first time my DnD group tried out roll20.net. The chat box allows players to type things like "/roll 1d6" or "/roll 2d12" to simulate rolling dice (in these cases 1 6-sided die and 2 12-sided dice). I quickly tried "/roll 1dNaN", crashed the chat, and we went back to physical dice for the rest of the session.
Perl 5 has a taint mode built into the language. If enabled, it forces the developer to untaint every bit of user-controllable data (by running it through a pattern match) before doing anything dangerous with it. I can't believe that this isn't a standard feature in all languages.
Popey, I think I see you around these parts from time to time. If you're reading this:

You and Martin Wimpress are constant sources of inspiration for me and many others, who want to keep on discovering the world of FOSS software. Thanks for the many hours of entertainment in your podcasts and the help you provide to people on the forums and mailing lists. Excellent work!

Kids, a story from the Old Days, c.1981.

DRI (since absorbed into McGraw Hill) had EPS, an advanced economic/financial analysis scripting language, provided via timesharing (mainframes on the East Coast of the USA). I was a customer support programmer in San Francisco the day that they rolled out a powerful arrays feature on the testing mainframe (no clients, but lots of real work going on).

One could put anything as an element inside an array. So I tried:

    X=array(123, "abc")
    Y=Array(X)
and it worked. You know where this is going, right?

    i=loop from 1 to 1000
    x(i+1) = array (xi)
It crashed the mainframe at i=67, if memory serves.

So far, so good, excusable as "clever programmer tests the limits". And then I ran it again.

Same result, plus, 2 minutes later, a call for me from my friend Kevin, who was a lead developer on EPS in DRI HQ: "Chris, what the ^&^&^!@@ are you doing?"

I once named a fat32 USB pendrive ЯBK

This data must have corrupted some firmware section or so because the drive was gone afterwards.

Couldn't format, couldn't dd, anything.

Fits the category, I think. Only less funny :( Well, depends on the observer :)

Reminds me of a QA buddy. One day at the crosswalk, he decided to, I believe, hold the button. For the whole wait. He apparently broke the entire intersection’s lights and a repair crew came out. He was unable to reproduce it after.
Reminds me of https://www.wired.com/2015/11/null/

Numerous cases of encoding out of band data as a special case of in-band data.

You should try using the direction changing unicode code points like 0x202E in your name. That will probably break many things.
I found something like this when managed to accidentally break the Drupal.org git parser by adding emojis to a commit message. It wasn't on purpose, I was just on a 2015 emoji kick.

That said, it did uncover a bug that obviously hadn't been tested for which gave the infra team more impetus to solve utf8mb4 support for the database.

https://www.drupal.org/project/infrastructure/issues/2531884 https://github.com/govCMS/govCMS7/commit/ab5da5fd0cb3d7e1d33...

I laughed out loud when he said he held down the print screen key until it started repeating. That’s exactly the kind of thing a user would do but a developer would never think of!
That’s a fun article!

I’m big on Quality. Comes from 27 years, working for a corporation that is pretty much synonymous with the word.

“Abuse testing” is very important, and almost impossible to automate. A good monkey tester will have a “sense” of where to go, as this chap indicates.

I worked with an enormous team of people like this, and they would regularly find things like sync bugs (he talks about one). Those take a lot of work (and RSI risk) to find.

Somebody should write a book like the original "Programming Pearls" for weird hacks and anecdotes like the author's. Pretty fun stuff.
> A year or so ago, at a company sprint I gave a lightning talk in which I wanted to make the tiniest possible snap

What is a snap in this context?

> When snaps are uploaded, there are security and sanity checks which run against the snap. My use of the (probably reserved) word null seemed to fool the backend checks script, live on stage, in front of my peers. That’s the way to end a lightning talk, I think!

The backend crashes and instead of getting an error message you are forced to watch a spinner forever?

If you want to find a bug in your software, make a live demonstration.
I experienced the Thunderbird bug mentioned in the article first-hand and freaked out for a moment. "Where does that damn turtle come from?" And had to search quite a bit until I recognized that it was part of the subject. Unfortunately, I no longer have a screenshot of it.
If these kinds of errors occur there might very well be an SQL injection going on.
Declarative programming is something that helps deal with weird edge cases like this right? I'm learning Elixir currently and the subtle semantics around the 'traditional' assignment operator (=) are quite cool. It means you can ditch most if blocks and provide a list of pattern-matchable functions (matched on their arity) to define logic which helps deal with edge cases a little better.

I'm curious how such a declarative paradigm _may_ help with the wacky usage of software old mate Mr. Null endeavours in. No one paradigm solves all problems I feel but perhaps some allow us to harvest some low hanging fruit for free?

Nice to hear about issues that got actually fixed. I tend to find some edge case issues regularly too, but usually most of bug reports end up in limbo.
I understand Gary Null has had some experiences.
Weird hacks, love it!
I know finding bugs is undeniably a good thing but I can't help but feel someone as obviously bright as this should be making more things

It's like the people who spent a lot of their time finding ever more pedantic inaccuracies and continuity errors in films.

The mute LED on your thinkpad sometimes goes out of sync? fascinating

Breaking things is far easier than making things. While this type of poking around might feel fun, it will mostly result in low value work to fix something very few (if any) actual users would experience.