back
38 comments
IG co-founder here: users 1 and 2 were our first two attempts at creating users end to end when getting Instagram v0.1 hooked up to the backend we'd written. There was a bug so they were left in an incomplete state; post bugfix, 3 was my co-founder Kevin and 4 is me.
Two ghost users, left in an incomplete state by a bug in a previous version of the codebase... you basically created the Twins from The Matrix!
"java.lang.NullPointerException"

That is probably responsible for more of the lost sleep in my life than any other single entity.

Various versions of "out of file descriptors" might be a close second.

Null pointer exceptions are typically avoidable though, running out of file descriptors (at an OS level) is really difficult to predict or deal with. Same as running out of socket handles or pipes. I've hit quite a bit of both doing HPC admin/setup.
"running out of file descriptors (at an OS level) is really difficult to predict or deal with"

Hard to predict I agree with. My experience, though, is that some ops person usually either overprovisions, or throttles, and that particular instance of "out of descriptors/sockets" goes quiet for a long time. Basically, people other than the dev team (who have conflicting priorities), can mitigate it.

Poorly written Java (like null pointer exceptions) otoh, is hard to mitigate from the outside. So it endures.

To be fair, all new serious languages now don't have null pointers.

... but we'll still struggle with java.lang.NullPointerException in 2080.

A null pointer is just a manifestation of a bug, usually unexpectedly missing data.

If you can't have nulls, you might have a better chance of spotting the possibility of that missing data and trying to deal with it, granted, but if it happens truly unexpectedly the bug is still going to manifest itself somehow, as an NPE or otherwise.

Following all the way through, let's say there is no exception anywhere and the missing data gets all the way through to the UI.. well then that's how the bug manifests.

Is the missing data just not appearing better than a 'oops, something went wrong' error or similar in response to a 500 caused by an NPE? Depends on the application and circumstances I guess, but in general probably not. The bug exists in both cases. You can present it in better and worse ways, but the bug is that the data is not there and it should be.

The point being, not having NPE doesn't fix any problems, fundamentally. It just shifts them elsewhere. No language feature saves you from bugs.

> The point being, not having NPE doesn't fix any problems, fundamentally. It just shifts them elsewhere. No language feature saves you from bugs.

NPE is the problem. The point of type systems with non-nullable references is that they force you to handle the null case explicitly using an Option/Maybe type by throwing a compiler error instead of an exception in production.

Nobody expects that getting rid of NPEs would get rid of bugs. It would, however, force one of the most common classes of bugs to be handled when it is cheapest to do so, rather than when the production system is on fire.

Respectfully I think you're not addressing the point I'm making, but a slightly different one.

In the case where a piece of data can be missing, and that's fine, I completely agree that optional/maybe types are useful to model that explicitly and unambiguously. They let you and future maintainters know that this is the case, via the type system, at compile time.

But what I'm talking about here is genuine runtime exceptions (which NPE is in Java, of course), where the design of the system says that this piece of data cannot be missing and yet, at runtime, you encounter a case where it is missing, erroneously, because of some bug somewhere.

Types like optionals won't help you here, because you wouldn't have used them at design time anyway. The same applies to languages completely without null references.

All type systems do is formalise expectations about how data is going to look. If the real data breaks from those expectations because of some bug at runtime, the type system can't help you.

> But what I'm talking about here is genuine runtime exceptions (which NPE is in Java, of course), where the design of the system says that this piece of data cannot be missing and yet, at runtime, you encounter a case where it is missing, erroneously, because of some bug somewhere.

My experience is that this is the less common of the cases in a production environment (vs dev where one may be experimenting a bit more).

I agree with you :-)
> But what I'm talking about here is genuine runtime exceptions (which NPE is in Java, of course), where the design of the system says that this piece of data cannot be missing and yet, at runtime, you encounter a case where it is missing, erroneously, because of some bug somewhere.

With checked optionals, you'll know that the code differs from the design at compile time. If a piece of data can indeed be missing you can't compile the code without considering that case explicitly.

Most modern languages have an explicit option type, which indicates that a variable may not have a value.

Java (and many other languages) allows all reference types to be null, meaning you have little compiler assistance in helping prevent NPEs.

See my response to the sibling - my point is that optional types are useful as a design tool for writing more robust code that deals with expected missing data as per the design, but don't help at all with actual bugs that cause unexpected missing data at runtime.
Yes and no. Consider that there are functions where returning "no value" (null in our case) is a valid option. It is possible that the callers of those methods forget to check for a null response; leading to a NPE.

If the language differentiates between fields/results that can be null and those that cannot, and makes the caller specifically handle the null case, then that cause of NPE will longer happen.

I politely suggest you re-read my comments, as I have indeed considered that and it's exactly what I'm saying :-) I agree with what you're saying 100%, I perhaps haven't written clearly enough because I think you're thinking I'm arguing against optionals being useful. I'm not, I'm just saying they're not a solution to the bugs that cause 'real' NPEs - only design bugs where the code doesn't model the data correctly.
> Is the missing data just not appearing better than a 'oops, something went wrong' error or similar in response to a 500 caused by an NPE?

No, but with checked unions like optionals, "not appearing" is a behavior that you explicitly chose to implement. Checked optionals force feed every opportunity to address and amend the error to the programmer. Null pointer exceptions bubble that far up either because the programmers didn't consider all the failure modes caused by implicitly nullable types or because they truly don't care. With checked exceptions, you have limited this to just the "truly doesn't care" category because the programmer is forced to consider every failure mode at every point in the call chain: "not appearing" in that case doesn't happen by mistake but by explicit choice.

While I'd generally agree it seems like a silly point to argue in this specific case - of course missing data in the UI is much preferable to the entire app becoming unusable!

Funny coincidence, by the way. I haven't been on Instagram this past month because the app instantly crashes on launch. On iOS though, and I doubt I've got anyone after me.

Sure, you're right. It almost always would be. That wasn't the best chosen example for what I was trying to get across!
So is Go not a “new serious language”? You can definitely have nil pointers panic in Go if written poorly.
Go having nil for various types is indeed a problem.

In fact, a few weeks ago I caught an issue where a map in Go was unexpectedly nil when it should have been full of data (missing initializer), and I caught it because the serialized data was then consumed by the client written in Swift which doesn’t have this issue. Go thought it was writing data to spec but the client immediately diagnosed it as having an unexpected null before it even tried to work with the data.

Go is even worse. They just use the word "nil" instead of "null". Go does a lot of really silly things too, like the nil pointer receiver's (basically something that allow you dereference null in certain contexts) ends up having terrible side-effects on the type theory and you end up with really stupid exceptions, like nil represents an empty slice and you can do stuff like get it's size or append to it, but the same operations would cause a panic with a map.

But both Java and Go are serious languages.

There are certainly people that feel like it was a mistake. https://getstream.io/blog/fixing-the-billion-dollar-mistake-...
There are only two kinds of languages: the ones people complain about and the ones nobody uses.
And serious languages are those that don't have null pointers?
The original 8-bit BASICs didn't have null pointers. They were _serious_ languages!
Original (non-Medium) post: https://www.valbrux.it/blog/2019/09/13/how-two-dead-users-al...
At my work there were some drivers who cloned the app and were using (a presumably hacked version of) it on another tablet with the "test user" ID to make a whole bunch of money, they could see where the trips were going and would only take the really good ones. Bare minimum we're probably talking at least an extra $1k/week.

Who knows how long they got away with this before someone noticed "ghost tablets" logged into the system and locked down the test user account -- which is also how they got caught because they then had to log in with their actual ID and The Powers That Be could pinpoint who exactly was doing it.

So, yeah, lock down invalid user account IDs.

So any account could do this? It doesn't sound like extra IDs were the real problem.

And there's something deeply sad about "they prioritized the better jobs, so we blacklisted them".

More like debugging than security researching? Wonder how much he got! Congratulations
Was about to comment the same thing. Just an average bug, not a security issue.
I would like to highligth that after the exploitation the instagram application could not be successfully opened until the affected users were removed from the group
Sounds like it’s security - DoS
"manually removed"

Which probably means Instagram admin intervention.

in what way is denial of service not a security issue? It may not be omg-sql-injection, or myspace worm-worthy.. but it's availability is the "a" in the C-I-A security triad.
Any speculation on what a bounty like this is worth?
Depends on their budget. Wild guess but 5-10k seems reasonable.