▲ 126 points
back
207 comments
I feel like floating point is a completely separate branch of programming that a lot of coders never use. When they do it's often a mistake, like for currency. Yet floating point is very useful for scientific calculations and simulations, which is what computers were all about for the first couple of decades.
I have a suspicion that on a fundamental level, floating point isn't actually good for games or machine learning. They're just used because existing computers are so good at floating point number crunching, especially GPUs.
> When they do it's often a mistake, like for currency.
The most beautiful FP bug I remember was a denial of service in some webservers where by setting the header "Accepted Language: en-gb;q=0.3 en;q=0.8 ..." to a specific value you could send the official Java floating-point parser in an infinite loop (and this affected several Java webservers).
So at each webpage request, you were sending one CPU core of the webserver basically busy-looping. Hence crashing the webserver in a few requests at most.
Now it'd be heresy if I were to say that maybe, just maybe, had the standards mandated to use 0 to 100 for weighting instead of 0 to 1.0 we could have dodged a whole source of potential issues!?
No, no, I realize this is heresy: let's all keep using numbers that cannot even be represented correctly (except as strings), parse those strings into something approximately representing what's written in the string, then make more approximation errors while doing computation with these numbers, propagation errors, epsilon estimation errors, and keep insisting we all could have authored "What every computer scientist should known about floating-point numbers" (which is 80 pages long and, well, approaching a treaty) ; )
/rant off
As someone who has started out making games on Playstation 1 that didn't have floating point hardware, I can with authority say that developing games with only integers suck. Floating point number are far easier and robust to use.
Floats are essential for machine learning, scientific computing, and any other application where the goal is to model mathematical processes involving real numbers. The idea that there is something out there that could do the job better than floats is almost certainly wrong.
I'm fascinated by how common it is for programmers to hate floats. Yes, if you write business software for a living you may not have much use for them. But there's a lot more to computing than business software.
One example where floating point does not match the problem field is GIS. Coordinates are bound by the earth's circumference. Fixed point would make much more sense, but neither hardware nor software support is there.
Lots of the math in games and ML assumes you’re operating in the reals. For example, the optimization in neural networks. Integers and fixed point won’t work.
In javascript, every number is technically floating point. :)
What better alternative is there for handling currencies? Multiplying the original amount by 100 and casting to an int? ($3.50 -> 350)?
It is the opposite.
From Microsoft BASIC to Javascript, many programmers have worked in languages that only have floats. Financial calculations involve exponential/log math much like scientific problems. There are issues with rounding and representation there (e.g. there is no such thing as $0.01 in floating point, only $0.50, $0.25, $0.125 and some sum of 1/2 fractions that comes very close to $0.01 and even reads in and writes out as $0.01 even though it isn't.)
> very useful for scientific calculations and simulations
Don't forget control systems.
In some game engines, Fixed Point Arithmetic is used instead of Floating Point, because it's faster, and the approximation is generally good enough for a game.
I recall there being an issue with double.GetHashCode() in the early versions of .Net, where it would return different hash values for 0.0 and -0.0.
That got fixed, but it seems a variation with signed NaN recently got fixed[1] in .Net Core (taking care to handle all the possible NaN values).
Floats are deceptively easy to use. Which is nice but kinda sucks too, given all the footguns they bring to the party.
You inspired me to check java world.
// 0.0d / 0 equals 0x7ff8000000000000 (NaN)
// Math.sqrt(-1) equals 0xfff8000000000000 (NaN)
// 0x0p+0d is a funky way to specify 0x0000000000000000 (0)
// -0x0p+0d is a funky way to specify 0x8000000000000000 (-0)
// without 0xHEXp+NUMd my compiller optimizes "-0" literal to 0
// 0 == -0
0x0p+0d == -0x0p+0d
// hashCodes for 0 and -0 are different
Double.hashCode(0x0p+0d) != Double.hashCode(-0x0p+0d)
// hashCodes for different NaNs collapse to the same value
Double.hashCode(0.0d / 0) == Double.hashCode(Math.sqrt(-1))> I recall there being an issue with double.GetHashCode() in the early versions of .Net, where it would return different hash values for 0.0 and -0.0.
I never thought about that (I avoid FP as much as I can) but, oh boy, the can of worms!
.Net (or Java) OOP where an object's hashcode needs to give the same (or not?) value for 0.0 or -0.0: this is the kind of stuff nightmares are made off.
I'm sure this can be turned into some very funny "Java puzzler".
The fact that NaN is technically a set of values in IEEE-754 (rather than just one agreed-upon value) was quite eye-opening to me.
What is wrong in having different hashes for different numbers?
Does anyone have a good example of a programming problem where we _need_ signed zero? Or where it makes things significantly simpler? As far as I know, there is no distinction between -0 and +0 in math, so I have never really understood why this is a thing in computers.
When does this need arise? Well, otherwise inverting a value can change it's sign and in particular inverting -∞
twice will give you +∞, and being off by "2∞" is a pretty large error for a lot of computations ;)
You can end up with zeros and infinities pretty easily because you overflow or underflow the range of floating point precision, and generally you want something sensible to happen in typical cases, even if some common arithmetic identities necessarily break down.
I would actually like a true signed zero (or rather "epsilon" value ), so -0 and +0 as distinct from "normal" 0 which is truly unsigned, neither positive nor negative. The former two would only arise from underflow, and the reason this is useful that if you underflow from below zero and invert that you want to get -∞ and if you underflow from above zero and invert that you want to get +∞. Inverting a signless zero should give NaN (instead it gives +∞, which is nonsense in basically any case where the domain is not inherently the non-negative reals already and the 0 did not come about by and underflow; in particular 1/0 should be NaN).
If anyone knows why this design was not chosen and what fundamental downsides it has, I'd love to hear it. Obviously representing three zeros is a tad more annoying, but IEEE754 has a lot of stuff that's annoying implementation wise but was added for nicer numerical behavior (e.g. denormals, and of course various "global" rounding modes etc. which probably qualify as a mistake in retrospect).
Latitude / Longitude coordinate as floating point.
https://gis.stackexchange.com/questions/211796/if-degrees-is...
I made the above post due to a bug in the GPS on our routers. We had a customer close to the meridian in England.
http://www.thegreenwichmeridian.org/tgm/articles.php?article...
Western hemisphere is negative longitude, eastern hemisphere is positive. If your degrees are 0 (within ~100km? I can't remember exactly), then you still need to know if you're east or west of the meridian.
Computers (obviously) have to approximate the majority of actual mathematical numbers, as they do not have infinite storage.
If you've got two numbers, +0.0000001 and -0.0000001, but you can't represent that precision, can you see how it's less bad to round to +0.0000 and -0.0000 rather than to just 0.0000? It's encoding strictly more information.
Negative zero was introduced because of branch cuts: https://people.freebsd.org/~das/kahan86branch.pdf
I agree with many of the other replies, but I would also add, perfectly serious and with no sarcasm, do not be fooled; computers do not use "math" numbers. They use what they use; machine-word bounded integers and IEEE floats most commonly, unbounded integers, some variants on rational numbers, and sometimes some more exotic things, but they never use real numbers. They're bounded to the computable numbers.
So whether or not there's a -0 or +0 in some "math" isn't relevant, because computers aren't using "some math" but very specific mathematical constructs that must be understood on their own terms. I haven't seen very many mathematical systems that have a reified "NaN" object that can be explicitly passed around as a valid value to functions. (I know of many systems that have a "bottom" but I would say bottom is usually presented not as an object you can "have" but as a property of some mathematical object. Haskell for instance uses this idea; there are some trivial ways to have or produce something that has the property of being "bottom", like calling "error", but you can't just "have the bottom value".)
Moreover, there are mathematical systems in which such things can appear. There are many exotic and interesting number systems in the mathematical world, which even a bachelor's degree in mathematics may only scratch the surface of, depending on which junior/senior level courses you take. Real numbers are without a doubt the most studied, and they do not contain a -0 (though even then you'll still see it show up sometimes as a special notation in limits), but they are the beginning of number systems, not the end.
I mention all this because it's important; it's a very common misconception that computers use the numbers like you learned in school, and it will lead to nothing but pain. The next misconception is that, ok, sure, they aren't those numbers exactly, but they're so close that I don't have to worry about it. This works as long as you don't push your floats too hard, and a lot of us don't, but also breaks down surprisingly quickly. It's important for programming professionals to understand in general that IEEE floats are their own thing. Whenever I use them I do at least take a couple of seconds to double-check mentally that the sharp pointy bits aren't going to stab me, even for simple things like adding durations of a request to some floating point accumulator for metric purposes.
Floating point math was devised to be used in numerical methods that are applied iteratively, like gradient descent. In that world what you are always doing is "converging" to solutions step by step, and a negative or positive zero can tell you which direction you're converging from.
It’s used when writing SIMD code all the time to quickly mask bits as needed or to check the sign of a floating point number with an SSE intrinsic. _mm_xor_ps(_mm_set1_ps(-0.f), reg) as an example negates all four components of reg.
Sign/magnitude ADCs almost have signed zero in hardware, by generating a sign bit and zero or more magnitude bits. The typical method to map to voltages doubles the span between adjacent values and skips over zero. So a 2-bit sign/magnitude ADC measures values in the series {-3, -1, 1, 3}.
So strictly speaking this number system doesn't have a representation of zero at all. Tiny measurements either round up or round down to +/- 1.
Amusingly enough the Apple Watch reports temperature as 0° and sometimes as -0°, I've not determined what causes that, perhaps rounding around 0?
It’s pretty common when, for example, you’re computing the flows in a bifurcating field.
Take 2d laminar flow around a circle. The flow field splits right in the middle.
Signed zeros ensure that, along with graceful underflow, the local solution is not wonky.
Lots of complex-arithmetic examples too.
0 does double duty as meaning itself and to indicate underflow due to multiplying very small numbers.
If you have a function that can experience underflow it can be useful to preserve the sign of the underflowing value.
Otherwise you'd need more checks to get that information.
> Does anyone have a good example of a programming problem where we _need_ signed zero? Or where it makes things significantly simpler?
Maybe when you're implementing floating point numbers in a way that's simple and widely applicable?
I'm only half joking, too, though I can't tell you what exactly having a distinct sign bit simplifies.
I can say off the bat that it is probably useful that a very small quantity that might otherwise lose enough precision to round to zero maintains its sign regardless.
Exactly for things like OP’s argument: 1/-0 is -inf, and that may be important in asymptotics
A distinction can be made when considering the limit of a sequence. If a sequence converges to zero from positive values (sometimes written -> 0+), then the inverse of the sequence will diverge to +inf, while if the sequence converges to zero from negative values (-> 0-), the inverse will diverge to -inf.
As a sibling comment wrote, rounding numbers to -0 and +0 can provide extra information, though it may not be useful in all contexts.
PHP just refuses to divide by zero:
php > $minus_zero = -0.0;
php > $plus_zero = +0.0;
php > var_dump(1.0 / $minus_zero);
PHP Warning: Uncaught DivisionByZeroError: Division by zero in php shell code:1Actually as shown by your output it doesn't completely refuse to divide by zero, a warning means that execution continued.
The result of that division is a float with a value of -INF, INF being a constant that PHP treats as infinite.
But as of PHP 8 dividing by zero causes a fatal error and execution is halted.
This site is really good for comparing results in different PHP versions: https://3v4l.org/9Tl1I
I actually wasn't aware that PHP had an INF contant but seeing the warning in your output prompted me to dig a little bit deeper :)
Interestingly Go will give you a negative 0 if you use strconv.ParseFloat("-0.0", 64) but I can't see any way to get -0.0 from a number literal.
Python Produces the following:
minus_zero = -0.0
plus_zero = +0.0
parsed = float("-0.0")
print(1/minus_zero)
print(1/plus_zero)
print(1/parsed)
ZeroDivisionError Traceback (most recent call last)
<ipython-input-1-747f1b708c86> in <module>()
2 plus_zero = +0.0
3 parsed = float("-0.0")
----> 4 print(1/minus_zero)
5 print(1/plus_zero)
6 print(1/parsed)
ZeroDivisionError: float division by zero
As an aside, historically many of the early binary computers used ones complement. So they had both -0 and +0 integers as well. While ones complement is probably more obvious than twos complement, it was near-universally considered a terrible mistake by the mid-1960s, and everything of note since has been twos complement.
It didn't die quickly though. The UNIVAC is still with us (in emulation) and that is likely the reason why the C standard addresses the question of negative zero integers. (Their handling is implementation specific, of course.)
Ruby seems to handle this correctly:
>> 1 / -0.0
=> -Infinity
The inverse of zero is infinity?
Shouldn't it be undefined?
There are still (a few) computers in the world (Sperry Univac legacy support) using one's complement arithmetic, hence having a positive and negative zero in the architecture.
the Erlang VM said "screw IEEE":
iex(1)> -0.0 === 0.0
true
This has, however, recently come up for Nx (numerical elixir) which had to implement and standardize ways to shim these IEEE concepts back into the VM for interop purposes.
It's also worth understanding whether +0.0 and -0.0 are equivalent in your environment:
* Do they compare equal?
* If they're convertible to boolean, do they both evaluate to false?
* If you serialize/deserialize the value (e.g., to JSON and back), does it maintain the distinction?
* If you send this value to a database (esp. one with stored procedures), how does its behavior compare to your language?
Does -0.0 == +0.0 ?
Common Lisp handles negative zero quite well.
the way c does because it is written in c++ using scanf lulz
I was just thinking about this 2 days ago, the coincidence!
Elixir does not distinguish signs in zero, 0.
Therefore; -0.0 + 2.0 = 2.0 and 0.0 + 2.0 = 2.0
Signed infinity is nowhere near the 'expected result'. Zero has no sign. NAN would be a better result off the top of my head.
But I see from the standard that dividing by zero returns infinity with the xor of the dividend and divisor. This also makes no sense - since zero has no sign, its not possible to tell what kind of infinity would result. It's strange that the standard permits both the infinity result from dividing by zero, and simultaneously supports signed zero.
<edit: add xor>