Formatting floating point numbers

I just wanted to say I appreciate the imagery preceeding you paper, as I think it captures the struggle of FPP conversion perfectly. As my office's unofficial, "FPP expert" I sympathize the struggle. You've done an excellent job crafting a paper that not only captures your work, but you've managed to articulate the FPP problem so accessibly. Anybody that cares to consider your paper can gain insight into the true nature of the dragon.

For those who are interested in the paper:

https://dl.acm.org/citation.cfm?id=3192369

"Ryū: fast float-to-string conversion", Ulf Adams 2018

"Section 5 reviews the existing literature. All of the early ap- proaches require arbitrary precision arithmetic in the general case. More recent developments work with xed precision arithmetic, which is faster, but can increase the complexity of the code. By contrast, Ryu ̄ is simple and fast."

Also interesting:

"We did not compare our implementation against the C standard library function printf, as its specification does not include the correctness criteria set forth by Steele and White [15], and, accordingly, neither the glibc nor the MacOS implementation does."

My understanding is that fixed-length conversion to string is more or less trivial, the tricky part is writing a fast algorithm to compute the shortest round-trippable conversion to string, correct?

What about libfmt? Frankly I find it a little easier to use than double-conversion and it's equal in speed.

MSVC has the std charconv stuff for this, and the other implementors should soon enough too. That means sooner or later all C++ compilers will probably have Ryu soon

It also creates a lot of misconceptions of how accurate floating point math is.

An interesting aspect of decimal formatting is how it frequently masks representation error (i.e encoding of 0.1 etc), because the error is symmetrical in the formatter/parser it makes such non-representable fractions appear to be stored perfectly to unsuspecting users.

This can be quite deceptive, if more users were aware of just how many of the simple rational decimals they input were converted into imprecise representations they probably wouldn't trust computers as much as they do. To confuse things more, when operating upon periodic representations the result often matches the representation error of the the equivalent accurate decimal value encoded directly (i.e there were errors, but everything canceled out through formatting) - when they occasionally do not (e.g 0.1 + 0.2) it makes the problem appear all the more elusive.

I think this detail is often lost in explanations of 0.1 + 0.2, that is: representation error is extremely common, 0.1 + 0.2 is merely one of the cases where it both persists through the formatter AND you notice it because the inputs were short decimals, and it's so obvious that the output should be a non-periodic decimal..

TL;DR formatting floats to decimals makes us trust floating point math far more than we should - it's healthy to remember that the formatting process is necessarily imprecise and that you are merely looking at a proxy for the underlying value. Remember that next time you look at _seemingly_ non-periodic decimal output.

I'm also not convinced they help human readability that much.

e.g., when I've done my own binary formats, I typically also create a "dump" utility that converts the file to a readable text dump. I find the ergonomics of this are just fine for my purposes.

  less somefile.json

isn't that much easier to type than

  dumpdat somefile.dat | less

That said, different purposes are different. I'm not so worried about human-readability for short-lived stuff, but, if you're talking about data that may be sitting around for decades, then the human readability question becomes something you might better characterize as "future comprehensibility": If someone's trying to dust off 50-year-old business data from deep in the archives, they're going to have a much higher chance of success if it's CSV files than if it's in some custom binary format whose documentation was lost 40 years ago.

The thing is, "human readable" files aren't, once they get to a nontrivial size. Picking apart a megabyte XML file in a text editor is arguably harder than the same data as a blob in a hex editor.

In fact, thinking about it now, I bet Wireshark would be awesome for that.

“Something in the middle” could be an option. E. g. (in C)

  > printf ("%a\n", M_PI);
  0x1.921fb54442d18p+1

  > printf ("%a\n", 42.0);
  0x1.5p+5

The "obsession" came about from all the problems with binary formats. Byte ordering. Unspecified signedness. Memcpy:ing structs with padding. Programmers micro-optimizing by choosing the smallest possible field size, and then you can't change it when it turns out to be too small. Format specs using WORD as if that's a well-defined size. Using int, long, etc, and assuming they are the same size everywhere.

It's a trade-off of CPU usage versus flexibility. It's not surprising that textual serialization formats took off together with managed memory languages.

>You're burning up a lot of CPU power here

This is easy to write off as micro-optimizing, but it's no joke. I recently profiled some hard real-time software and was surprised to find that >50% of busy processor time was spent somewhere in the fcvt() family of functions.

It's fine for the use case in the medium term (we're within deadline and not starving for more cycles) so I'm avoiding the serialization format transition headache, but it did send me down the rabbit hole of floating point conversion techniques for half a day.

A few years ago I wrote a naive Java string-to-floating point conversion routine that only converted 'simple' numbers, i.e. no scientific notation, only one (fixed) decimal separator, etc. It was the first Java code I had written in 10 years and the approach was probably not very Java-esque (I imitated what I'd do in C++, using (IIRC) java.nio.ByteBuffer or something similar). Anyway this simple implementation was ~10 times faster than the 'officially' suggested way, using formatted string conversion routines, even after asking around (on SO and similar) on how to do high performance string-to-float conversion.

There are probably similar issues in other programming languages, this is not to rag on Java; my point is that when you don't need the 'fancy' floating point formatting stuff, and you need high performance (in my case, I had to convert many billions of floating-points-as-strings and the conversion took ~50% of my program's runtime, which was measured in days), it can pay off to write a custom version of something as seemingly mundane as converting a string to a number.

> that could be addressed with a simple 4 or 8 byte memcpy.

This won't account for endianness.

Indeed, "human readable" really means "in something similar to English" much of the time, and is a vague and quite subjective term; would someone who knows only English think Chinese is not "human readable"?

Personally, I consider binary formats to be just as readable (from the hexdump, or sometimes even the ASCII directly) and in some ways even easier to parse without ambiguity --- it just requires a bit (pun intended) of learning, like any language. I've worked with someone who could "read" TCP/IP; and I can read much of Z80 and some x86 Asm as well as a few other binary formats.

Sometimes I wish one could configure REPLs, for example, to print floats not as the shortest input which maps to that float ("0.1"), but the actual number represented by the float ("0.1000000000000000055511151231257827021181583404541015625") by default.

Somewhat annoying, but might avoid a lot of confusion ("Is floating point math broken?", https://stackoverflow.com/questions/588004/is-floating-point... ).

As someone from a country which uses commas as a decimal separator (Norway)... Just use a dot. If you're not going to make a _serious_ commitment to actually make your software work well in all the various locales, use a dot. People will understand it regardless.

You're probably going to do something stupid like showing the user a comma-separated list of numbers at some point, which will be needlessly hard to parse for a human when your numbers use a comma as a decimal separator. You (or someone else, or your users if they are technical) will probably at some point make something which tries to parse some output, and that will break if you switch between points and commas arbitrarily. Your users will want to copy a number your software prints and paste it into a calculator or REPL or something, and that probably doesn't work with comma as a decimal separator.

Half-assed "localization" from people who don't know anything about how other countries work is just needlessly annoying to be subjected to.

That's at least my perspective as a Norwegian who experiences a lot of _bad_ localization even though I know English fairly well and configure all my computing devices to use English. The perspective of someone from a country where English is less well known might be different.

<rant>

Examples of horrible localization from clueless American companies or organizations include:

* A lot of software will use your IP address to determine your language. That's annoying when I'm in Norway and want my computers to use English, but it's horrible when abroad. No, Google, I don't want French text just because I'm staying in France for a bit.

* Software will translate error messages, but not provide an error code. All information about error messages online is in English on stackoverflow or whatever. If Debian prints an error message in Norwegian, there's absolutely no information about the error anywhere on the web.

* There was a trend for a while where websites would tick the "localization" checkbox by adding a Google Translate widget, so English websites would automatically translate themselves into completely broken Norwegian automatically. That would've been useless if I didn't know English, and it's even worse considering I already know the source language just as well as Norwegian. Luckily, most websites seem to have stopped doing that.

</rant>

Yes.

These algorithms (Grisu, Ryu, ...) all satisfy the "internal identity requirement", which means that they can print a double and read it back in to the same double.

The harder part is to also produce the shortest of all possible string-representations. (And then picking the closest if there are many).

These are two separate problems. This is talking about printing floating point numbers, and does not talk about parsing algorithms at all.

When considering the correctness of a floating point decimal printing algorithm, you could use "round trips with Y parsing algorithm", but that's flawed -- it ties the printing algorithm directly to a particular parsing algorithm.

Instead what is usually considered is "what is the closest representable floating point value to the real value output by the printing algorithm?" If the printing algorithm outputs a string representation which is closer to the input float than it is to any other float, then it's correct. It's up to each parsing algorithm to provide the reverse condition -- that it parses each string to the float which is closest to the real value of the represented string.

Modern float-print algorithms like Grisu / Dragon / etc. also add the additional restriction that they output the shortest possible string that meets that condition; for example, the real number 0.69999998 is closer to the 32-bit float (0b1.011_0011_0011_0011_0011_0011 * 2^-1, represented in memory as 0x3f333333) than any other float. The real number 0.7 is slightly further from that float, but it's still closer to it than any other float, and is much shorter -- those algorithms should print that float as "0.7".

A correct parser should parse both the string "0.7" and "0.69999998" to the same 32-bit float result.

I'm not sure, but generally speaking converting doubles to strings will fail to round trip at least the NaNs

If round tripping is important, my recommendation would be to output something that directly corresponds to the binary representation of the float. For example, printf %a

Ryu is only a year old. Grisu is 10 years old. It takes time for people to absorb these things.