back
22 comments
As a teen I had this idea to make a polyglot server that makes the same data accessible over different protocols, and I tried to be creative. I had FTP working fully, HTTP mostly, IMAP to an extend, and a very rough IRC prototype before I lost interest. This was vb6 in the late 90s iirc, so no protocol libraries and a rather limited language when it comes to parsing and string handling.

All these protocols were made to give you a view on a simple directory tree with files in it. FTP was obvious as that's just what the protocol is for. For HTTP it felt a bit like cheating since I had to create directory listings in HTML which didn't feel pure. Also it only had read access. Same for IMAP. It worked well as you could map directories onto folders and the file content was the mail body. IRC was where it got weird; with channels you only got one level of directories, the files were users on those channels that you could say something to on the channel, and they'd DCC you the file contents. It crashed some clients though when the filenames were too long...

Some database protocol would've been interesting too I guess.

Also, the server listened on one port only and looked at the incoming data to determine the protocol, which I also thought back then was pretty 1337.

> For HTTP it felt a bit like cheating since I had to create directory listings in HTML which didn't feel pure. Also it only had read access.

There’s also WebDAV, which may be interesting to implement.

Although, I am not aware of any actually great WebDAV clients.

Only time I ever used WebDAV was a decade ago on my first smartphone, when I set up a WebDAV server on my computer and used a WebDAV client app on the phone to transfer files between the phone and the computer over WiFi.

WebDAV is one of those things where because I personally haven’t used it for so long it could be something that “no one” uses anymore in present day, or it could be that unbeknownst to me it’s hugely popular.

A long time ago I used the built-in Windows support for WebDAV and it seemed to work fine. I think Microsoft Sharepoint uses WebDAV. I think nextcloud supports WebDAV although I haven't used that in years either
> For HTTP it felt a bit like cheating since I had to create directory listings in HTML which didn't feel pure.

FTP’s directory listings are essentially ASCII art, which is IMO just as bad, except that the conventions involved were developed enough that machine parsing them mostly worked.

At least HTML has the a tag, which is genuinely machine parseable.

WebDAV is still used for cloud storage (e.g. https://www.transip.nl/stack/). You can do offsite backups this way (if you’re not using rsync- or S3-compatible storage).

CalDAV and CardDAV (which are based on WebDAV) are of course also still around and used to sync contacts and calendars.

I encourage anyone to study these protocols, they are sometimes a lot more prevalent than one might think. Take the Postgres wire protocol, it's supported by a number of databases out there.

Some years I built a data grepping tool, but didn't want to build a UI around it, so I started looking into the Postgres protocol, so that I could plug in existing tools. Getting the basics right was not that difficult, the docs are pretty clear. https://www.postgresql.org/docs/current/protocol.html

Thanks for mentioning Postgres! I am actually pretty curious about the difference between MySQL and Postgres when it comes to their docs about the protocol. My superficial impression is that the Postgres docs are more thorough, leaving less room for ambiguity (some pages of the MySQL docs seem sloppy in comparison, e.g. those documenting the text encoding of values). I would be interested to read an analysis of the development culture of both projects, based on their docs.
> I wonder why number 36 was forbidden

The most likely reason is because MySQL used Modular Crypt Format. From https://man.freebsd.org/crypt(3) "The salt must be terminated with the end of the string character (NUL) or a dollar sign".

It is stuff like this that answers the question: What could possibly go wrong?

Great catch (for the author) and thanks for the explanation (for the parent).

One gotcha in modular crypt format is that the format is different between glibc and BSDs. Glibc documents that the salt should contain only the same characters that are used in the crypt(3) output (ie. [a-zA-Z0-9./], note that this is different from the two most commonly used base64 alphabets) and recent versions of glibc crypt(3) check for this and will return an error (actually "*0" and not NULL with set errno) when the salt contains invalid characters.
This is a writeup about one of the most interesting contracts I've done. The resulting implementation is (unfortunately) not open source, but I'm hoping it will be eventually!
> I kept technical notes about the protocol, to serve as a future reference for myself and for other developers.

As someone who's written a MySQL client (in C#: https://github.com/mysql-net/MySqlConnector), I'd be very interested to see your notes. Are they available anywhere, or were they the confidential IP of your client?

> Unsurprisingly, bugs were uncovered every time a new client was added.

I've faced the same thing, from a client perspective. Every time a new server is tested (MySQL, MariaDB, Amazon Aurora, Azure Database for MySQL, etc.), I find slightly different interpretations of the protocol and have to accommodate them.

> I'd be very interested to see your notes. Are they available anywhere, or were they the confidential IP of your client?

Hmm... Not sure about that. One part of my notes is embedded in comments to the code, another part exists as a standalone markdown file providing a high-level overview. I'd rather be overly cautious for the time being, but since they are part of the repository they might become available if my client ends up open sourcing the library.

If you have any specific questions feel free to shoot me an email (contact details in my profile)

This reminds me of a forgotten project I did for a previous employer. We had a lot of MySQL databases and wanted to be able to run test environments that were much lighter than production. I implemented in Typescript the minimum of the MySQL network protocol so that it could be called from our applications, using an SQLite database as the real data store. It worked pretty well for our purposes, and it was tons of fun. Alas, I don't have access to that codebase anymore.
https://github.com/dolthub/go-mysql-server

One item under "Scope of this project":

Provide a runnable server speaking the MySQL wire protocol, connected to data sources of your choice.

Careful, I remember the time when MySQL claimed any independent implementation of their protocol was a derivative work and must be licensed under the GPL...
Interesting, I wonder how they could make a case for that. Do you have any references? I have heard of developers independently reimplementing GPLed code, being careful not to read a single line of the original code to avoid any chance of their work being seen as a derivative work (this is quite similar to what I did in this case btw).
It used to say this on the MySQL web site. IIRC this is back in the 3.x days so quite some time ago now.

At the time it seemed like a ridiculous claim that no one took seriously. In the days of the 'structure, sequence and organization' of an API being copyrightable, I'm no longer so sure...

Thanks! Hopefully we are safe then :)
Sounds like the Bitkeeper drama that lead to Linus creating Git
The article doesn't mention the use of JNI, was it that difficult to apply in this case? Google seems to enjoy it on Android making new native libs in Rust and then exposing JNI interfaces.
It doesn't mention the JNI because I didn't use it (the Java implementation was from scratch and did not include native code)