back
85 comments
As a hacker/tinkerer type programmer I can understand why it might be fun to build something like this. All of the data structure fun with none of the distributed difficulty! NoSQL! I can even build in that cool new language I've been designing in my head! But like everyone else I'm straining to see a use case. "Serverless" NoSQL? If you've got something small enough that you want an embedded datastore, why use a NoSQL variant?

I understand that NoSQL literally means "no SQL," but typically it's a buzzword reserved for unconventional datastores used under heavy distribution.

Maybe you'd want a local embedded "NoSQL" to act as a local datacache? But then for that use case why create an impedance mismatch with the central datastore by rolling your own?

Jx9? So wait, I need another language to interact with the thing that's already embedded directly within my program?

Local offline data analysis? Now that makes sense. Does this have intelligent/optimized paging mechanisms? I have a feeling not, since the author seems to be couching this as an alternative to sqlite for people who like NoSQL.

I'd sincerely love to hear more about what's motivating this project. Was it just for yucks, or is there some problem that the author(s) needed to solve that wasn't well-solved by another tool?

> I'm straining to see a use case. "Serverless" NoSQL? If you've got something small enough that you want an embedded datastore, why use a NoSQL variant?

Serverless NoSQL embedded datastore aren't really that unusual. BerkeleyDB, Tokyo Cabinet, LevelDB are all commonly used serverless NoSQL.

The problem isn't the author's lust for tinkering and improving his dev chops. The problem is with HN, how it's designed to drive fads and constantly disturb our focus, and immature developers who change tech stacks as if it was underwear.
"serverless" databases are usually called storage engines: LevelDB, BerkeleyDB, BitCask, InnoDB. There is a (good) trend towards modular software, separating the storage engine from the server is a natural consequence.

But yeah, requiring a new obscure language pretty much obliterates any reason you'd want to use this. Lua is fast, popular, made for this exact use case and already used by Redis.

When I was at Singly working on the locker project, this was actually the absolute perfect solution to how we wanted to store data (it just wasn't available then.) We wanted to store all the raw JSON that was coming down the pipe from various sources (Facebook, Twitter, etc.) and wanted it all to be queryable. Also, since each user's data was completely isolated, having a tiny embeddable solution made a lot more sense vs. firing up a MongoDB or something similar for every user.
There are two different camps within NoSQL; some people are webscale and some people want NoSchema. This looks like it's aimed at the latter category.
I'd have to agree with you here... I mean, there are lots of interfaces for reading/writing json, and simply outputting a JSON as a UTF-8 .json.gz file seems more appropriate... especially if your data can fit into memory.

I would be more inclined to simply have my object structure in memory, using a more convenient high-level language, and do a load/dump from .json.gz files as needed. It's fast enough, and worst case scenario, I can backup/extract and read/write in any number of programming languages.

If you need multiple records, that may be easier, having an index, and using line delimited flat json structures can work as well... (breaking on \n)

This really seems like an also ran, where you could have abstracted out an SQLite db with a single table of (key VARCHAR(100) PRIMARY KEY, value VARCHAR(4000)) or something similar.

The licensing is strange as well. The BSD license is listed as a feature, but the embedded scripting language (Jx9) is SPL (their own GPL-alike/Sleepycat license).

Since the Jx9 language is fully embedded (even stated as " All C source code for UnQLite and Jx9 are combined into a single source file."), they seem to contradicting their own licensing. The resulting library would also retain the copyleft SPL license.

The base code may be BSD, but incorporating the whole database into your code infects it with their SPL too.

From UnQLite core developer: No, Jx9 (the standalone library) is under SPL, while UnQLite is 2-clause BSD (including the Jx9 core), so there is no worry about that since the two software are developed by the same company. In other words, use UnQLite without restrictions
This turned up recently on proggit too.

http://www.reddit.com/r/programming/comments/1etfxi/sqlite_n... was heavily downvoted for saying its an order of magnitude faster than SQLite

http://www.reddit.com/r/programming/comments/1etkix/unqlite_... raises concerns about the license, and there's a Global Lock in there too? The scripting performance gives cause for pause too.

"It furthermore states that commercial licence must be acquired for closed source applications."

I love the description of the scripting language as "Turing complete" based on "JSON". That is the first time I have ever heard turing complete used to market a programming language!

The scripting language is incredible, I have never seen a more verbose way to program. It is a scripting language, yet seems to combine JSON, PHP and C standard library functions with C type casting. Look at the samples: http://unqlite.org/jx9_samples.html

I am speechless, the description of the project is the most vacuous collection of buzzwords imaginable. "Built with a powerful disk storage engine which support O(1) lookup." "Support Terabyte sized databases." Of course with a global lock and after taking a look at the scripting language I have reason to doubt the claims made

Isn't BerkeleyDB the quintessential "embedded NoSQL" database engine? I'm an RDBMS+SQL kind of guy, so can someone enlighten me about the differences between bdb and UnQLite?
Yeah, another alternative with friendlier license is leveldb

https://code.google.com/p/leveldb/

It's also well engineered (written by some of the best Google engineers) and well supported.

Is this a buzzword-laden way to describe a serializable hash table? [I'm asking seriously]
You can also script it!
If you actually need something like this, there's Kyoto Cabinet, BerkleyDB and others that are well known and well tested.

Why use something new with no apparent benefits? (and a lot of drawbacks)

http://en.wikipedia.org/wiki/Berkeley_DB

http://en.wikipedia.org/wiki/Kyoto_Cabinet

Edit: For extra points, read this: http://www.aosabook.org/en/bdb.html

BDB is really slow, and KyotoCabinet is GPL. For some better alternatives, check out LevelDB: https://code.google.com/p/leveldb/ and LMDB: http://symas.com/mdb/

There are some decent benchmarks put out by the mdb guys here: http://symas.com/mdb/microbench/ .

The site seems like a computer generated mashup of keywords and phrases. It's a document store database similar to MongoDB, Redis, CouchDB etc and and key-value store similar to BerkeleyDB, LevelDB, etc.?

> UnQLite is 100% hand-coded, written in ANSI C, Thread-safe, _Full reentrant_ ...

It also uses its own scripting language

> [jx9] uses a clean and familiar syntax similar to C, JavaScript and JSON

The copy on this site leaves me puzzled.

Not that I am envious for the karma, but I wonder why this was not merged into my exactly identical submission 1 hour earlier (https://news.ycombinator.com/item?id=5749969) -- afaict there was no ?repost or similar stunt ...
Being that HN was designed to handle a metric shit ton of traffic on very little hardware, I'd imagine that it has adopted a weak consistency model for these types of things.
From the markup, that submission has a trailing slash while this one does not. This is one of few ways two URLs can differ but cannot ever point at separate resources, so more clever URL handling would have noticed it's a dup.
I am working on a similar project: a persistent database in Node for Node projects, with no external dependencies. That means you can use it with a simple require(), no external software needed.

This is useful for small projects that don't need the power of a behemoth like MongoDB and want to be installable by a simple git-clone + npm install

I am very interested in feedback on it! https://github.com/louischatriot/nedb

What an unfortunate name. I spent some confused minutes thinking that the author of SQLite had finally done a complete implementation of [1], which he has been involved in.

[1] http://unqlspec.org

Yes, so did I, especially given that the author of SQLite said he was working on a new database with exactly this name, UnQLite.

This UnQLite developer is a first-class tool.

I imagine that Dr. Hipp won't be happy with the choice of project name when he finds out, if he hasn't already. When I first saw the name "UnQLite", I figured this was his project, since I knew that he was working on something called UnQL.
If I wasn't going to get the benefit of my data automatically being distributed across servers, why would I use this? In many cases, NoSQL solutions seem to be a compromise that you make, giving up ACID (yes I know this has it) and other nice query features of databases and in return gaining the ability to scale to much larger amounts of data and having redundancy without needing to think about it.

In this case though, it's all embedded so I don't gain any of the benefits.

The point being that if you need a local data store, sometimes you would rather have a document-based one where you don't have to create the code to serialize your objects into (e.g.) a SQLite database, and to reconstitute them back.

A good example would be something like an address book data store. Especially if you want it to be dynamic so that the user can add/remove fields (e.g. allowing the user to attach as many phone numbers as they like to the contact, rather than just a static 5 numbers). If you have to implement this in SQLite, then you have to develop the schema for it, and the (de)serialization code. With a document-store, you can just do something like say "store these fields" and you're done with it.

  | giving up ACID (yes I know this has it)
If you know that this has ACID, then why are you talking about giving up ACID? The fact that many NoSQL implementations give up ACID doesn't have any bearing on this discussion.
Is this what Richard Hipp was working on? It doesn't sound like what I remember hearing about back in 2011, but maybe things changed and I missed the news?
No. Dr Hipp and Damien Katz were working on UnQL (http://www.unqlspec.org/display/UnQL/Home), which is unfortunately quite similar in name to this new UnQLite project.
It doesn't explain the advantages over all the other dbm alternatives:

http://en.wikipedia.org/wiki/Dbm#Successors

I'm yet to see a good replacement to Kyoto cabinet both in terms of ease of use and performance. I feel it'd be better energy to pick up kyoto cabinet and maintain it than re-write something from scratch.

Isn't Kyoto Cabinet GPL licensed though which makes it less useful for embedding?
I would love an embeddable nosql database engine but this isn't it. I'm looking for something like MongoDB but embeddable. RaptorDB comes close but it doesn't have full support for Mono yet. https://raptordb.codeplex.com/

Can anyone recommend one?

Honestly, you could build a decent embedded document store on top of SQLite pretty easily. It's hard to beat SQLite's long and solid track record for embedded data persistence.

The default BLOB limit is around 1GB.

http://www.sqlite.org/limits.html

Can I ask why? From my other comment on this article I don't see the use case for an embedded NoSQL, but I'd like to.

Where and how would you use it? What, if anything, do you currently use in its place? If nothing, is there something you could use instead? Why is it better than a serializing your own domain model (assuming you have a domain model), or other alternatives?

https://github.com/Softmotions/ejdb

exactly what you wanted.

Nice, I'm currently using SQLite for key value storage, because there aren't better light options afaik. Python shelve module is totally useless with multi gigabyte tables and millions of keys.
What's wrong with BerkleyDB, Tokyo Kabinet, Kyoto Kabinet, LevelDB .. ?

It's a real question because I haven't actually used them. SQLite is fine, but if you don't need sql they probably fare better.

Perhaps a little tangential, but that reminds me I wrote a SQLite interface with a shelf-like API, for when I wanted incredibly simple data storage which may grow more complex and relational later:

    from sqliteshelf import SQLiteShelf
    d = SQLiteShelf("filename.sdb", "tablename")
    # now use d like any other dictionary, but this one has
    # persistent storage
https://github.com/shish/sqliteshelf
Check into Google's LevelDB library.
This seems backwards to me. Sure, use the right tool for the job, but still. SQL server -> noSQL / embedded SQL -> embedded noSQL. At every step there's someone going "X is hard, let's not do X" and we end up with a serverless, configuration-free, noSQL transactional database. The next logical steps are presumably the removal of transactions (who needs them, anyway?) and volatile storage (RAM is fast, let's use that).

I can't wait for the day malloc appears on a page with bullet points citing all the latest buzzwords.

I wrote a very simple thing to sort-of emulate this on SQLite. It works rather well for prototyping!

https://github.com/stochastic-technologies/goatfish

It also supports indexing on arbitrary fields and is only a few lines of code. It did come in very handy.

I think some folks are missing the point of being able to scale up when needed by just changing the connection string. If you are trying to prove a big data concept with small data, this might be a cheaper path. Now, if there were just standards for NoSQL query interfaces, this might be 100% true...
Does anyone know how this compares in terms of performance to leveldb, lmdb and kyoto cabinet? I've been doing some potentially terabyte scale stuff and while another option is welcome I don't feel like re-integrating yet another local db to find out.
similar project

https://github.com/Softmotions/ejdb

A very uncharitable reading of the features seems to suggest that this is some strange bastard child of a hashmap, a scripting language interpreter, and a JSON serializer.

I'm sure it's much more than that, but one wonders where this stacks up compared to existing solutions.

So dbm, but actually with a dbm-incompatible API? Why?
isn't leveldb is there for this?
This would be a great replacement for the fairly awful sqlite on android - accomplishes much the same goal, but should fit into Android a lot better.

Someone should write a android/java wrapper for it in the same way Google has a built in wrapper for sqlite.