▲ 297 points
back
35 comments
Soooo stoked. This should make futzing with it to get started a lot easier.
I've been thinking about going to the FoundationDB summit to hopefully learn more... Does anyone have any good resources for learning about it? I've tried a few times but find the documentation a bit hard to grok. Maybe I'm just slow.
Also if you haven't seen it, this is one of the most amazing talks on distributed systems (because it's about testing them) I've ever seen (by Will Wilson from FoundationDB pre-Apple merger): https://www.youtube.com/watch?v=4fFDFbi3toc
Truly mind blowing how good FoundationDB is supposed to be for its use cases, and this Mongo layer is just the business when it comes to helping adoption.
That talk was glorious. "We want to build a database, but first we need to build a simulator for the database, but first we need to write a compiler to help us write the simulator." It would be self-parodying engineering hubris if it hadn't worked.
> Document Layer has a familiar API, and is compatible with the MongoDB® protocol
Was wondering how I would replace MongoDB due to the shift in Licensing become more and more agressive.
Well , I think I found it.
Farewell MongoDB.
It's like hard slap straight in the face. They changed license because they perceived that multibilion giant(s) feasts and earn money upon their work and gives back not that much.
In return giant just slaps
I’ll never understand why we collectively wag our fingers at individuals or companies that try to keep the likes of Amazon from building a profitable service off of their hard work then contribute back little-to-nothing. Would redis, mongodb and dgraph have even considered alternate licensing if companies like Amazon had thrown them a minuscule amount of funding and patches? We can’t know because they didn’t. And then we sneer at them for having the audacity to try to stay open but stop these giants from using them and throwing them away.
more like a scaling issue, as mongo is master-slave and a distributed horizontally scaled world can't live with such a bottleneck.
Well that’s just a ridiculously uninformed statement. Of course mongodb scales horizontally. I can share many success stories of massive scale with mongodb.
Depends on your deployment specifics and your requirements. I would say that a replica set of mongodb may be faster, but a sharded replica set in mongo may be slower than a local cluster of foundation with the document layer interface.
I'm a little confused. Does the "layers" concept mean the data which is stored in the core can be accessed in any valid manner that the layers allow? Or stated differently: if I've got a dumpster fire of data in a MongoDB that should have been in an RDBMS but moving to SQL would be too risky/slow would this allow me to schlep my Mongo data to FoundationDB and have the app continue to use the Mongo access methods while I start switching APIs over to querying by SQL in batches instead of One Big Volatile Move?
foundationdb is just a KV store that doesn't even authenticate clients connecting to it. the idea is you put layers in front of it to take nice structured requests (of some sort) and turn them into transactional KV manipulations, while also doing things like authentication, access control, logging, whatever else you need.
so you could (hypothetically, if such software existed), have this mongo layer, and then another layer off to the side which took SQL queries, and figured out how to turn them into KV queries against the data that the mongo layer has stored.
more realistically, you could have your SQL clients know how to talk to foundationdb directly (making them into the layer; presumably they're a web backend or something already), and they could know how to execute the high level operation they want to perform against the KV data stored by the mongo layer. conveniently skipping all of the SQL translation that there isn't software for.
Really excited that they were able to release this layer. With the changes to the license of MongoDB it could give people a real alternative.
This is massive for adoption. Can't wait to give it a try.
Is there any reason to expect fundamental inefficiencies with this approach vs a vertically integrated approach like a conventional document store? Any other downsides? I ask because this seems like to obviously good an idea to be true, so I need to look for the flip side before I get to excited. Otherwise, this laying seems like the obvious approach, and I can only lament that it's taken this long.
Yes, the layered approach will always consume more bandwidth unless you just happen to hit the case where your transaction is read-only and the layer process is colocated with the storage server where your data is.
The upsides far outweigh this downside in my opinion. The fact that writes are buffered locally before committing the transaction (improving latency) is probably enough by itself to overcome that downside.
Interesting, TiDB also three components [0](KV, DB and PD), I guess this MongoDB layer in FoundationDB is the DB piece of the cluster.
Haven't dug in, but depends on if there's a system in place to support multiple indexes on your documents and how the system uses them. This could be a relatively small or huge impact depending on your interface requirements, sorting etc.
I don't know much about foundationDB, but when they say API compatible, are they referring to the technical TCP/IP connection stack, or ALSO the concepts of collections and the query language of MongoDB (such as aggregations, find, limit, etc..)
In other words can I throw a giant aggregation query with all mongo's various crazy query language syntax and it will.. work the same?
It doesn't support the aggregation framework currently.
Here is a list of differences:
https://foundationdb.github.io/fdb-document-layer/known-diff...
Can someone who uses MongoDB at scale (many machines, single db) talk about how big gap is between what FDB supports and what their application uses?
Here's what seems to me comprehensive list:
https://foundationdb.github.io/fdb-document-layer/known-diff...
Looks like there are definitely some shortcomings, but also some wins... From the link:
> Sessions
> MongoDB® has introduced sessions in v3.6. The Document Layer doesn’t support sessions yet. Sessions in MongoDB® enable better consistency guarentees. It’s important to note that, even though the Document Layer doesn’t support sessions, it has better consistency guarentees by default due to FoundationDB backend.
Sounds kind of "old" tech to have different layers for supporting different data models. Like the native multi-model approach of ArangoDB much better.
Their aggregation framework is pretty neat and the database also has full graph capabilities. Heck, I can even combine joins and graph traversal in the same query.
i think we are going to generally see more and more object to key-value mapping layers as the latter are widely available via cloud providers, and one basically needs something in the spirit of persistent-volume-claims from kubernetes land in the database layer that can be fulfilled by some key-value backend...
FoundationDB always sounded amazing, BUT, the way they disappeared when it was bought by Apple left a bitter taste in my mouth and I don't think I would ever trust them enough to build my business around their product.
It's all open-source and out there now though, so I don't think you really need to trust them, right?
Are there any plans to extend the API compatibility of the document layer to be compatible with other APIs, such as, DynamoDB?
That's huge. I could have used this a few years ago
Just wandering how will foindationDb compare to mongo in terms of the non functional s - performance, scale of a collection, number of collections, queries with and without indexes, aggregated, etc.
Can someone write a Google Datastore layer so that I can finish my appengine/datastore adventure once and for all?
Is there anything we could be doing to serve you better?
Yes, let me run datastore on my servers. Surely I may use your cloud offer but in certain cases I need to run it on my servers. I can't build/invest in tools/drivers for a proprietary cloud service.
well there is https://github.com/AppScale/appscale which probably has some kind of layer on top of cassandra that will emulate it.
but it's a really big project... (But you can run just the Datastore: https://github.com/AppScale/appscale/tree/master/AppDB)
I already tried it...it looks good on paper but bad in practice. Think of a graveyard where appengine go.
Be the change you want to see
I think it's not worth it. I rather not use Datastore for anything from now on. You have to cut your losses at some point. We are using microservices so it's not hard to make the change(though we loose some tools we designed for datastore).
Wonderful. I just updated the FoundationDB packages in NixOS to 6.0.15 at the release earlier this month. Will try packaging this up soon as well...