back
98 comments
I have yet to check it out, but it's very cool when people try to rethink SQL, and also revive Datalog. I (morally) support this effort.

Myself, I would like to see data query/manipulation language as a total functional language, possibly based on the idea of categorical data transformations: https://www.categoricaldata.net/

Also - bit of a rant - if you're creating a new programming language, consider making syntax and semantics separate in the specification. Lots of people get hung up on arguing about language syntax but it's really semantics differences that are important for compatibility. Lot of new languages comes up only to fix syntactic problems with existing languages but create small semantic differences in the process, making automated translation from and to existing languages difficult. I wish we could move to a world where syntax and semantics in programming languages are discussed separately from each other.

> Also - bit of a rant - if you're creating a new programming language, consider making syntax and semantics separate in the specification.

I agree. Implementations should accept a stable, machine-friendly format (doesn't matter which; JSON, s-expressions, or even XML would do). If they also accept a human-friendly format, there should be a standard/built-in translation from human format -> machine format (optionally the other way too).

This way, we can always convert random real-world code (scraped from GitHub, or whatever) into a language-agnostic format (yes Python has an `ast` module; that doesn't help a Python linter written in something else, like Go); tools can manipulate this format without having to care about the surface syntax (e.g. linting/doc-gen/static-analysis/versioning/diffing/refactoring/macros/etc.); the output of such tools can always be fed back into the main implementation to compile/run/type-check/syntax-check/etc.

Good point about separating syntax and semantics. Maybe new languages should use something like lisp syntax to nail down the semantics, and then people could create their own syntaxes from there.

IIRC, Ohm (successor to OMeta) separates syntax from semantics!

> Myself, I would like to see data query/manipulation language as a total functional language

Erm, like PromQL?

Nice to see Datalog being validated by a big name, though I don't see what's modern about Logica in particular, or why one should use it over plain Datalog (as syntactical Prolog subset) when the available backends are restricted to SQL rewriters or locked-in to BigQuery. Will have to look into the model for aggregate queries which I guess is a selling point for Logica (as is modularization/composition with optimization), and a weak point since typically neither in Datalog (the decidable logic fragment) nor portable.

Edit: also I find the title a bit grandiose since this isn't about Logic Programming in general, but only database querying

> ...or why one should use it over plain Datalog...

I have been looking for examples of how to use a practical implementation of Datalog for years and the closest I've come to is actually miniKanren instead. Could you point me to codebases that productively use Datalog internally?

It seems that it's open source (Apache 2.0) and can generate SQL for PostgreSQL and SQLite in addition to BigQuery.
My hope would be mainly that this can get datalog into mainstream use, and soon get more (and more mature) libraries created by the community. That is in itself very exciting to me though.

Would be pretty awesome if we could have logica (or something similar) for dataframes (including pandas), and so could build pipelines of transformations-via-queries on those.

(If there is anything like this already implemented, I'm all ears!).

Not a great introduction to the language, IMHO. There is not a clear use of logic to automatically reason about anything, just query composition. It seems the language is much more powerful than what this introduction makes it to be!

> English words (...) often capitalized to keep the old-fashioned COBOL spirit of the 70s alive!

I like logic programming a lot but a convention not technologically enforced is a poor reason to argue for a language change. When arguing about SQL limited abstraction capabilities that space would have been better spent talking about CTE limitations, for example.

Also:

> To make things worse, SQL code is rarely tested, because “testing SQL queries” sounds rather esoteric to most engineers, at best

So nonexisting best practices require a language change, apparently. It was also not showcased how this can be done in Logica, beyond the table mocking that could be done with a "with xxx as (values a, b, c) select (query to be tested)" approach in sql.

> So nonexisting best practices require a language change, apparently. It was also not showcased how this can be done in Logica.

Look for the section containing the text "As a final example, let us mock the comments table, in a unittest of a query." That demonstrates mocking and is explicitly pointing towards testing. The article is only a high-level intro document.

I have a hard time understanding what problem this solves versus SQL?

This really should have demonstration of some of the actual use cases where this shines vs SQL.

Also: SQL-92 has values clause which makes the example provided a little bit silly, you could just use

    values (2),(3),(5)
Another example they gave is a 5-line (excluding imported code) mocking code with a comment "compare that to what you would have to do to achieve the same using bare SQL". Okay..

    select * from (values (1, 'hello'), (2, 'logic'), (3, 'programming')) as mocktable(user_id, comment);
I guess the appeal of Datalog is the recursion that can be expressed in rules:

    has_descendant(?ancestor, ?descendant) :-
        has_child(?ancestor, ?child),
        has_descendant(?child, ?descendant).
Here, the assumption is that you have explicit `has_child` facts (expressing vertices in a graph, essentially), and the above rule gives you paths of arbitrary length.

In SQL, given a table has_child(parent, child), it is not clear to me how you can get all descendants of a given person, or all ancestors.

Other people talk about recursive extensions to SQL, maybe that provides a way.

The problem Datalog tries to solve is complexity: SQL "pulls" data (what's a query after all) to a calling application. Datalog builds up data relationships through declarations. That means that: a) that entities can be inferred from these relationships as opposed to large complex queries, b) that some of these relationships can be built up by code/robots as opposed to humans declaring them.

The end result is (you hope) a very complex database where the smaller blocks/relationships can be audited and verified quickly, and where parallelization more or less comes for free.

The reality is that Datalog systems end up being massive hairballs of declarations that are hard to unravel for mere humans (well, regular developers) and that query-based solutions are 10x faster to develop for 80% of the application use cases.

The closest parallel is functional-vs-procedural programming (don't flame me); it's a niche solution for niche problems.

Source: former Datalog developer for ERP systems.

I actually mostly agree with you, except for the fact that in reality SQL is not a language, but a family of languages, some of which don't support the syntax[1], including Google's own BigQuery. Whether or not this is a reason to create a completely new unrelated language is still up for a debate.

Tangentially related, but does anybody know of a program or a library that takes standard SQL queries as input and outputs one or multiple equivalent queries using the SQL dialects of a set of DBMSs? That is, compiles a standard SQL query into a PostgreSQL one, an SQLite one, etc.

[1]: https://modern-sql.com/feature/values#compatibility

That's explained carefully in the first 5 paragraphs, especially paras 1, 3, 4, and 5.
SQL queries aren't just esoteric, they have highly opaque performance implications. Two ways of doing an SQL query that might look mathematically equivalent to a human could result in orders of magnitude speed difference due to one of them using the proper index and the other having to do a sequential scan, or various other performance issues like creating temporary data.

So this is going to run into the same issues as any SQL code generator (compare Hibernate for example): you need to know what query it will output. And you need DBA skills to know what that query means in terms of performance. Neither of those steps can be skipped.

Nor is unit testing necessarily helpful when using small n. Issues of poor scaling don't show in tests unless the data is large.

Compared to Prolog, which is sensitive to declaration/search order and can assert enough new facts to not terminate, the risk of inefficient but always correct queries is a marked improvement.
This is just pure speculation on my part, but:

What about optimizations? It seems like it should be possible to construct a SQL query that doesn't hit the pain points (e.g. avoids queries that do not use indexes). Although from my experiences with other ORM frameworks, that probably isn't an easy problem.

Even then though, since it looks like it somewhat aims to replace SQL even in the database-construction step, that might help in this regard, by constructing a more optimal representation of the data (which doesn't seem to be tabular)?

Unit testing I am similarly skeptical about though. The article does mention it being "rather esoteric [sounding] at best", I would actually agree with that expression, haha. I don't think I've ever written or even seen, in my 8 years as a developer, a 100-line SQL query that was not at least partly generated (and hence required testing as a unit, and not just the code around it). I suppose Google operates at a different scale, but still.

There's Datomic[1] though it's proprietary. Regarding the Prolog, I hope they will take a look at newly emerging "GHC of Prolog" in Rust - Scryer[2].

[1] https://www.datomic.com/

[2] https://github.com/mthom/scryer-prolog

What do you mean with "GHC of Prolog"? I don´t know a lot about the Haskell ecosystem so I dont know what that implies.

Edit: Never mind, the Scryer Prolog github page states it as follows:

    Scryer Prolog aims to become to ISO Prolog what GHC is to Haskell: an open source industrial strength production environment that is also a testbed for bleeding edge research in logic and constraint programming, which is itself written in a high-level language.
I recently gave Rego a shot but had a difficult time grokking it. It's also inspired from Datalog. How would you say Logica compares to Rego?

> It supports modules and imports, it can be used from an interactive Python notebook and it even makes testing your queries natural and easy.

I don't see any examples of how to do tests in the announcement. Consider adding some.

Nice to see some new initiatives in this domain (or old ideas resurface), but there is a long way towards mainstream adoption IMHO:

* My business clients and I speak SQL together. I don't see them learning a new language. I don't have the authority nor any will to force them to.

* I can spin up a container for testing business rules logic (and often share the results back to the client: here is what the impact of updating rule A is, rows of type W will be affected in this way).

Even though SQL has ceremony/verbosity, I'd rather see the standard be evolved. My clients and I could pick it up more easily.

----

That's great for BigQuery though. You can't spin up a BigQuery docker container anyway, and testing with another schema/project is risky while you have interns around.

Logica compiles to SQL. It is semantically equivalent but not syntactically so the gain has to be in differences in the grammar.

I see that one can create predicates (functions) with parameters as a means for code re-use. I could also see that having implications for testability. That's interesting.

I could see how one could build a DSL with Logica to make fairly tricky queries easier. That's interesting.

Has anyone used this? If so could you explain how are SQL functions called? Do they have to specifically exist in Logica or are they just assumed to exist in SQL? (I'm thinking about geographic functions in particular for example. Are window functions also possible in Logica?

Fantastic to see a logic programming/datalog/prolog-like from a big actor like Google. Perhaps this can make more such tools become mainstream.

Only I wish we had such a language for a more generic streaming / data processing framework, such as materialize [1].

I was very optimistic about that for some time, as the guy behind the technology, Frank McSherry, wrote some datalog tooling as well [2].

[1] https://materialize.com/

[2] https://github.com/vmware/differential-datalog

It astonishes me that it took so long for Datalog to be legitimized as a query language.

It's almost as if people saw OWL 2 DL, didn't believe what it had accomplished, and didn't try to make anything better.

Related question: Is there a good resource to learn logic programming (using Prolog or something like that) for an experienced programmer ?
The Reasoned Schemer is a highly praised book for that. It teaches minikanren and builds it from scratch. I personally did not like a lot its "socratic" style.

The best way to learn it, in my opinion, is to implement microkanren, which is micro by design for teaching purposes. It is small enough to fit in your head, understand what's unification, and play with it. Then you can jump into other implementations.

If you like clojure, you can use core.logic, although documentation is not abundant.

More prolog-related, The Power of Prolog https://www.metalevel.at/prolog has been praised here several times.

Without any hesitation I would recommend the Ivan Bratko book, it is so densely filled with knowledge. Prolog is vastly different from other programming systems and some of the concepts take a bit of exposition before they sink in, and this book very much strives to explain quite a lot of mysterious things.

Prolog Programming for Artificial Intelligence by Ivan Bratko.

https://news.ycombinator.com/item?id=18188003

My recommendation would be to learn the real thing, not an almost, sort-of Prolog that's actually a LISP dialect in disguise.

I recommend taking a look at 'Learn Datalog Today'[0] first. Although it's just Datalog (in S-expression form) which is a subset of Prolog but makes people get the gist of it very quickly.

For Prolog me too wondering if there's a great source. But I have read 'the Reasoned Schemer', it used a simple Scheme-based logical programming language for teaching purposes and it's very educative and entertaining.

[0] http://www.learndatalogtoday.org/

http://amzi.com/AdventureInProlog/index.php is i feel the best for learning to actually write something in prolog. Though it's maybe not so great for logic programming as a paradigm.

Also it's got some small incompatibilities with SWIprolog and I don't know how well amzi works under Wine so it can be frustrating if you're on linux.

Maybe I'm missing something, but why don't query languages allow for sum types and pattern matching? Why is it so hard to express in SQL a table that can contain either this schema or that schema?
The relational model describes a relational algebra, which in turn specifies a "relation" (often called a table) to be a set of tuples (rows), and "relational operators" that accept and return relations. And the relational algebra is complete in the sense that it can express all queries expressible by predicate logic + types and changes through a small set of operations.

The relational model adds constraints, state, and a mechanism for first-class derived relations (updateable views).

And while you can stick anything with a well-defined equality in a relation, including other relations, the point of the relational algebra is to describe structure using relations, thus making it all accessible to relational operators. In a properly normalized database, all structure can be manipulated through a common set of operations.

So you could, e.g. create a relation with a single JSON attribute and call it a day. But now, in addition to the relational operators, you need a whole mess of JSON operators to query it.

Thus, while you could have a sum type, you don't need this because you can put the various summands into separate relations. For instance, the simple case of booleans:

    Persons(key id: int, name: str, is_tall: bool)
    ... noramlizes to ...
    Persons(key id: int, name: str)
    TallPersons(key id: int)
Or for an Either:

    Persons(key id: int, name: str, zing: either<int, str>)
    ... noramlizes to ...
    Persons(key id: int, name: str)
    LeftPersons(key id: int, zing: int)
    RightPersons(key id: int, zing: str)
    AssertEmpty: LeftPersons{key} & RightPersons{key}
What you really want your database to do is to let you enter that first "Persons" table with the sum type. That should logically be a derived table that is backed by the normalized tables.

Then, you'd get the simplicity of entering Persons.insert(key=5, name='bob', zing=Left(5)), but that's simply an updateable view. It will really update the base tables Persons / LeftPersons with simple atomic values.

I’d also really like to understand this better. I did some (casual) research and couldn’t even find any decent papers addressing this.
It seems misleading to call this "Datalog". The GitHub repo even says "among database theoreticians Datalog and SQL are known to be equivalent", which is absolutely wrong without qualification. Some flavors of SQL will have recursive extensions so that they could be considered equivalent, but that is not true in general.

I can't find any mention of recursion on the original blog post or the GitHub page. Without recursion it isn't Datalog.

A major difference between Datalog and SQL is that Datalog uses set semantics, whereas SQL uses bag semantics. For those not aware, that means facts in datalog are unique. SQL’s equivalent to facts (records) are not unique, and relations can contain duplicates.
SQL 99 has recursion via recursive CTEs, so the claim is probably valid.
Datomic-flavoured Datalog: http://www.learndatalogtoday.org/
The C-like syntax is a bit depressing.

There is a syntax debate I respect. While I prefer austere syntax, deeper thinkers like Bill Joy note that programmer productivity increases with more information on screen at once. Syntax that improves both code density and clarity is a good thing. I love Haskell and Ruby in actual use, even if I want to prefer Lisp without parentheses (an easy preprocessor if one thinks it through).

I cannot respect perpetuating C syntax just to attract users who would otherwise be challenged (that Apollo 13 astronaut who "never trained in the LEM"). Rob Pike once gave the only justification I can understand: Code used to need to survive communicating through channels that mangled whitespace.

That is no longer the case, and modern editors all support syntax highlighting. (We've reached the point where one should develop an editor language server in parallel with any new language.)

If your editor can figure out your language's grammar, and then you can with the editor's help, then one achieves greater code density and clarity at once. Some people do love terminals, but most people use graphical user interfaces. Why is language design stuck in terminal pre-history? There is no excuse in 2021 for lots of stray punctuation that's just ground glass in programmers' eyes.

> There is a syntax debate I respect. While I prefer austere syntax, deeper thinkers like Bill Joy note that programmer productivity increases with more information on screen at once. Syntax that improves both code density and clarity is a good thing. I love Haskell and Ruby in actual use, even if I want to prefer Lisp without parentheses (an easy preprocessor if one thinks it through).

I strongly prefer verbose type systems, in particular, some lightweight type inference is good as long as it's not full fledged HM type inference (like Haskell, Rust etc). The problem with HM type inference is although it's extremely powerful and makes the code look cleaner, it hides important data from programmer, which ultimately causes 2 bugs:

* variables being inferred to have types slightly different than ehat programmer expects. E.g. I expected foo to be A(B(C)) -> D(C) turns out it's actually A(B(X)) -> D(C) which also type checks.

* Errors can be harder to read.

This seems to mostly be Prolog syntax, a thing which I can assure you has never been chosen to attract users.
I'm surprised there's no trademark issues with the name. I worked for Logica, the company for many years. At some point later on they got bought by CGI but you'd have thought they still held the rights to the name. CGI is a stupid name - a three-letter acronym that conflicts with more than one unrelated IT term. Seems logica.com still redirects.
I don't believe you can trademark the name of a computer language
I am trying and find concrete examples of where this is superior to mainstream SQL. Right now the documentation and examples are targeted at higher-level users than myself and it is not clear (to me that is) what it is trying to solve. Anyone have an ELI5-type link?
In the examples I saw, I never saw one that did a "join". I guess I'm surprised that they positioned this as a replacement for SQL but gave examples like detecting if something is prime. Might be useful, but I didn't get a good sense for it by reading the article or nearby docs.
> The main flaw of SQL, however, lies in its very limited support for abstraction.

This is objectively bullshit.

What SQL database engine doesn't provide views, stored procedures and/or user-defined function support?

Failure to construct higher-order abstractions in SQL is a failure of the engineer to understand the problem domain, not a failure of the tool.

SQL is capable of operating at any level of abstraction you wish for it to. It is all engineering from there.