Item — HN Reader

After C and DasBetterC (safer C) have officially become the subsets of D language [1], it will be very interesting what languages will be the de-facto superset of D. With excellent support for metaprogramming and CTFE in D (amongst other features), these supersets language can be very flexible and fast at the same time.

Kudos to the TIL's author for trailblazing this idea based on TCL. It will be very beneficial and handy for scripting commands and shell like behaviors. Just wondering is this type based TCL like language similar to Little? [2] Will it eventually support compilation similar to Emacs Lisp? [3]

Personally I'd love to have superset language in D for data science. It should be also easily embeddable and support prototyping like Lua. On top of that it should have excellent support for array, ndarray and dataframe like R [4]. Since it is based on D, then it can fulfil the the requirements for both type A and B data scientists [5].

[1]https://news.ycombinator.com/item?id=27102584

[2]https://wiki.tcl-lang.org/page/Little

[3]https://arxiv.org/abs/2004.02504

[4]http://adv-r.had.co.nz/Data-structures.html

[5]https://www.quora.com/What-is-data-science/answer/Michael-Ho...

2 comments

The De facto super set of D code will always be D. DasBetterC is mainly a thing to use if you really need it, and the ImportC work is supposed to import C.

If you want to see data science done in D take a look at the Mir libraries for example.

Symmetry Investments have their own DSL which has a bunch of stuff in it for data wrangling. It's closed source (I have no insider knowledge here so potentially for good), but uses lots of things D makes extremely easy (like talking to other languages) to pull in data from all over the place - like compiling C++ using Cling called from D.

D is lacking libraries but makes writing them much more tractable than other languages, this is a shame.

To clarify a bit on the difference between DasBetterC and ImportC:

DasBetterC is what its name suggests, it's for C lovers who enjoy programming in C, but:

1. are sick and tired of buffer overflows

2. want to do metaprogramming but realize that metaprogramming with the C preprocessor is barbaric

3. want modules and struct member functions

4. don't want a runtime other than the C standard library

ImportC is a straight up ISO C11 compiler. It's not for programming in (although you can do it easily enough), it's so D can directly access your existing C code, without having to tediously translate it to D.

To that end, ImportC doesn't warn about execrable coding practice, the error messages are utilitarian, and has no extra features. If ISO C11 says it's ok, ImportC stamps it with approval.

> The De facto super set of D code will always be D.

Well, I'd like to make it clear that creating a "de facto super set of D" is not my objective, anyway.

But I'd like to say that, although compiled languages are nice and D is really amazing, I don't even dream of using D to create some quick-and-dirty scripts, the same way I don't even dream of using Til to create shell scripts (I never met any general-purpose language that served as a nice shell, Tcl being the closest one, but still - it's not a traditional shell and the experience is only tolerable, IMHO).

Furthermore, it's also not my objective with Til, but I believe, anyway, that a nice scripting language on top of D would prove very beneficial to the language - it's interesting to try it, right? At least to see what happens...

I'm not sold. Why do I want a scripting language to glue together that is designed to be good a glueing things together?

It's a common misconception that programming languages are either fast or expressive - D is both. The rub is probably that you need to know more about programming in the sense that you have to have good aesthetic taste but also a knowledge of what makes programs actually fast to achieve both.

And besides, what is a script? Code. If you're really only going to use a file once, then I guess do what ever you want but most "scripts" end up being programs.

>Why do I want a scripting language to glue together that is designed to be good a glueing things together?

A good example might be calling legacy libraries like BLAS and LAPACK. These are battle-tested libraries that represent probably hundreds of millions of dollars of developer time, but Fortran is annoying for IO-heavy tasks (at least in my opinion).

I like that I can spin up a scripting language like python and call into LAPACK for performance and correctness, but still have modern and ergonomic networking or what have you.

Again, D is designed to make calling into C code easy within the same program you were working on. The idea that you can't have a language that is both expressive and fast is just Stockholm syndrome from C++ being unergonomic.

Check it out on the website, you can write an RPN calculator in D in less lines than a Haskell implementation last time I checked, all while checked to be pure by the compiler.

Not only would a scripting language be slower because the compiler would be a hack without access to LTO and not have a proper type system etc., It would he a complete waste of time.

You are entitled to your opinion, sir. Have a nice day.

The question is why not? If creating a superset languages based on D will encourage more people to use D either directly or indirectly me think it's really worth it.

Please check Terra language (now in beta), on how they created Lua like language to enable easier optimization in C++ and other languages [1]. They reported Terra-based autotuner for BLAS routines performs within 20% of ATLAS, and DSL for stencil computations that runs 2.3x faster than hand-written C [2].

According to D language origins paper, D is meant to be general-purpose programming language with support for procedural, object-oriented, generic, generative, and functional programming [3]. It'll be a shame not to fully utilize the powerful nature of D. In addition, D also supports D language compiler DMD as a library [4], and there's also libDSL, a library for developing embedded domain specific languages in D via template metaprogramming [5].

[1]https://terralang.org/

[2]https://terralang.org/pldi071-devito.pdf

[3]https://dl.acm.org/doi/pdf/10.1145/3386323

[4]https://dlang.org/blog/2017/08/01/a-dub-case-study-compiling...

[5]https://dl.acm.org/doi/10.1145/2658761.2658770

dmd as a library does not give you access to the compilers backend as of today.

The whole point in D is to write the program in D or write the DSL directly into a D file.

If I can be blunt here, why not? Because it would be crap. It's like sprinting in Wellies, by using a scripting language you either create an absolute Herculean task to wrap everything properly or immediately lose all the powerful things D let's you do.

Well, "wrap everything" is not really a concern (phew!) and I can't see how would anyone "lose all the powerful things D let's you do" with D being the language you'll write most of the code, anyway.

But maybe you're right about writing everything always in D!!! It's, hands down, the final answer to everything anyone would want, isn't? I wonder if there is some way of warning people who write software in Javascript, PHP, Ruby, Python, Perl, Erlang, Elixir, Tcl and all those other clearly inferior languages about that... I'm pretty sure they'll switch to D right away!!!!1!

(Just kidding: I'm writing Til because I want to and nothing more.)

> Kudos to the TIL's author for trailblazing this idea based on TCL. It will be very beneficial and handy for scripting commands and shell like behaviors.

Thanks! I love the concept of "scripting" (that is a bit different from simply a "dynamic language"). I'm quite aware it's just "yet another programming language" but if I can dream of something is that it serves as some kind of incentive for people to develop more libraries in D.

I mean, if you just want to create a Til module that allows you to serve some Web pages using HTTP/2, it shouldn't be that difficult and, at the same time, it could be the end goal itself: just creating a useful module, not something like "it's a crucial part, besides other five, of this big project X I'm working on" (I believe this kind of situation almost always ends with "nah, I'll just use <mainstream language> instead").

> Just wondering is this type based TCL like language similar to Little?

No, it's not. I first heard about Little a couple months ago and it's a very interesting project. But I don't plan, right now, to include any kind of builtin Tcl compatibility layer in Til (although users are free to create its own implementations, of course).

> [2] Will it eventually support compilation similar to Emacs Lisp? [3]

I created the language much more as a tool to learn how to create languages than anything else, but now it's kind of mature enough, I'll confess my dream is to implement JIT compilation, following the steps of LuaJIT (that is an AWESOME project IMHO).

> Personally I'd love to have superset language in D for data science.

That would be nice. Having a autowrap-like way of exposing D code to Til would be even nicer. (https://github.com/atilaneves/autowrap)

> It should be also easily embeddable and support prototyping like Lua.

I believe embedding it is already in a very tolerable state. If you look into the "interpreter" code you'll see it is only 82 lines (actual 69 LOC).

(https://github.com/til-lang/til/blob/master/interpreter/sour...)

And it has a lot of debugging code. Loading a string, parsing it as a "SubProgram" and running it is kind of trivial.

Now, about the prototyping part, I never thought about it, actually...

> On top of that it should have excellent support for array, ndarray and dataframe like R [4].

It's very easy to create new types in Til and they support both "operate" (to apply, you know, operators, like +, -, /, etc) and "extract (to index things or extract information in general from values). I believe it wouldn't be difficult to create a nice module for using these things.

> Since it is based on D, then it can fulfill the requirements for both type A and B data scientists [5].

Maybe. But, I don't know... isn't data scientists all over the world happy and satisfied with Python, already?

Thanks for the replies, very informative.

For the background on the prototyping aspect of Lua, you can check this excellent blog post from Steve Yegge [1].

Regarding Python for data scientist, the type A is happy but not type B. The main reason the ill-fated Swift for TensorFlow project was proposed, is to better serve type B data scientists [2].

[1]http://steve-yegge.blogspot.com/2008/10/universal-design-pat...

[2]https://www.tensorflow.org/swift/guide/overview