Isn't Clojure the literal "Lisp for the modern day(a.k.a JVM)"?
I hope I don't get kicked out of HN altogether for this comment :)
back
4 comments
IMHO Common Lisp is the modern one. Using the JVM is nice for certain uses, and clojure is a fine language, but going bare metal from higher abstractions is some power of Lisp that we should not give up. Same with macros and code generation.
Luckily all of them can coexist, so we do not need to choose.
I agree, Common Lisp is crufty but really stands the test of time. I have been using Common Lisp since about 1982 and old code still runs fine.
My relative use of Lisp languages: Common Lisp 60%, Racket 20%, Haskell 5%, Clojure 5%, and various other Schemes 10%. Unfortunately since most of my work in the last 8 years has been deep learning, LLMs, LLM chains, etc., I spend a little over half my time now using Python. So it goes...
That Vonnegut reference at the end ;D
"old code runs fine" is a poor benchmark for how well a language holds up over time. All x86 code still runs fine. Perl 5 is from, I think, 1994. It's still maintained and in development. Most C code from the '80s would still compile and run just fine (at least POSIX-based, Windows however...). You can trivially find Pascal, Fortran, COBOL, and Forth code that all "still runs fine."
I don't think it's a particularly unique or interesting quality, that some old code still runs. After all, I can go to archive.org right now and run all of that ancient DOS, Amiga, whatever code in a 100% exact (or close to) emulator in my browser.
There is also ABCL, an implementation of common lisp on the JVM https://armedbear.common-lisp.dev/
And it has a great name: Armed Bear Common Lisp.
I am curious how performant compiled Common Lisp is compared to GraalVM compiled Clojure native images. You certainly don't give up macros and code generation when using Clojure, though you do give up a specific class of explicit reader macros. Some reader extensibility can be had (and is used in Clojure.core) via reader functions (data_readers.clj)
You don't give macros and code generation when using compiled Common Lisp, either.
How is code generation handled on modern operating systems and CPUs? Isn't there normally a strict separation between code and data to prevent exploits?
I just checked: SBCL on Linux/amd64 puts compiled functions in a r+w+x page. You could write Lisp that writes Lisp that writes self-modifying machine code, and it would (I think) run by default.
(compiled-function-p #'f) ; T
(disassemble 'f)
; disassembly for F
; Size: 58 bytes. Origin: #x5350CD14 ; F
; 14: 498B4510 MOV RAX, [R13+16] ; thread.binding-stack-pointer
; 18: 488945F8 MOV [RBP-8], RAX
; 1C: 4883EC10 SUB RSP, 16
; [...]
$ pmap $PID --range 5350CD14
442331: /usr/bin/sbcl
0000000053498000 122272K rwx-- [ anon ]
total 122272KWas lisp compiled code, not code.running in a repl, ever able to self modify?
It looks like that wasn't the case even in 1996:
Self modifying code in the sense of changing one instruction to another at runtime (as was commonly done in assembler in the 1960s) is not generally possible with modern Common Lisps mostly because modern operating systems don't allow it. And that's a good thing because such code would be hopelessly insecure and impossible to reason about.
But if you mean "compile a new function called X and replace the old X at runtime", that's easy in Common Lisp. It's not commonly done unless you're explicitly writing some kind of compiler.
What is commonly done is to create a lexical closure at compile time and change its bound values at runtime. IOW changing the private data of a function at runtime is more generally useful than changing its instructions.
What's most common is to write lisp programs that emit lisp source code and compile it at compile time (but usually not run time). Such programs are called macros.
One can define and compile functions in a running Common Lisp. There's a standard function, compile, that does this.
In SBCL, any evaluation of an expression is done by first compiling it. Compiled functions that are no longer accessible (including not having any frames on the stack) are garbage collected.
The really interesting question is not whether users can mutate existing compiled code, but whether it's useful for the Common Lisp implementation to do so. This is because Common Lisp is a dynamic language, where generic functions and classes can be redefined on the fly. If you want to implement such things efficiently, it would be useful to be able to change existing compiled functions to reflect that (for example) this slot of this object is at this offset rather than that offset.
A scheme has been proposed to do that that puts such code off to the side in little chunks accessed with unconditional branches. When a redefinition occurs the branch is redirected to a newly created chunk; the old one is GCed when no longer referenced from the stack. You have to pay for the unconditional branches, but those are typically quite fast.
> How is code generation handled on modern operating systems and CPUs? Isn't there normally a strict separation between code and data to prevent exploits?
You compile code, which is text (data), all the time, don’t you?
The difference is that the output of the compiler is usually not loaded into the same process that did the compiling. That is not the case for applications that are developed On Lisp™.
At worst you have to ensure that you separate what needs to be writeable from executable, and then flip mappings RW->RX when generating new code, just like other JIT compilers do.
Oh okay. I was thinking of self-modifying, not just code generation. That was something that was done occasionally by DOS and Windows programmers before the introduction of DEP in Windows XP.
No, 'eval' is available in many dynamic languages and needs to be utilized with care
It is not that strict. Many years ago machine code was directly loaded into memory and wherever the cpus program counter was, that's what it'd execute.
These days, a page of memory can be set to
Read Write Execute
The exploit mitigation you refer to is having the program typically set pages of memory to never have both write and execute set at the same time.
However, this is ultimately controlled by the program. On Linux, the program can invoke the os call 'mprotect' to change the permissions on a page (though a program can also voluntarily use seccmp routines to forego ever invoking this ever again)
And this is basically what browsers do. They compile the code into memory that has been set to 'write' (but not execute) and proceed to then set it to execute (but not write).
The effectiveness of this mitigation is mitigated by the existence of ROP techniques, which is why Intel started introducing Control-flow Enhancemnent Technology (CET), which is intended to ensure you can only branch to certain locations in memory.
the problem with clojure is you have to learn java (or at least some java)
same issue i see with F#, you still need to have at least some basic knowledge of C#
unfortunately running on JVM actually really means running on Java same for .net it mean running on C#
i dont mind learning two languages, its expected from most developers to know more than one, but context switching in a single function between two language is not fun
The curse of guest languages.
However F# suffers from a bigger issue, that clojure doesn't, belonging to the platform owner, that behaves as if it was an error to make it part of VS 2010.
Isn't that because F# failed to win any significant market share? I think F#'s problem is that C# is too good and this was true even in 2010, its not like Scala vs. Java in 2010.
I worked with F# for about 3 years and the sentiment of most F# developers after using it in anger, including myself, is "I don't want to go back to C#". In my opinion, it didn't win significant market share because Microsoft didn't want it to. This goes even further back than 2010. I don't know if it's because F# was open source and C# was not, as Microsoft's FOSS stance hadn't really come around yet till what, 2016?
F# is way nicer than C# (and Clojure is still way nicer than Java)
And I dont think F# failed, I just think it needs to find a way to hide C# and OOP better
But the language is doing fairly well, tons of educational resources, tons of videos on youtube , several nice projects on github, solid vs code mode, it is part of .net , but still a lot can be improved, and microsoft is far from abandoning, C# is one of MS flagship products , F# is not a flagship product
And I also think that Don Syme is a lot more active working for F# and promoting it, than Rich Hickey is currently working for Clojure
Big F# fan here, but i don't really "doing fairly well" is a fair assessment of the F# ecosystem right now.
> microsoft is far from abandoning
Might has well. When you look at the resources actually being deployed for F#, it's clear that MSFT either don't care or don't really have plan for F#. Most of the work is done by the community. The number of actual paid/full time MSFT dev on F# is very limited. The tooling is extremely limited, there is no official F# libraries for pretty much any MSFT and azure services (have to relie on C# libs.) In leetcode where the have C# (so already have the infra to run .net stuff, and language like elixir and racket (so they do have more niche language), they still don't have F#.
The salvation for F# would come from finding a killer app, something akin to pytorch/numbpy or rails.
Maybe, and it's a good case study to compare F# vs. C# with Java vs. Scala to understand why Scala seems to have higher penetration than F#.
In my opinion, and I mean this in the broadest way possible, the average C# developer is probably less motivated to learn a new language than the average Java developer. That's because C# developers are prescribed by Microsoft what to do, and Microsoft could deliver a productive-enough experience with C# + .NET Framework + Visual Studio for a long long time.
Java's fractured IDE story and runtime (OpenJDK vs. HotSpot) I think lead to a higher concentration of programming language people looking at the JVM as a viable target to build a better experience on top of, so we got Groovy, Scala, Kotlin, (anyone remember Xtend?), IDEA, Eclipse, etc...
The bazaar produced and embraced a functional programming language whereas the cathedral treats F# as thunderdome for new C# features.
Reading C# code makes me shudder. It is so hard to read with all the brackets, spacing, indentation levels, and overall verboseness. I honestly don't know how people do it.
F# can do OOP just as well if not better than C#, and F# is anywhere from 2-10 times more concise than C#.
That being said, the mistake of Microsoft was viewing F# as a competitor to and research lab for C# and the community for viewing and selling it as a replacement for C#.
What Microsoft and the community should have done is treated F# as a competitor and replacement for Python from the start.
> The curse of guest languages.
Every language lives under the iron fist of libc and the C ABI.
Not every OS is an UNIX clone.
You really don't need to know C# to use F#. You need to understand .NET, but that is not the same as needing to know C#. F# sits on top of .NET and the CLR in a much more clean way than Clojure sits on top of the JVM. F# also has excellent error messages and is also much easier to install.
I know F# quite well at this point but only barely know C#. I know C# maybe only mildly better than C++ or Java in terms of being able to read it and guess what it's doing. I certainly can't write in it without looking up a bunch of stuff.
Aren't most libraries you want in C# though? Isn't there a bit of impedance mismatch between how functional F# is and how OOP C# is? I have heard from a few F# fans how frustrated they are that they have to keep interacting with C#
OOP is very natural in F#, so it's not that. Interacting with .NET is not an issue. Interacting with a third-party library can sometimes be an issue, if they're doing a bunch of funky C# stuff. For my own use cases, I haven't needed to pull in a bunch of C# developed, third-party libraries. When I did, for example Silk.NET's window bindings, it was easier to just write my own. I couldnimagine that's different for some more industrial use cases though.
I recently did a small project in F# just to try it out. This was my exact experience. Syntacticly it seemed very nice and in some ways it was cool to have all the libraries I'm used to in C#, however due to the latter it ends up being possible to suddenly have `null` or other weird non-functional thingies suddenly pop up.
Clojure is opinionated against using all the potential features of Lisp.
That is fair, though not a negative for many. Clojure prefers immutable data structures and functional constructs. Rich Hickey even wishes that he added fewer OOP constructs in the language (such as structs) and leaned more heavily on Clojure rooted functional concepts like 'transducers' which came later in the life of the language.
What would be those potential features of Lisp that Clojure is missing?
Multi paradigm support, low level / C interop capacity, really any sort of native features
Clojure is a multi-hosted language. Clean native interop would be nice to have I agree though.
It steers you pretty heavily toward functional programming.
Unfortunately, IMHO Clojure’s maintainers hold an iron grip on the language and actively limit the growth of an ecosystem around it. See “Open Source is not about you”.
It’s too bad, because itself is really terrific.
I think being conservative about evolving the language, and ecosystem growth are mostly different issues, the open source post was about the core language.
Comparing to other languages I think the stability of the language has served the user community really well and been an enabler of the ecosystem. I guess the continuous language additions and resulting complexity & library/framework chuirn in some otherlanguages can also be seen as vitality and growth, but for many of us it's the wrong kind of growth.
(And yes there are also disadvantages to the centralised development model of the core language)
edit: there's a good summary of the discourse surrounding this post in the reddit comments at the time: https://www.reddit.com/r/Clojure/comments/a0pjq9/rich_hickey...
Link to Open Source is not about you:
https://gist.github.com/richhickey/1563cddea1002958f96e7ba95...
The iron grip is on a decidedly double-edged sword. I applaud the continuity and maintainability of Clojure and its ecosystem. The consistency of the language has come from this stern stewardship. Rich makes very clear that the source code for the language is available to be forked. Very interesting developments have come from this, most notably babashka.
To their right, Rich Hickey almost got bankrupted while developing Clojure.
The usual "expect all for free, give nothing back" attitude.
Rich Hickey basically wrote this in response to one of the notable and respected contributors of his ecosystem. If Rich wanted open source to be nothing more than a license and delivery mechanism then he shouldn’t have accepted volunteers. But a language without an ecosystem of volunteers is a dead language.
Clojure "the language" is certainly ossified, but Clojure "the ecosystem" is doing very, very well.
Many languages do quite well as commercial products.