I wonder when we'll see new languages created specifically with LLM's in mind.
back
4 comments
I think it would be better to approach it from the other side, the priority is not to design a language for LLMs but a language more suitable for humans to think with. And not a natural language like English, which is inconsistent and allows illogical formulations, but something like Esperanto or Interlingua (Latino sine flexione). Something that is based on mathematics and logic at the bottom, like Lean, with enough abstraction layers for a person to be able to "speak" with the machine intuitively.
The way forward will be more focused on formal specifications kind of approach, and there are already a few attempts at it.
We’ve seen several already, just search it.
What would that look like?
An LLM-only oriented language doesn't make sense, because without human generated training data there is nothing for the model to learn from.
But if a human-oriented language were to be designed to also be better for LLMs, I think it would involve deeply expressive syntax that can succinctly but distinctly represent a very broad set of common operations. Succinct so context can be managed well, distinct so completions don't confuse one thing for another, broad so as much "reasoning" can be taken away from the LLM as possible. An anti-C. A new take on the goals of Java and Go to be languages that protect the application from the Junior Developers You're Likely To Hire.
I somewhat think it would also involve application state images ala Smalltalk. LLMs seem okay at generating small deltas. Many deltas sequenced together invites compounding error. LLM generated apps are unlikely to lead to common libraries being compentized and extracted out of the application to share with other applications; it seems like LLM code generation is already a "married to a specific project" act already. So, having a living state image might reveal some benefits by leaning into incrementally developing the application in situ, as a whole.
I would guess a good way to implement a language meant for LLMs would be to strip out as many features as possible, so that all programs needed to be composed of just a handful of common structures written in shorthand. As much compiletime enforcement as possible. Maybe human-crafted abstractions for more error-prone concepts like multithreading that limit how they can be used in the language but provide ample, testable bumpers.
It would be interesting to design some mechanism where an external actor could program extensions into the language easily, which then would become a new black-box tool that an LLM could program with. Ideally though, I think you want to remove as much agency in reimplementation and re-abstraction as possible from the LLM. This is really just some tier above current high level languages, with as much dumbing down as possible.
I'm not sure this is really necessary or would even be effective because there wouldn't be a million examples for the training data of an LLM. But I imagine something along these lines could make an LLM more productive and token efficient.
I don't know but I would imagine there are a lot of inefficiencies in modern languages from an LLM perspective that it could strip out, reduce token costs, improve speed etc.
So, assembly?
That would be less efficient
There are already plenty of PLDI and SIGPLAN talks on LLM => Assembly, with some guardrails for determinism, instead of having a "classical" language as translation layer.
Sorry are we referring to the efficiency of generating programs (i.e. optimizing LLM software development) or the efficiency of the output (i.e. optimizing software through the generation of assembly)?
We are referring to not having to use 3GL languages as intermediate step for LLMs to generate native code.
"Programming Language Design and Implementation in the Era of Machine Learning"