Mill: A fast JVM build tool for Java and Scala

How possible is it to make your tool "zero-config" by default? I see a lot of comments in this thread and elsewhere on twitter asking for essentially `go build`, `go fmt`, `go test` for Java/JVM. I think the language has quite strong convention around directory layout and file naming already, so do you think it would be possible for mill or a mill wrapper to offer the same kind of standardized zero config workflow? I think a JVM tool that gets that right - takes it as far as possible to the golang model - would have a lot of happy users.

I couldn’t quickly find how dependency resolution and versioning works in Mill. Can you give any pointers?

Also, what’s with the “ivy” on https://mill-build.org/mill/0.12.1/comparisons/maven.html ? Any relation to Apache Ivy?

Hi, it looks pretty interesting! There's a broken link in the homepage though, Mill vs sbt links to the gradle page

Good luck for your project!

I’d be interested to read a comparison with Bazel (which you already mention as one of the influences).

For somebody looking to escape from Gradle, Bazel seems like one of the most promising alternatives, as it’s built on sane and sound fundamentals. Although in practice it has plenty of rough edges and annoyances, so maybe there are areas where Mill can do better.

The current version is 0.12.1.

What's required for v1.0?

Does it support Quarkus (esp. native build)?

Depending on when you tried, it could be worth trying again: support for Java has improved greatly in the last few months, as have the documentation. Come by our discord channel if you get stuck and i can help unblock you

I've had the same gripe about having to keep up with a second language just for the build tool for a while. Try taking a look at JeKa https://jeka.dev/

It's intended as a replacement for _scala_ builds. Having a build definition in the native language that doesn't require a different syntax (like a declarative syntax such as maven xml or toml) makes task customization easier for the maintainer of a given project. Unfortunately, it also means that you have to know the language and read the documentation for the build system.

If you want something declarative, there's also bleep[1] in the scala ecosystem. And for single module builds there's scala-cli[2]. It's also possible to use gradle and maven for scala projects, but for an java-only shop I wouldn't be using mill or bleep because there's no need to introduce a new language just to manage the build. For scala/java/kotlin hybrid projects though, gradle or mill or sbt would be my recommended tool because of how tightly they are coupled with the cross-platform build matrix nature of scala library and build system plugin ecosystems. For larger builds, it's mill or bazel because there s a performance cliff in sbt and gradle, and bleep is too new to have all the standard plugins ported. We use mill at writer.

1. https://bleep.build/docs/

2. https://scala-cli.virtuslab.org/

Even in Java, because the language is relatively verbose, many frameworks fall back to an "inner platform" of magic annotations which have the same problem: e.g. just because you know how Java works doesnt mean your mind hasn't unloaded all the SpringBoot annotation semantics! But despite that it is worth it, because conciseness does matter.

Mill using Scala syntax is like that, but with the added advantage that even if you forget how Scala works, your IDE does not. You can really lean on Intellij or VScode to help you understand and navigate around a Mill build in a way that is beyond what is possible for most build tools: You can autocomplete things, peek at docs, navigate the build graph and module tree, etc. and learn what you need to learn without needing to reach for Google/ChatGPT. I use this ability heavily, and I hope others will enjoy these benefits as well

My problem with gradle is that its configuration language is a programming language.

Sounds amazing in practice. And it is. Until you need to fix a 3 year old build that has some insane wizardry going on.

Not to defend Gradle too much, but Groovy is a superset of Java. So if you want, you can just use the regular Groovy dialect and then write Java in your build scripts, it should work.

This is not entirely a solution though, because Gradle's APIs are fairly complicated and change regularly.

Yes, IDE support in Mill is key. Without IntelliJ or VSCode, Mill would not be nearly as pleasant to use as it is today.

Mill and Maven both let you declare goals for what you want to do. One does it in XML and one does it in typechecked code. While XML does work, doing things in code with typechecking and full IDE support turns out to be pretty nice as well!

Author here. Unfortunately this is because my own experience with Gradle is not up to date; I've only lived in the Gradle Groovy world! If anyone is interested in helping out, I have a 1500USD bounty on porting a gradle.kts build to Mill, so we can do a fair up-to-date comparison https://github.com/com-lihaoyi/mill/issues/3670

I never got why people thought Kotlin would help Gradle. It absolutely doesn't.

Groovy was never the problem (Groovy has types, always had, you could use them if you wanted).

Think about it: what do you do with a build tool? You write a little recipe, then you run it. Does that remind something? Yes, it reminds scripts, like bash scripts you run all the time in your terminal. And why are scripts almost universally written without types... and no typed alternative, of which there are many, has caught on? Because if you're just going to modify a script and immediately run it, while keeping it short enough so you can know what it does without reading a book, how does adding types help you? Quite to the contrary, scripts (including build scripts) should be small, and having types all over the place make it far more verbose than it should be, likely pushing it out of your comfortable local memory in your brain, at which point you need something akin to a "real" programming language and a compiled program, not a script. Larger programs benefit from types because you don't just run the program, make changes, and run them again, like you do with scripts. You write them, test them, compile them, package them and finally you distribute them to your users who hopefully only need to configure them, not modify their internals. If your build is that complex, that's exactly what you should be doing instead of trying to shoehorn types into your scripts and expecting them to look like real programs.

Also, the Kotlin DSL just doesn't assist in the most problematic aspect of Gradle: its total lack of discoverability. Try doing something on your Kotlin Gradle file using a plugin you're not familiar with (which is all of them for most of us). It's completely impossible unless you know the DSL of the plugin, just like it was the case with Groovy... Once you know the DSL, it's fairly easy, but even in Groovy you will get auto-completion once you've got to the DSL "entry point", no need for Kotlin. I've been saying this since before they introduced the Kotlin DSL, and now i feel completely vindicated. I've never met anyone who told me "Gradle is so much easier now with Kotlin". But it did mess up plugins I wrote in Kotlin as now Gradle has a dependency on a very particular version of the Kotlin compiler, and God help you if your plugin was written with a different version in mind.

Author here!

`go build` and `go test` do work, at limited scale and complexity. In Scala there's Scala-CLI which is excellent. If they work for you, you probably aren't the target market for these build tools. Once you start layering on bash scripts, layering on make, layering on Python scripts, layering on manual steps written down in a readme.md somewhere, that's the time when you should consider a proper build tool. And if that has never happened to you in your career, count your blessings :)

Why not just write some boring, pure code to handle the build? Why not write my build system in vanilla LYAH Haskell? It turns out that builds do have some specific requirements that most programs do not need to care about: caching, parallelization, introspection, and so on. Check out the following blog/talk for more details:

* Blog Post: Build Tools as Pure Functional Programs https://www.lihaoyi.com/post/BuildToolsasPureFunctionalProgr...

* Video: Mill: a Build Tool based on Pure Functional Programming https://www.youtube.com/watch?v=j6uThGxx-18&list=PLBqWQH1Miw...

Thus "naively" building your project "directly with code" ends up not working, so you do need some additional support. While most build tools end up constructing a complete bespoke programming environment from scratch, Mill tries to leverage the Scala language and JVM as much as possible, so you can re-use all your expertise and tooling (e.g. IntelliJ, VSCode, Maven Central, etc.) almost verbatim while getting all the necessary build-tool stuff (parallelism, caching, introspection) for free. Check out those two links if you want to learn more!

To be blunt, nothing.

The issue is that most people in jvm land are on a closed bubble and haven't seen anything else. This is true for build systems as is for non OO design for example. Most simply don't know better and the rest of us are simply stuck.

Ant and then Maven started simple enough but people always find a way to justify adding more stuff. Gradle already started complex enough and they keep adding more stuff...

In JVM world, the de factor equivalent to `go build` and `go test` are `mvn compile` and `mvn test`, which works 99% percent of the time.

Other build tools and plugins just compete/fill in for:

* improved build speed / test speed: using background daemon to reduce strtup speed, intelligent caching / task reordering to avoid redoing, etc..

* extra functionalities like code generation, publishing or deployments. As code generation is really big in JVM world, and there are many ways to deploy an application: jar + libs in a zip file, uber jars, container image, etc...

Try to build a Go project that uses Cgo and non-trivial C/C++ libraries. Throw in cross-compilation for more fun. You'll end up with an external build system that invokes Go build as one of the steps.

Go projects normally just tend to be self-contained server-like software that doesn't need a lot of external libraries. But once you step away from that, you're on your own.

I guess my problem with Gradle is that app building should be way simpler than it is. Apps are not something niche anymore, but the tooling is still similar to the embedded software for microcontrollers.

I'm working on a project that encompasses both JVM (Gradle, Kotlin) and Golang.

My hot take: JVM build tools, especially Gradle, are a soup of unnecessary complexity, and people working in that ecosystem have Stockholm Syndrome.

In Golang, I spend about 99% of my time dealing with code.

In JVM land, I'm spending 30% just dealing with the build system. It's actually insane, and the community at large thinks this is normal. The amount of time it takes to publish a multi-platform Kotlin library for the first time can be measured in days. I published my first Golang library in minutes, by comparison.

I recommend the "build systems a la carte" paper for a good overview of the various problems build systems address

There are quite a few cases: the moment you touch another language, resources that require a compile-step (e.g. xml schemas to code dtos, protonbuf, all that kind of stuff), sometimes even the code itself requires generation.

Build tools sit in an unhappy corner of the design space where they provide features not found in the core of regular programming languages, and which are so generally useful that there's a temptation to make them very abstract, but then they often lack some of the features that let regular programs scale well.

The key feature that justifies their existence is parallel and incremental execution of DAGs of world-mutating tasks. This is an awkward fit with most programming languages, hence the prevalence of DSLs. But people don't want their build system to become a general purpose programming language, because they don't want to think about build systems at all and because programmers don't buy programming languages anymore, so, this causes a big design tension between generality (people want to use build systems to automate many things) and deliberately limiting the expressive power to try and constrain the design space and thus tooling investment required.

Java is in an awkward place because the JDK was born in the 90s on UNIX, by people who thought make is a sufficiently good solution. You still see remnants of this belief in the official Java tutorials, in JEPs, and in the fact that OpenJDK itself is compiled using an autotools based build system! (fortunately it's one of the nice make based build systems out there).

The problem with make is twofold:

1. It assumes a CLI that's both powerful and standardized provided by the host OS. Windows violates this assumption but Java is meant to be portable to Windows.

2. "Plugins" are CLI tools or scripts and so make implicitly assumes that subprocess creation is cheap. But process creation on Windows is expensive, and starting up JVM programs is also expensive due to the JIT compiling.

Therefore make just doesn't work well in the JVM ecosystem. At the same time, the Java project wasn't providing any competing solution, so the wider open source community was left to fill in the gaps. These days language developers provide build tooling out of the box as part of the base toolset along with the compiler, but Java still doesn't.

So - you ask, what are people doing with Gradle/Maven that requires all those features. The answer is: everything! Gradle builds frequently orchestrate dozens of different tools as part of a build pipeline, build documentation websites, do upload and deployment, download and manage dependencies, run security scanners and license compliance checkers, analyze dependency graphs, modify compiler behaviors, and so on.

Additionally Gradle isn't specific to Java, or even JVM apps. It can also be used to compile C/C++ programs, run native code compilers like Kotlin/Native, and it abstracts the underlying platform so Gradle builds aren't tied to UNIX.

That's why it's so complicated.

Author here! The issue here is that builds, and many other "just configuration" scenarios, are fundamentally complex. So many projects that start off as "just XML" or "just YAML" end up implementing their own half-baked programming language interepreter inside of their XML/YAML/JSON/whatever.

Examples:

* Github Actions Config Expressions https://docs.github.com/en/actions/writing-workflows/choosin...

* CloudFormation Functions https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGui...

* Helm Chart Templates https://helm.sh/docs/chart_best_practices/templates/

There is a reason why Bazel went with Python/Starlark, why Pulumi and CDK and friends are getting popular. Fundamentally, many of these use cases look surprisingly like programming languages: maybe not immediately, but certainly after you've dug in a bit. And having a properly designed purpose-build programming language (e.g. StarLark) or a flexible general purpose language (e.g. Typescript, Kotlin, Scala) does turn out to be the least-bad option

There is basically no DSL. You simply write what a build needs, e.g. you write a function `collectCFiles()` that collects every file with extension `.c`. You then issue a command like `gcc ${collectCFiles()}`. And pretty much that’s it - you can use shell commands, or do anything in scala (or java or whathaveyou). You simply have your functions return either a value (e.g. a checksum) or a location, which is the only mill-specific logic. So your compileC() function will simply invoke your collectCFiles() function, and this invocation implicitly creates a dependency between these tasks. You have written literally the simplest way to describe your build logic. But in the background mill will cache your functions’ inputs outputs and parallelize those that need re-run, which is what a build tool should do.

The implementation may not be the theoretical best, but I think the idea is pretty much the perfect build system out there.

The first advantage the homepage lists is:

> Mill can build the same Java codebase 5-10x faster than Maven, or 2-4x faster than Gradle

Speed per se can be a good selling point (having to wait for slow builds is really annoying).

I can't really comment on anything else though as I just stumbled upon it here in HN ;)

I'm not sure I like the daemon approach. But the cache extension provided me fantastic gains: https://github.com/apache/maven-build-cache-extension