back
23 comments
Allow me to shamelessly throw my project into the ring: https://github.com/ipartola/groper. It is Python-specific, but for me it fulfills a very real need: to have a single library that I can use to both read a config file and command line parameters. getopt/argparse/optparse do not do config files, while ConfigParser and friends are at least less than friendly to use and definitely don't to command line arguments.

My colleagues and I have used this library in at least a dozen projects and it worked great. I am actually wondering if this is worth writing a PEP about to have something like this included in Python's standard library.

Edit: I forgot to mention my favorite feature of this library. As you develop your program, you define the parameters that each module will use, optionally with defaults. When you are done, you can get groper to generate a sample config file for you that includes all the options you defined in your program.

Edit 2: It looks like there are a few people downloading this package from PyPI which makes me very excited. Please feel free to let me know how your experience was with groper and what features you'd like to see added.

Before trying to create yet another configuration library, consider whether your efforts wouldn't be better spent on adding another language binding to an existing configuration language.

I'm rather partial to TypeSafe Config myself - similar to the extended JSON described herein.

https://github.com/LuaDist/lua

The repository size is exactly the same as the submission, but you get a fully-fledged language - which allows you to make parametrized configs, etc, etc, etc.

And the syntax is quite convenient. Look at it for your next project.

Turing-complete, non-easily parseable by external tools configuration? No thanks.
What would make it easier to parse by external tools than a portable library that's insanely easy to embed? I mean, there's not that many now, but that's how every idea starts, good or bad.

Whether Turing-completeness is a good idea depends on how much you trust the configurers. In cases where they're writing code anyway, you may as well have configuration be code too, among other cases.

> What would make it easier to parse by external tools than a portable library that's insanely easy to embed?

A small, documented spec? Something you can easily validate, even if it doesn't feature a schema?

> Whether Turing-completeness is a good idea depends on how much you trust the configurers. In cases where they're writing code anyway, you may as well have configuration be code too, among other cases.

It makes it much easier to misconfigure the application and, depending on how much api surface you're going to expose, can prove to be an absolutely terrible idea in terms of maintenance (I'm looking at you, Awesome window manager). I'll take a "dumb", predictable configuration language for which I'm extremely likely to already have bindings in whatever language I'm working with than having to embed a Lua parser. Look at Ansible for how you can make a DSL with YAML for a good example of how it can be done.

I actually find YAML the most annoying part of Ansible, but that's probably specific to YAML. The syntax drives me crazy.
Really? I find YAML pretty nifty myself, though it has a few gotchas. Many softwares can get away with nothing more than INI-style configuration, but for more complex cases, YAML is much nicer to edit than, say, XML or JSON.
Very nice.

I wrote ARON (a righteous object notation)

https://code.google.com/p/aron/

Whereas your Libucl comes from the config management world, ARON is derived from my prior VRML work. Though ARON is a general purpose object graph serializer, I currently mostly use it for config, mostly as sane alternative to the DI and IoC nonsense in the Java world (Spring).

Both Libucl and ARON have a minimalist, typed grammar. Personally, I very much dislike goo like JSON. So you have my enthusiastic support.

I applaud the syntactic sugar you added, like units suffixes, eg 1k. I added UTC date time. I'm still pondering how to add URIs, both relative and absolute.

For reuse, ARON uses labels and references (my take on VRML's DEF/USE) vs Libucl's macros. Your strategy hadn't occurred to me.

Libucl's try/catch is VERY interesting. ARON's include is modeled after Java's imports, where declarations are at the top of the source file. Libucl's strategy is far more robust.

Okay. Enough stream of conscious replying.

Libucl is great.

Ignore the haters.

Damn. I thought "universal" meant "can parse multiple configuration formats (JSON, XML, INI, YAML...)" but it just looks like yet another configuration format.
Unfortunately external configuration is much more than just serialisation. For me, an ideal universal configuraion library will provide schemas, type checking, full structural validation, and externalised defaults so I can tell the difference between a value explicitly set (or reset) to the default by the user, and a value that has never actually been changed. Hardcoded defaults are evil, configuration is actually a stack of developer-set defaults, packager/distro set defaults, sysadmin-set defaults and user preferences.

Imho these are much more tiresome problems than serialisation imho. If I were going to build a configuration library I'd probably use Apache Avro[1] as a base. It has a lot of these ingredients already.

For parsing atm, since I write mostly C++, I'm prone to just using Boost Property Tree[2], which exploits the fact that most of these formats (even XML) are just trees of key-value pairs. There are parsers for XML, JSON, INI and INFO formats[3]

With regard to this project, it'd be nice if the author would write a complete grammar for his UCL format. The advantage of JSON is it's really well documented[1] and I can always go and grab another implementation. There's nothing in this repo except the hand-written C code. Other than that, the cleanups to the syntax look nice and friendly.

[0] https://avro.apache.org/docs/current/

[1] http://www.json.org/

[2] http://www.boost.org/doc/libs/1_55_0/doc/html/property_tree....

[3] http://www.boost.org/doc/libs/1_55_0/doc/html/boost_property...

"...much more than just serialisation. ...provide schemas, type checking, full structural validation, and externalized defaults..."

I agree with your goals. But I'm confused by the serialization concern.

If one has classes, modeling the configuration schema, and you serialize those config object graphs, isn't that what we're talking about here?

For me the problem with configuration is not the config files - but getting the config into the running application. There is never a perfect way - it is a chicken and egg situation - pass it in from the command line? Ok now do that for 100 servers, I know let's have an app - but that needs to know which 100, for which we write a config file ...

it's all doable, but I always think I could make a better choice.

Interesting - so would you be looking for some kind of centralised configuration deployment system?
No. I am just moaning on the Internet.

To be clear, I am occasionally frustrated by the Chicken and Egg nature of configuration, when there is always another layer of abstraction to start with. So how do I deploy my saltstack servers so that I can pass them the config so that they can ...

a recent example - you start a script / daemon running with a arg pointing to a config file. But that config file might fail and I cannot log that it failed cos the log location is in the config file. There are half a dozen solutions but it just annoys me.

Augeas plus ssh (ansible, whatever) plus some special sauce will do it for you mostly.
As an amateur sysadmin, I'm curious - why use Augeas instead of having a template of each config file be filled in and copied to each node?
Augeas provides some abstraction over different config formats so you don't even have to know the syntax and are less likely to make typos.
Um... Puppet/Chef/Ansible/CFEngine/Bcfg?
Yes, and Puppet etc. has its own configuration format (typically Hiera for Puppet), and now you need to configure your configuration (typically with erb or puppetlabs-concat). Most of your fancy configuration format features are wasted.
Project idea: a tool to compile yaml(/xml/json) files to apache, nginx, php or any other format.
Maybe http://augeas.net/.
I was thinking about converting from a "well known format" to application specific format. File formats like yaml or xml are more popular then augeas tree. Or maybe you are trying to say that augeas supports this scenario?