Eigen: A C++ template library for linear algebra

Using it for years, and mostly happy with that library. The performance is awesome for very long vectors / large matrices.

It’s less than ideal for small things when the size is known at compile-time. If one knows SIMD intrinsics, in some of these cases the Eigen’s implementation can be outperformed by a large factor like 2-4. Also it’s very hard to mess with RAM layout of some things (like sparse matrices), just too many layers of abstraction and too much template metaprogramming.

But still, out of the box the usability of Eigen is awesome. And until the code is written, debugged, integrated and benchmarked, it’s generally impossible to tell whether a particular algorithm gonna be a performance bottleneck. That’s why I’m mostly happy with the library.

Eigen is the standard choice (for good reason!) for many linear algebra projects, especially in robotics, but there is a big downside users should be aware of before they chose it.

Eigen makes extensive use of expression templates in C++ to collapse complex operation sequences into streamlined and minimal calculations. This is generally ok, until you need a debug build. I've regularly seen debug builds of software using Eigen run 1000x to 10000x slower than the release build, which seriously complicates various debugging workflows. It also makes it a nightmare to run your test suite through valgrind, for example.

I've seen several engineers attempt (and fail) to try creating/linking a release build of Eigen with a debug build of the rest of the program to try to regain most of that speed while still allowing a decent amount of debugability, but this is really hard due to all the aggressive inlining and heavy use of templates.

In my experience, I would happily accept a 2x or more slowdown in linear algebra performance in release builds in exchange for significant boost in debug execution speed. If you're starting a greenfield project, you should consider how important decent debug performance is before choosing Eigen by default.

The first thing I wondered was about the license. Fortunately, it mostly uses the MPL license:

https://eigen.tuxfamily.org/index.php?title=Main_Page#Licens...

> Note that currently, a few features rely on third-party code licensed under the LGPL: constrained_cg. Such features can be explicitly disabled by compiling with the EIGEN_MPL2_ONLY preprocessor symbol defined. Furthermore, Eigen provides interface classes for various third-party libraries (usually recognizable by the <Eigen/*Support> header name). Of course you have to mind the license of the so-included library when using them.

> Virtually any software may use Eigen. For example, closed-source software may use Eigen without having to disclose its own source code. Many proprietary and closed-source software projects are using Eigen right now, as well as many BSD-licensed projects.

My only complaint is that using OpenMP Eigen can be slower with SMT than without SMT. They even suggest telling to use half as many threads as you have "cores" when "cores" means twice as many due to SMT.

Otherwise, we've seen a 8-10x performance increase in SolveSpace (CAD) in some situations after switching from home-grown matrix operations to Eigen.

One of the best things about Eigen is that it is the only linear algebra package that I know of that easily supports using the same code for low, high, and arbitrary precision floating point numbers. I had some linear system of equations that I need to solve in grad school where the condition number was small enough that double precision was not sufficient to solve the problem. I was able to very easily use double double, quad double [1], and arbitrary precision floating [2] point number implementations to solve these problems. The matrices I used were not especially large, but I couldn't find any other existing packages that fit this use case.

[1] https://www.davidhbailey.com/dhbsoftware/

[2] http://www.holoborodko.com/pavel/mpfr/

Probably won't ever catch Fortran, but using Eigen templates for reductions really opens the door for compile time optimizations; e.g. these are all reductions that do the same thing

    const float residual = (L.array() * P.array()).colwise().sum().square().mean();
    const float residual = L.cwiseProduct(P).array().colwise().sum().array().square().mean();
    const float residual = (L.transpose() * P).diagonal().array().square().mean();

The compiler can optimize using static information, e.g. these would all be handled differently for the following types

    Eigen::Matrix<float,3,3> L(3,3), P(3,3);
    Eigen::Matrix<float,3,Eigen::Dynamic> L(3,K), P(3,K);
    Eigen::Matrix<float,Eigen::Dynamic,Eigen::Dynamic> L(N,K), P(N,K);

I can’t stress enough how this library is a saving grace. It would be incredibly difficult to port my python prototype code with numpy to C++ production without using Eigen.

God bless Eigen, I am using it to implement the state matrices of a Kalman Filter and it's a joy to use its APIs.

Eigen is really nice for getting code that looks like the underlying math and it also optimizes away unnecessary temporaries by using template expressions. However, when you mess up, you get a screen full of template errors that take some experience to understand and debug.

It makes me wish for a language + compiler where linear algebraic objects are first-class values.

This library is used heavily inside Tensorflow. It is “production ready”.

Eigen is a great library.

A similar one to consider that can at times be slightly easier to use coming from a python background is armadillo: http://arma.sourceforge.net/

Not widely publicized, but the benchmarking code is in the source. At one point I was running it on my specific target machines to get performance estimates in support of porting some large-ish CPU stuff from Matlab into C++.

The max performance was in Eigen-calling Intel MKL, but it was a big plus to not need MKL licenses on every development machine.

For anyone still reading this: I'm now confused by Eigen reportedly being basically on a par with optimized L3 BLAS, which I assume doesn't just mean punting to optimized BLAS, which it seems it can do. I can't see any indication that you can do run-time dispatch on the micro-architecture, which I think is important; I saw very poor results until I examined CMakeLists.txt. Anyway, I cmade v3.4 with -DEIGEN_TEST_AVX512DQ=on, which seems most appropriate for my SKX system, and then did make blas. I ran my normal initial quick test using the OpenBLAS 0.3.15 dgemm benchmark, which reports a plateau of ~99000 MFlops. LD_PRELOADing libeigen_blas.so with the same binary under the same conditions (bound to a core, with the hpc-compute tuned profile) gave ~66000 MFlops.

What did I do wrong?

A great library, a cornerstone. Many big libraries build on top of it (e.g. OpenCV, PointCloud Library)

It supports operator overloading too, by the way. This makes it usable where many LAPACK wrappers like Numpy are not.

Is there anything like Eigen for data that's kept on the GPU? I've been looking at porting some performance-critical scientific computing code from Python to C++ and numpy -> Eigen seems like an obvious migration path, but it's harder to figure out what to do with cupy matrix operations.

Recently encountered Eigen. It seems awesome - easy to work with, with lots of options. mat.conjugate() * mat.transpose() seems to take a while for my stuff, but I think I just haven't found the right method (next is to look at mat.adjointInPlace()).

Failed to see what chips are supported. Assuming x86 and ARM and their vector/SIMD are leveraged for performance, what about RISC-V's vector/SIMD, is there a plan to add those in the future?

Ahh, the horror of getting this to compile last week in Python :(

BLAS vs Eigen

Go