It also means I can disable functions or entire pieces of code that I don't need. Less enabled functions _often_ equate to less attack surface. This also results in smaller binaries and _sometimes_ smaller memory footprint. In a world of VM's and containers, optimization of memory and disk usage become more relevant.
Building from source is also useful if I happen to need the latest feature of a specific application that is not available in the shipped/supported binaries provided by an OS distribution. It also means I don't need to start adding software repositories from other 3rd parties and drag in additional library dependencies that can spiral out of control. I keep all my custom builds confined in /opt. The OS is kept pristine and easy to update.