I take it you've read the source code of Apache and Nginx then and confirmed that they aren't doing anything malicious as well? :)
In any practical security system you are always going to have to trust that certain components are doing what they say they're doing. Fortunately, Let's Encrypt is from a reputable group, and sponsored by organizations with good reputations (Mozilla, EFF, etc.), which is about as good as you can hope for.