Very true. We tried to accomodate for audiobooks and podcasts by adding 30 second back button and a larger slider that's easier to scrub. It isn't perfect, but is much better than the Music app.
You're right though, an app especially for audiobooks with bookmarks, notes, etc. would be great.
Thanks for the feedback!