I didn't have to pay the fee, I think because I produced through Findaway? (I can't remember the reason!)
If you don't have to pay the fee, there's no reason not to do it. I set up a store with all of my audiobooks at a reduced price (giving me the same royalties as selling through other retailers).
Not sure if Authors Direct is worth it with a fee. In the last three months, I've made 8 sales there for a total income of $39.12. This is despite adding the link to my website and sending an email about it out to my 9,000+ subscriber list.