At WebmasterWorld, there is an excellent thread that describes some of the reasons why a site might be banned by Yahoo! Search. These reasons include a "demotion penalty", "linking penalty" and more. Good thread, Tim Mayer from Yahoo! and his team are working hard to be responsive to the SEO/SEM community.
So what do you do if you think you have been penalized? Visit The WMW Thread Named: Yahoo What gives? How do you tell if your penalized? and then if your still not sure email email@example.com.
The thread is a must read for you affiliate marketers out there.