This is a long time problem with the current Yahoo! Search Marketing platform. The only way for you to turn off the auto pay feature, the feature that funds your account when the funds are depleted, is by calling Yahoo!. A WebmasterWorld thread tells a newbie to the AdWords game that phone is his only option.
I do believe email also works. But there is no simple button to shut if off in the advertiser's console.
Forum discussion at WebmasterWorld.
