Vanessa Fox from Google replied saying;
We recommend that you leave the verification file or meta tag in place after verifying because we periodically check for it. If the file or meta tag is still in place, the site remains verified. If it is not in place, we prompt for verification again. (This is to ensure that the person who verifies the site continues to own it -- sites change ownership from time to time and this verification check ensures that sites don't remain verified after they change hands.)
This seemed a bit obvious to me, but if one person didn't seem so, then I am sure many others may think it is ok to remove it once you are verified.
Again, you can verify your site by either adding a META tag to your home page or adding a special page to your main root directory of the site. Google has personalized instructions on how to do this for your particular site, when you login to Sitemaps.
Forum discussion at WebmasterWorld.