It is true, a site can lose their rich snippets over quality issues or never get the rich snippets in the first place. But what is not true is that if you do have rich snippets, it doesn't 100% mean all of Google's algorithms think your site is high quality. It just means that you past the benchmark of quality for your site to show rich snippets in the Google search results.
Google's John Mueller said this in the Google webmaster hangout from this morning on YouTube at the 9:42 mark.
The question was:
If my site regains rich snippets review stars, it lost them after a while, does that mean from the quality algorithms, all the algorithms think that this site is awesome or can it still be that the site is hit by Panda or other quality algorithms despite the fact that it has regained rich snippets?
It doesn’t really mean that your website can’t be affected by any other algorithms.
Essentially for rich snippets, we do take into account the quality of the web site to some extent but we also take into account wether or not these snippets are set up correctly, they are complying with our policies, all of that.
So that is something that just because we are showing rich snippets doesn’t mean that we think that this web site is above everything else, is fantastic and perfect. There can still be parts of the web site where, our algorithms or some of our algorithms, are thinking mehh, maybe this is something not so perfect.
So it sounds like you are on the right track but I wouldn’t lay back and say I don’t have to do anything more with my web site, it’s perfect the way it is. And continue working on this, this is something in general all web sites are dynamic and keep living so you can’t just stop working on it.
Yes, I have heard SEOs say, if a site has rich snippets, it means Google thinks the site is high quality. I guess that is not 100% true.
Here is the video embed:
Forum discussion at Google+.