Google's John Mueller posted on Google+ an overview of how to remove your staging server from Google and then the best way to block it going forward. Some call it a staging server, some a development server, some a testing server or some have all of these three and more. Heck, we even have pentest servers here, so we need to block them carefully from search engines and open them to those that need access.
First John explained that if these servers do find their content indexed by Google, the fastest way to remove the content is via Google Search Console. Of course that means you need to verify the staging URL with Google Search Console, but that is only long enough until you do the remove content method. After that, you then want to close the site from the outside world.
John wrote:
The fastest way to get the staging site removed from search is remove it via Search Console. For that, you need to verify ownership via Search Console [1] (ironically, this means you'll likely have to make it accessible to search engines again, or figure out DNS verification, which isn't that common but also not that hard). From there, you can do a site-removal request [2], which will take the whole hostname out of Google's search for ca 90 days. During this time, you can figure out and implement your general plan to block the staging site from search.
Then going forward, you want to lock down access to the server. John shared these tips, i.e. blocking on the server side with HTTP authentication or IP address whitelisting. He said all the other alternatives he does not like. John wrote:
My recommendation for staging sites is to block access on the server side, either with server-side / HTTP authentication [3] or IP address whitelisting (IP addresses can change, and this would block you from using tools from home, etc, so it's worth being cautious there and whitelisting rather than blacklisting).
He shared more details on Google+ adding that it's normal for URLs to not be recrawled in months, so if you add any block on the URL level, it can easily take a half year or longer to be fully processed for all URLs." "The site-removal request gives you most of that time, and you can submit another one should you need to extend it," he said.
Forum discussion at Google+.
