(1)
Google's John Mueller said again that the best method to block Google from crawling and then indexing your staging site is to block GoogleBot and other humans with authentication. Prompt the access request to the staging site by requiring them to enter a valid username and password to access the staging environment.
I believe most developers and SEOs know this by now, but it is the best solution by far.
Here is John Mueller saying this on Twitter:
Using authentication (username/password) makes it so that you don't accidentally upload a blocking robots.txt or robots meta tag, and prevents the pages from being indexable. That's the best long-term solution for staging sites.
— 🍌 John 🍌 (@JohnMu) April 5, 2021
Just be careful with your staging sites, you don't generally want Google to index them and you don't want to accidentally push up to production the mechanism you use to block the on staging to production.
Forum discussion at Twitter.
