How to prevent crawling of non-public ACSF domains

Until a platform-wide solution is in place, the best practices approach is to add the following code to your codebase:

In the `.htaccess` file in the docroot, add

# Ensure non-live sites prevent robots from crawling. This is similar to what
# Acquia Cloud does for the *.acquia-sites.com domains.
RewriteCond %{HTTP_HOST} \.acsitefactory\.com$ [NC]
RewriteCond %{REQUEST_URI} /robots.txt [NC]
RewriteRule ^ no_robots.txt [L]

And add a file called `no_robots.txt` to the docroot with the following contents:

#
# no_robots.txt
#
# This file prevents indexing of domains in the acsitefactory.com DNS zone
#
# See also: https://docs.acquia.com/cloud/configure/domains#robots
#
User-agent: *
Disallow: /