# This robots.txt file disallows all engines/crawlers/indexers by default. # Global directives, but not always obeyed. User-agent: * Disallow: / # Named directive(s) specifically disallowing the most common engines/crawlers/indexers. User-agent: Mediapartners-Google User-agent: Google-Display-Ads-Bot User-agent: adsbot-google-mobile User-agent: Googlebot User-agent: Bingbot User-agent: Slurp User-agent: DuckDuckBot User-agent: YandexBot User-agent: facebot User-agent: Applebot # Prevent the indexing of any sites that are no longer active but still accessible online. Disallow: /*-del