# robots.txt file for http://www.idg.no # please contact staff@idg.no with questions or concerns # "Any empty [Disallow] value, indicates that all URLs can be retrieved. # At least one Disallow field needs to be present in a record." # Define for Mediapartners @ google User-agent: Mediapartners-Google* Disallow: # Define for Google User-agent: Googlebot Disallow: # Define for Yahoo! Slurp User-agent: Slurp Disallow: Crawl-delay: 1 # Define for Fast User-agent: FAST Disallow: Crawl-delay: 2 # Define for MSN User-agent: msnbot Disallow: /eceRedirect Crawl-delay: 2 # Define for WebTrends User-agent: WebTrends Disallow: /eceRedirect Crawl-delay: 2 # Define for all User-agent: * Disallow: /multimedia/archive/ Disallow: /eceRedirect Crawl-delay: 30