# This is a file retrieved by webwalkers a.k.a. spiders that # conform to a defacto standard. # See User-agent: * Disallow: Crawl-delay: 2