# all crawlers User-agent: * # disallow nothing; i.e. allow all pages to be crawled Disallow: