User-agent: Slurp
Crawl-delay: 0.5
User-Agent: *
Allow: /