User-agent: htdig
Disallow: /
User-Agent: *
Allow: /