# All robots will spider the domain User-agent: * Disallow: # Disallow directories and files User-agent: * Disallow: /images Disallow: /awards.asp Disallow: /runsearch.asp Disallow: /cgi-bin