# /robots.txt for webcrawler.com frontend machines User-agent: * SITEMAP: http://www.medhelp.org/MHIsitemap_index.xml Disallow: /mhli/ Disallow: /mhli/cgi/ Disallow: /mhli/data/ Disallow: /MessageControl/ Disallow: /search Disallow: /people/search Disallow: /watches Disallow: /invite_friend/ Disallow: /user_trackers/graph Disallow: /noisy_image Disallow: /private_messages Disallow: /NJGroups Disallow: /account/set_mobile Disallow: /notes/new