User-agent: * Disallow: /*&DCMP=* Disallow: /*?DCMP=* # Don't spider the results past the second page Disallow: /Pages/JobResults.aspx?*page=* Allow: /JobResults.aspx?*page=2 User-Agent: OmniExplorer_Bot Disallow: /