User-agent: AhrefsBot Crawl-Delay: 5 User-agent: * Allow: / Disallow: /account/ Disallow: /demopages/ Disallow: /API/ Disallow: /handler/ Disallow: /restaurants/review/add/ Disallow: /restaurants/search-help/ Disallow: /cdn-cgi/