A summary of the coccocbot-web Internet robot. Including details for the owner, description, HTTP user agent and whether this robot adheres to the robot exclusion standard.
Who owns the coccocbot-web robot? Is it a good or a bad robot? And why is it visiting your website?
Shown below is a sample log file entry for the coccocbot-web web robot. It’s derived from an Apache web server log file. From the log entry information about how the robot identifies itself, HTTP User Agent, and where it is hosted are given.
Server Log File
vntweb.co.uk 103.131.71.193 - - [22/Apr/2019:12:44:31 +0100] "GET /mailchimp-modify-line-spacing/ HTTP/1.1" 200 10263 "-" "Mozilla/5.0 (compatible; coccocbot-web/1.0; +http://help.coccoc.com/searchengine)"
HTTP User Agent
coccocbot-web/1.0
IP Addresses
The observed IP address was 103.131.71.193.
WHOIS DNS command gives the following information about the IP address:
| inetnum: | 103.131.68.0 – 103.131.71.255 |
| netname: | COCCOC-VN |
| descr: | Coc Coc Company Limited |
| descr: | Room No. 1202, 12th Floor, HITTC Building, 185 Giang Vo, Cat Linh, Hanoi |
| country: | VN |
| last-modified: | 2018-10-17T09:09:09Z |
As can be seen from the above the observed IP address is a part of a block assigned to Coc Coc.
nslookup DNS command gives
193.71.131.103.in-addr.arpa name = bot-103-131-71-193.coccoc.com.
Owner
Cốc Cốc a search engine company.
Country
Vietnam
Exclusion
The user-agent string includes a reference to the website http://help.coccoc.com/searchengine
The referenced website confirms that the bot supports the robots exclusion text and also obeys the crawl delay.
The Cốc Cốc website provides details about how their robot complies with the robots.txt exclusion standard, which was described at http://www.robotstxt.org/wc/exclusion.html#robotstxt, but is not currently available. Information is available on the same website https://www.robotstxt.org/robotstxt.html and also on the w3c website at https://www.w3.org/TR/html4/appendix/notes.html#h-B.4.1.1.
Details are given about both preventing the robot from indexing the website and how to adjust its crawl rate.
Their advice is to include the following entry in the robots.txt file to prevent coccocbot-web robot from visiting the directory cgi-bin.
User-agent: coccocbot-web Disallow: /cgi-bin/
Also to control the frequency of coccocbot-web visiting your site, setting a minimum acceptable delay between consecutive requests can be set with the following added to the robots.txt file:
User-agent: coccocbot-web Crawl-Delay: 10
In this example, taken from their website the delay has been set to their maximum value of 10 seconds.
As is common with website crawlers there is a delay between changes made to the robots.txt file and the change being implemented.
Take care making changes to the robots.txt file. A misunderstanding in configuration or an error in configuration can lead to important search engines excluding your website.
Further Info
Cốc Cốc is an internet search company based in Vietnam
On their website the company claims that it is “one of the biggest local Browser and Search Engine companies in Vietnam as well as the biggest Vietnamese digital advertising publisher with more than 22 million users“.
