A summary of the Buck Internet robot. Including details for the owner, description, HTTP user agent and whether this robot adheres to the robot exclusion standard.
Who owns the Buck robot? Is it a good or a bad robot? And why is it visiting your website?
Shown below is a sample log file entry for the Buck web robot. It’s derived from an Apache web server log file. From the log entry information about how the robot identifies itself, HTTP User Agent, and where it is hosted are given.
Server Log File
www.vntweb.co.uk 107.23.132.64 - - [07/Jan/2020:18:38:17 +0000] "GET /robots.txt HTTP/1.1" 200 114 "-" "Buck/2.2; (+https://app.hypefactors.com/media-monitoring/about.html)"
HTTP User Agent
Buck/2.2
IP Addresses
The observed IP address was 107.23.132.64.
WHOIS DNS command gives the following information about the IP address:
| NetRange: | 107.20.0.0 – 107.23.255.255 |
| NetName: | AMAZON-EC2-8 |
| OrgName: | Amazon.com, Inc. |
| Address: | Amazon Web Services, Inc. |
| Address: | P.O. Box 81226 |
| City: | Seattle |
| StateProv: | WA |
| PostalCode: | 98109-1226 |
| Country: | US |
| Updated: | 2019-07-24 |
As can be seen from the above the observed IP address is a part of a block assigned to Amazon EC2.
Owner
Hypefactors
Country
Denmark
Exclusion
The user-agent string includes a reference to the website https://app.hypefactors.com/media-monitoring/about.html
The referenced website confirms that the bot follows all established policies and etiquette on crawling, with a reference to a Wikipedia page: https://en.wikipedia.org/wiki/Web_crawler#Politeness_policy.
The Hyperfactors website does not provide details about how their robot complies with the robots.txt exclusion standard, which was described at http://www.robotstxt.org/wc/exclusion.html#robotstxt, but is not currently available. Information is available on the same website https://www.robotstxt.org/robotstxt.html and also on the w3c website at https://www.w3.org/TR/html4/appendix/notes.html#h-B.4.1.1
The referenced web page includes a form for a request to exclude the buck web robot from visiting a website.
With no advice as to how to include a custom entry in the robots.txt file to prevent Buck from visiting your site , you may wish to try
User-agent: Buck Disallow: /
You may wish to trial controlling the frequency of Buck visiting your site, setting a minimum acceptable delay between consecutive requests can be set with the following added to the robots.txt file:
User-agent: Buck Crawl-Delay: 10
In this example the delay has been set to 10 seconds.
As is common with website crawlers there is a delay between changes made to the robots.txt file and the change being implemented.
Take care making changes to the robots.txt file. A misunderstanding in configuration or an error in configuration can lead to important search engines excluding your website.
Further Info
Hypefactors aims to help organisations to have a better media impact. To do this they have a number of tools to automate and help with the work.
The platform offers the following areas
- Monitor
- Measure
- Report
- Connect
- Create
- Dashboard
Looking at the sequence of testimonial comments Hypefactors data is used in ways such as:
- to provide an overview of the daily press coverage of a company
- feed back on the efforts of PR and communications
