Our scanner
BerryCookieScanner
The automated browser of BerryCookie, the cookie consent platform (CMP).
If you found BerryCookieScanner/1.0 (+https://berrycookie.com/scanner) in your server logs, this page explains who sent it, what it did and how to stop it.
Who runs it
BerryCookieScanner is the browser of BerryCookie, a consent management platform. It visits a website for two reasons:
- someone entered the address in the free website check, to see what the site does before visitors consent;
- the site's owner uses BerryCookie and asked for a scan of their own website (from their dashboard, or the monthly scan of their cookie declaration).
What it does on a free check
- One visit per request. Nothing is crawled on a schedule; each check is started by a person.
- Up to 5 public pages: the address entered and up to four links of the same site found on it. It skips sign-in, account, cart, checkout and admin pages and file downloads.
- No login, no forms. It never signs in, never fills in or submits a form and never buys anything.
- Only the consent banner's buttons. After looking at the pages without consent, it clicks the banner's "Reject" button and, in a separate fresh visit, its "Accept" button, to see what the site does after each choice. It clicks nothing else.
- It records what any visitor's browser sees: cookies, the addresses the pages contact (shortened to the host and path, without query strings), Google Consent Mode settings and the banner's buttons.
- Time and rate limits: at most about 90 seconds per check, a few checks at a time in total, and 5 checks an hour from the same visitor.
- A real browser from our servers (Chromium), through a filter that only reaches public addresses on the standard web ports.
What it does on a customer's own scan
- Only the customer's verified domains, up to the pages of their plan (or fewer, and only the paths they chose), started by them or once a month.
- The same rules: no login, no forms, a real browser that only reaches public addresses. On the home page it clicks the consent banner's "Reject" button and, in a separate fresh visit, its "Accept" button; it clicks nothing else and then follows the site's own links.
robots.txtand our exclusion list do not apply, because the owner asked for the scan of their own website.
The results
The result is private: only whoever started the check sees it, at a secret link. It is deleted after 30 days, together with the email address of anyone who asked for the report by email. We never publish lists of checked websites or their results.
How to opt out
With robots.txt. Add a group for our user agent to your site's robots.txt:
User-agent: BerryCookieScanner
Disallow: /The free check reads robots.txt before each check and does not check addresses that this group disallows (a path such as Disallow: /shop/ keeps it out of that part only). Rules for all crawlers (User-agent: *) do not apply to it, because each check is one visit asked for by a person, not a crawl; name it explicitly. If robots.txt cannot be read within a few seconds, the check goes on.
By asking us. Write to us through the contact page from an address of the website's domain, and we add the site to our exclusion list. The free check then refuses it for anyone, with the message "The owner of this site asked not to be checked by our scanner."
Neither way stops a BerryCookie customer from scanning their own website from their dashboard.
Questions about the scanner
Why did BerryCookieScanner visit my website?
Someone entered your address in the free website check, or the site’s owner uses BerryCookie and scanned their own website. Each free check is one visit started by a person; nothing is crawled on a schedule.
What does it do on my pages?
On a free check it opens up to 5 public pages as a new visitor, clicks only the consent banner’s Reject button and, in a fresh visit, its Accept button, and records what any visitor’s browser sees. It never signs in, fills in a form or buys anything, and a check takes at most about 90 seconds.
How do I stop it?
Add a group for it to your robots.txt: “User-agent: BerryCookieScanner” followed by “Disallow: /”. Rules for all crawlers (User-agent: *) do not apply to it. Or write to us from an address of your website’s domain, and we add the site to our exclusion list. Neither stops the site’s owner from scanning their own website.
How long are the results kept?
30 days. Only whoever started the check sees the result, at a secret link; then it is deleted, together with any email address given for the report. We never publish lists of checked websites.
Own a website and want it excluded?
Write to us from an address of the website’s domain and we add it to our exclusion list.