RewaltoBot
If RewaltoBot is in your logs, this page is what it points at. What it fetches, how often, and the two lines that stop it.
- USER AGENT
- RewaltoBot
- REQUEST RATE
- 1 / second / host
- IP RANGES
- [NOT PUBLISHED YET]
Everything you need to identify it, and to stop it.
| Fact | What it means for your server |
|---|---|
| RewaltoBot | The product token. This is the word to put after User-agent: in robots.txt; the version and URL that follow it on the wire are not part of the match. |
| RewaltoBot/1.0 | The full agent string is RewaltoBot/1.0 (+https://rewalto.com/bot). The version moves with the crawler that produced a capture, so two runs under different versions are not assumed comparable. |
| What it fetches | Publicly reachable pages of a site we were given as a target. Nothing behind a login, nothing it was not pointed at, and no form is ever submitted. |
| How often | One request a second to a host, plus up to half a second of jitter, whatever else is queued. The floor is per host rather than per crawl, so running two crawls does not double the rate you see. |
| robots.txt | Fetched once per run, obeyed, and kept. A Disallow for RewaltoBot or for * stops the fetch before it is made, and the run records the file it obeyed so a later question about why a page was skipped is answered from the capture rather than a log. |
| crawl-delay | Read and honoured, since 4 September 2026 — this row said not read until then, and said it for the reason below. Our matcher is Google's, which does not implement the directive, so we parse it ourselves. A crawl-delay aimed at RewaltoBot or at * becomes the wait between our requests to your server. There is no ceiling on it. If you ask for one request an hour, you get one request an hour, however long that makes our crawl. The one-second floor still applies underneath: a value below it does not make us faster. A group naming us does not cancel a crawl-delay you set for * — if you asked everyone to wait, we wait, even where you named us for something else. |
| Response size | Capped at 8 MB. Anything larger is abandoned rather than stored, so a large media file costs you one truncated request and nothing else. |
| IP ranges | [NOT PUBLISHED YET] |
| Where to complain | security@rewalto.com |
It reads pages. That is the whole of it.
What it is for.
Reading a site the way a search engine or an assistant would, so we can say what is missing. Titles, headings, structured data, internal links and what a page states about itself.
What it will not do.
It does not log in, submit a form, add to a cart, or write anything anywhere. It follows links and reads pages, and a page it was told not to fetch is not fetched.
How to turn it off.
Two lines in robots.txt: User-agent: RewaltoBot, then Disallow: /. It takes effect on the next run, because the file is read once per run rather than cached across them.
A crawl of your site is something you asked for. RewaltoBot runs against a property a client named, and the first runs are started by a person rather than a schedule. If it reached you and you did not expect it, the Disallow above is the immediate answer and the contact route is the one we owe you.
Ask us to crawl your site
It joins the same speculative crawl every other storefront in the index goes through. We do not fetch it on request — a public page that fetched on demand would be a security risk.
Ask us to stop crawling your domain
This does not delete what we already hold, and it is not instant — the fetcher checks this before its next crawl of the domain, not mid-crawl. A Disallow in your robots.txt does the same thing and takes effect the same way.
See what it finds on your own site.
Find out whether AI assistants and Google recommend you, with a ranked list of fixes.