MaterialFilterBot
MaterialFilterBot is the web crawler operated by Mikoshi d.o.o. (Ljubljana, Slovenia) for MaterialFilter.com, a material-first search engine for natural-fiber clothing. It indexes public product information — fiber composition, price, and availability — so shoppers can find clothes by what they are actually made of. Every listing links back to the brand's own store.
What it fetches
MaterialFilterBot fetches public product feeds and public product pages — the same pages any visitor or search engine can already see. It never requests user accounts, checkout or cart flows, personal data, or anything behind a login or paywall.
How it behaves
- Identifies itself on every request with the User-Agent
MaterialFilterBot/1.0 (+https://materialfilter.com/bot), from a consistent set of IP addresses. - Crawls slowly — about 1.5 requests per second per site — and backs off when a server returns 429 or 5xx responses.
- Reads and honors your robots.txt at the start of every crawl.
How to opt out
You can tell MaterialFilterBot to stop in either of two ways.
robots.txt (honored automatically). Add the following to your site's robots.txt. It is checked at the start of every crawl, and a match stops the crawl of the disallowed paths immediately. Path-specific rules are honored too, so you can disallow only part of your site.
User-agent: MaterialFilterBot Disallow: /
Email. Prefer to just ask? Write toinfo@materialfilter.com. We delist any brand within 48 hours of a request, no questions asked.
Run a brand?
We would rather work with you than around you. We can list your full catalog with verified composition data, correct any entry that is wrong, or set up a direct feed so your products stay up to date. Email us atinfo@materialfilter.com.
Last updated: 24 July 2026