https://hobops.io/en/Recent content on HOBOPSHugoenhttps://hobops.io/en/crawler/Mon, 01 Jan 0001 00:00:00 +0000https://hobops.io/en/crawler/<p><strong>HobopsResearchBot</strong> is a polite crawler HOBOPS uses for market research in Mexico. It only visits each domain&rsquo;s public homepage, respects <code>robots.txt</code>, and identifies itself on every request.</p> <h2 id="request-removal-self-service">Request removal (self-service)</h2> <p>Enter your domain and <strong>we stop connecting to your site</strong> immediately: no new connections or crawling. Subdomains are covered too.</p> <form class="optout-form" data-lang="en" style="max-width:32rem;margin:1rem 0;display:flex;flex-direction:column;gap:.6rem"> <input type="text" name="domain" placeholder="example.com" required style="padding:.6rem;border:1px solid #ccc;border-radius:.4rem;font-size:1rem"> <input type="email" name="email" placeholder="Contact email (optional)" style="padding:.6rem;border:1px solid #ccc;border-radius:.4rem;font-size:1rem"> <input type="text" name="reason" placeholder="Reason (optional)" style="padding:.6rem;border:1px solid #ccc;border-radius:.4rem;font-size:1rem"> <input type="text" name="website" tabindex="-1" autocomplete="off" style="position:absolute;left:-9999px" aria-hidden="true"> <button type="submit" style="padding:.6rem;border:0;border-radius:.4rem;background:#2b6cb0;color:#fff;font-size:1rem;cursor:pointer">Exclude my site</button> <p class="optout-msg" role="status" style="margin:.3rem 0 0"></p> </form> <h2 id="other-options">Other options</h2> <ul> <li><strong>robots.txt</strong> — add <code>User-agent: HobopsResearchBot</code> and <code>Disallow: /</code>.</li> <li><strong>Email</strong> — write to <strong><a href="mailto:apps@hobops.io">apps@hobops.io</a></strong> with the domain to exclude.</li> </ul> <p>We do not use proxies or IP rotation to evade blocks.</p>