IndexPulseBot

Website status checker

Check if crawlers can reach a website: robots.txt on every version of the domain, the homepage, which bots are allowed, noindex, speed and the sitemap. Every result below is a real answer from the site — nothing is guessed.

Takes a few seconds. Results are kept for 10 minutes.

What our bot does

Users of LINKBOOSTER can let IndexPulseBot crawl their own website: it reads robots.txt and every sitemap (post, page and category sitemaps too), visits the pages, follows internal links to find posts that are not in a sitemap, and sends the URLs for indexing checks. Each crawl collects at most as many URLs as the user's plan allows. Every step is shown live in the user's account.

How it behaves

  • Identifies itself as Mozilla/5.0 (compatible; IndexPulseBot/1.0; +https://seobacklinkoworld.online/bot).
  • Respects robots.txt (user-agent token IndexPulseBot, falling back to *).
  • Waits at least 1 second(s) between requests to the same host.
  • Downloads at most 2 MB per page, never executes scripts and never submits forms.
  • Skips admin, cart, login and feed URLs, images and files.
  • Never impersonates Googlebot or any other crawler.

Blocking the bot

User-agent: IndexPulseBot
Disallow: /

Changes to robots.txt are picked up within one hour. If you believe the bot is misbehaving, contact us with the affected host and timestamps.

Crawl & index your site

Free accounts can crawl too — the bot collects as many URLs as your plan allows and sends them for indexing. You watch it work live.

Create a free account