SaaS Browser Bot - Crawler Information

SaaS Browser Bot - Crawler Information

SaaS Browser is a free and open search engine dedicated to helping users discover new and exciting SaaS products and software solutions. Our platform makes it easy to explore the ever-growing world of software-as-a-service offerings, completely free to search with no account or signup needed.

Getting your SaaS product listed on SaaS Browser helps potential customers discover your solution when they're actively searching for software. Being indexed in a dedicated SaaS search engine puts your product in front of users who are specifically looking for tools like yours, targeted traffic from decision-makers ready to evaluate new solutions. Best of all, it's completely free. If our crawler can access your site, you're likely already getting this exposure.

Want more control? You can claim your profile, upload screenshots, and display your product in its best light. Just make sure your product pages clearly showcase what makes your SaaS unique, and we'll help people find you.

If our bot doesn't find your software product, its free and easy to submit it for inclusion (signup is required).


About Our Crawler

The SaaS Browser Bot crawls the web to discover and index SaaS products, software tools, and related content to provide comprehensive search results for our users. We are committed to responsible crawling practices and respect for webmaster preferences.

User Agent String

Our bot identifies itself with the following user agent:

  1. Mozilla/5.0 (compatible; SaaSBrowserBot/1.0; +https://saasbrowser.com/bot)

Respecting Your Preferences

robots.txt Compliance

We fully respect and honor robots.txt directives. Our bot checks your robots.txt file before crawling any pages on your domain. To block our bot, add the following to your robots.txt file:
  1. User-agent: SaaSBrowserBot
  2. Disallow: /

Meta Robots Tags

We respect the noindex directive in HTML meta tags:

  1. <meta name="robots" content="noindex">

HTTP Headers

We also honor the `X-Robots-Tag` HTTP header:

  1. X-Robots-Tag: noindex

Please note that by adding any noindex tag to your website, will result in removing from our index of SaaS products.

Crawl Behavior

- Maximum 1 request per second per domain
- Typical crawl frequency: once every 30 days for updated content, to check if your website is still online or to check if you wish for us to remove your site from our index.
- Only makes requests over HTTPS with a valid SSL certificate.

What We Collect

We Index:
- Product names and descriptions
- Pricing information (when publicly available)
- Features and capabilities
- Company information
- Public reviews and testimonials

We Do NOT Collect:
- Personal user data
- Login credentials
- Private or gated content
- Content behind paywalls

Verifying Our Bot

To verify that a request is genuinely from SaaS Browser Bot and not a malicious scraper spoofing our user agent, you can perform a two-step DNS verification process. First, do a reverse DNS lookup (PTR record check) on the IP address making the request. The hostname should resolve to crawler.saasbrowser.com or *.saasbrowser.com. Second, perform a forward DNS lookup on that hostname to ensure it resolves back to the original IP address. This forward-confirmed reverse DNS check confirms authenticity.

For example:
  1. ~/$ nslookup 45.55.138.37
  2. 37.138.55.45.in-addr.arpa    name = crawler2.saasbrowser.com.

  3. ~/$ nslookup crawler2.saasbrowser.com
  4. Non-authoritative answer:
  5. Name:    crawler2.saasbrowser.com
  6. Address: 45.55.138.37

Privacy & Data

We are committed to protecting privacy and handling data responsibly. For information about how we handle indexed data and user privacy, please see our Privacy Policy.

Questions? Feel free to reach out to us at hello@saasbrowser.com

    • Related Articles

    • SaaS Browser Data

      SaaS Browser was built to solve a growing problem: discovering new and innovative SaaS products has become increasingly difficult. Traditional search engines often bury emerging tools beneath well-funded competitors and sponsored results. Our mission ...
    • How can I add my SaaS to SaaS Browser?

      Our bot, SaaS Browser Bot, often finds SaaS on its own by crawling the web. However, in some case's we may miss your SaaS, or we might determine that it isn't an ideal fit for our index. Read this article to see more information about how we source ...
    • How SaaS Browser is Monetized

      At SaaS Browser, we believe in transparency, not just in how we list and categorize SaaS products but also in how we sustain and grow the platform. This article explains how we monetize our operations while staying true to our mission of making SaaS ...
    • How can I remove my SaaS from SaaS Browser?

      At SaaS Browser, our primary goal is to enhance the discovery of great SaaS businesses. We come from a background of building and growing SaaS, and getting a SaaS off the ground isn't easy. We gather data through web crawling, similar to how Google ...
    • Which types of SaaS are added to SaaS Browser?

      We welcome any online service where the product is delivered automatically and does not rely on heavy, one‑to‑one human work. If users sign up, push a button, and get value without waiting for someone on the team to step in, the product fits our ...