MarketNow Registry / Developer Tools / awesome-ai-web-scraping

awesome-ai-web-scraping

A curated list of AI-powered web scraping tools, LLM-friendly crawlers, MCP servers, and infrastructure for turning the web into data.

Developer Tools v1.0.0 by h4ckf0r0day

Trust score

40/100

Install risk

Review advised

Source

github

Indexed

2026-09-10

Install

$ git clone https://github.com/h4ckf0r0day/awesome-ai-web-scraping

Heuristic signals advise review before installing: young package, low adoption, stale repository, or name similarity to a popular package.

How MarketNow scored this

Sentinel Index Heuristics — a security-first estimate from public signals:

Package age and release activity (stale repositories lose points)
Verified adoption: registry downloads or GitHub stars, not follower counts
Typosquat distance against the 150 most-installed MCP packages
Injection markers scanned in the package description

Heuristic, not a guarantee. For verified agent credentials and revocation, see the MarketNow Trust API.

What is awesome-ai-web-scraping?

A curated list of AI-powered web scraping tools, LLM-friendly crawlers, MCP servers, and infrastructure for turning the web into data. Indexed by MarketNow with trust 40/100 (Review advised).

How do I install it?

Run $ git clone https://github.com/h4ckf0r0day/awesome-ai-web-scraping Heuristic signals advise review before installing: young package, low adoption, stale repository, or name similarity to a popular package.

What does the trust score mean?

Security-first heuristic over public signals (age, adoption, typosquat distance, injection markers). It is not a popularity ranking and not a guarantee of safety. awesome-ai-web-scraping scores 40/100.