Any URL → clean markdown. Strips nav, cookie banners, ads, and JS cruft. A 50k-token HTML page becomes 2–5k tokens. Your agent spends less, learns more.
Every token your agent wastes parsing a cookie banner is a token you pay for. CLEANING fetches any URL, strips the junk, and returns clean markdown — so your agent gets the content, not the clutter.
Strips navigation bars, footers, sidebars, cookie consent banners, ads, tracking pixels, and social widgets. What's left is the actual content.
Every response includes a token estimate. Your agent can decide whether to fetch the full page or just the summary — before spending a cent.
Same URL requested 100 times = 1 fetch. 1-hour TTL per URL with LRU eviction. Cached responses return in under 50ms.
Clean markdown, structured JSON, MCP-compatible. No HTML parsing. No guesswork. Ready for your context window.
CLEANING uses trafilatura — the state-of-the-art open-source content extraction library. No proprietary black boxes. No API keys. Just clean, structured output from any URL.
Python library for web content extraction. Handles 95%+ of web pages correctly. Extracts main content, strips boilerplate, preserves structure (headings, lists, links). Apache 2.0 license. The gold standard for open-source extraction.
Modern async HTTP client. Follows redirects (up to 5 hops), respects robots.txt, handles gzip/brotli decoding. Fetches the raw HTML that trafilatura processes.
Local cache with 1-hour TTL per URL. LRU eviction when the cache fills. Same URL from any agent = one fetch, many serves. Reduces load on target sites and speeds up responses.
Local inference server for LLM-enhanced extraction. Summarization, field extraction, and token-budget compression. Zero marginal cost — runs on our own hardware. Paid tier only.
Fetch any URL and return clean markdown. Free tier: 25/day. Paid: up to 500/day.
curl "https://cleaning.4bots.ai/v1/free/clean?url=https://example.com"
Example response:
{
"url": "https://example.com",
"title": "Example Domain",
"cleaned_markdown": "# Example Domain\n\nThis domain is for use in illustrative examples...",
"token_estimate": 847,
"model": "trafilatura",
"cached": false,
"free_remaining": 24
}
Include a valid payment header for rule-based or LLM-enhanced tier. Same endpoint, upgraded output.
curl -H "PAYMENT-SIGNATURE: <base64-payload>" \ "https://cleaning.4bots.ai/v1/free/clean?url=https://example.com"
Health check. Always free, always returns 200.
curl "https://cleaning.4bots.ai/health"
25 free requests per day covers development and light agent workloads. Paid tiers unlock higher limits, JS-rendered pages, and LLM-enhanced extraction.
No signup. No API key. Just send a request.
How x402 works