What Cloudflare Actually Checks

Cloudflare is the CDN behind a significant fraction of web traffic. When scraping operations encounter Cloudflare, they encounter one of several distinct product configurations: the free tier with basic DDoS protection, Cloudflare Pro or Business with Managed Challenge capability, or Cloudflare Bot Management — an enterprise product with a substantially more sophisticated detection stack. These are not the same system. A technique that passes the free tier fails on Bot Management. A proxy that succeeds on Managed Challenge fails on a site with custom WAF rules that blocklist specific ASNs.

Identifying which Cloudflare product a site uses determines what the detection stack actually evaluates — and therefore which tools are required to operate against it.

The free Cloudflare tier applies IP reputation scoring against Cloudflare's threat intelligence database and serves basic JavaScript challenges to requests that score above a threshold. The IP reputation check covers known bad actors, IP ranges associated with DDoS attacks, and commercial proxy ASNs that Cloudflare has flagged. Clean residential IPs typically pass this tier without challenge. Datacenter IPs from providers with abuse history on Cloudflare's network may be challenged or blocked.

Cloudflare Managed Challenge — available on Pro and above — serves a challenge that includes JavaScript execution requirements and browser environment verification. The challenge evaluates whether the client can execute JavaScript, checks for headless browser indicators, and validates that the execution environment matches real browser behavior. HTTP-based scrapers fail this challenge because they cannot execute JavaScript. Headless browsers that don't disable standard headless detection markers also fail.

Cloudflare Bot Management — enterprise product — adds TLS fingerprinting, behavioral analysis, and machine learning classification to the detection stack. It maintains fingerprint databases of known bot clients and human browsers. It evaluates inter-request timing, navigation sequences, and resource loading patterns across the session. A bot session that passes IP reputation and executes JavaScript correctly may still be classified as automated on behavioral signals accumulated across 10–20 requests. This is the product tier that closes most workarounds that succeed against lower tiers.

A request reaching a Cloudflare-protected endpoint passes through evaluation layers in sequence. IP reputation is checked first — requests from IPs on known threat lists or blocked ASN ranges receive an immediate response before further evaluation. Requests that pass IP reputation proceed to TLS fingerprint evaluation — the ClientHello is inspected and the fingerprint is scored against known browser and bot fingerprint databases. Requests that pass TLS evaluation proceed to the challenge layer if behavioral or environmental scoring warrants it.

The challenge layer presents a JavaScript challenge to the client. The challenge is invisible to users with full browser support — it executes and passes automatically. Clients that cannot execute JavaScript, execute it in a detectable headless environment, or fail browser API availability checks receive a challenge page that requires human interaction (CAPTCHA) or fail the challenge entirely. The challenge result feeds into the session's ongoing risk score.

After challenge, requests proceed to custom WAF rules the site operator has configured. These rules operate on request content, headers, geographic origin, and ASN classification — and are entirely site-specific. A residential IP that passes all Cloudflare's own detection layers may be blocked by a custom WAF rule the site operator wrote specifically to block residential proxy ASNs, or IPs from specific geographic regions, or requests with specific header patterns.

Residential proxies with HTTP-based scrapers pass IP reputation on most Cloudflare tiers but fail Managed Challenge because HTTP clients cannot execute JavaScript. The challenge is not a CAPTCHA requiring human interaction — it is a programmatic verification that the client environment is a real browser. HTTP clients that receive the challenge page and attempt to parse it or submit the challenge token without executing the embedded JavaScript produce a response that the validation system rejects immediately.

Headless browsers pass JavaScript execution but may fail headless detection embedded in Cloudflare's challenge script. Cloudflare's challenge code probes for browser APIs, canvas fingerprinting signatures, WebGL renderer strings, and navigator property values that differ between headless and headed browser environments. Standard Puppeteer and unmodified Playwright installations expose headless indicators that Cloudflare's challenge detects. Patched headless configurations that mask headless indicators have a variable success rate that changes as Cloudflare updates its detection logic.

Error 1020 — Access Denied — indicates a custom firewall rule block, not a Cloudflare detection system decision. The rule was configured by the site operator. It may target the specific proxy IP, the proxy's ASN block, the requesting country, or a header pattern. A 1020 from a residential IP means the site operator has explicitly blocked that IP or its subnet. Switching to a different IP from the same pool likely hits the same rule. The fix requires understanding what the rule targets.

Against free tier Cloudflare: clean residential proxies combined with HTTP-based scrapers with browser-consistent headers are typically sufficient. IP reputation is the primary detection layer; behavioral and TLS evaluation are minimal. This covers a large fraction of Cloudflare-protected sites — most sites use the free tier.

Against Managed Challenge: JavaScript execution is required. Headless browser automation that passes headless detection is the appropriate tool. TLS fingerprinting from the browser engine is correct by default; behavioral signals from a headless browser without interaction simulation may accumulate over multiple sessions. For many Managed Challenge targets, headless browsers with standard stealth configurations pass at acceptable rates.

Against Bot Management: the full stack is required — residential proxies for IP classification, browser-matched TLS for fingerprinting, headless browser with instrumented human interaction simulation for behavioral and challenge evaluation. This is the most resource-intensive configuration and appropriate only for targets where Bot Management is confirmed active. Applying this configuration to a free-tier target is unnecessary overhead.