Reliability and proxy escalation

Normal text scrapes first try a lightweight direct fetch. Pages that require rendering continue through Patchright direct. Berrycrawl escalates to IPRoyal only after detecting an access block, challenge, rate limit, or retryable upstream error. Bright Data is the final, single-attempt rescue provider when it is configured.

The active paid fallback is IPRoyal rotating residential access. Proxy credentials stay in backend configuration and are never exposed through the API or dashboard.

Why escalation matters

Proxy bandwidth is a variable infrastructure cost. Sending every request through residential IPs would increase cost and can reduce speed. Direct fetching and rendering handle the majority of sites; residential traffic is reserved for the fraction that needs another network identity. Proxy escalation is included in the flat scrape price and does not create a customer surcharge.

Responses and internal metrics record:

• fetch tier;
• proxy tier and provider;
• request and response transfer bytes;
• retries and block reason;
• delayed hedge attempts;
• cold and cached latency.

That telemetry feeds benchmark cost reports and future pricing decisions. It does not change the response content.

Clean failure

When proxies are disabled or unavailable, a blocked site returns a failure. Berrycrawl will not cache a CAPTCHA, Cloudflare challenge, generic error page, empty document, or HTTP error as valid page data.