Data and security

Berrycrawl accepts public web URLs. Do not use the service to reach private infrastructure, cloud metadata endpoints, loopback addresses, or credentials embedded in URLs. URL validation and redirect checks reject private-network targets.

Secrets

• Store bc_ keys only on trusted servers.
• Keep Stripe, Cloudflare, AI, proxy, and database credentials out of frontend builds.
• Commit .env.example files with names and safe placeholders, never real values.
• Rotate a credential if it appears in a log, screenshot, issue, or model transcript.

Artifacts

Persisted screenshots, brand logos, and backdrops are stored in a general Cloudflare R2 bucket and served through cdn.berrycrawl.com. Downloads are bounded by MIME, byte, pixel, timeout, redirect, and concurrency limits before storage.

Content retention

Cache age depends on the endpoint and caller preference. Brand full profiles default to 90 days; scrape cache age is controlled by request options and server policy. A forced refresh does new upstream work. Failed and blocked content is not written as a successful cache entry.

If a workflow handles regulated, personal, or contract-restricted content, apply your own authorization, minimization, and retention rules before sending URLs or storing results.