Agent setup

Berrycrawl publishes machine-readable discovery assets for agents:

• /llms.txt describes the product and important documentation entry points.
• /openapi.agent.json contains the public data plane plus agent registration and claim operations.
• /.well-known/oauth-protected-resource and /.well-known/oauth-authorization-server describe the protected API and authorization service.
• /api/v1/mcp exposes the hosted Streamable HTTP MCP server. See MCP server.

Recommended sequence

1. Read the agent OpenAPI document.
2. Register an agent workspace through the documented agent route.
3. Store the returned API key as a secret.
4. Test against berrycrawl.com; that domain does not consume credits.
5. Give the claim URL to the human operator.
6. Continue using the same workspace after the human accepts ownership.

Claim links are credentials. Send them only to the intended owner and do not place them in public logs or model transcripts.

Tool design

Expose narrow tools to your model: scrape_page, map_site, start_crawl, get_crawl, and get_brand. Validate URLs before invoking them, set explicit limits, and return Berrycrawl's failure reason to the model instead of replacing it with guessed content.

For most agent reading tasks, request Markdown with onlyMainContent: true. Use mapping before crawling when the relevant page is unknown. Use maxAge to reuse recent results within one reasoning loop.