Agents
Give a coding agent a server-side Berrycrawl key and a clear integration path.
Berrycrawl publishes machine-readable discovery assets for agents:
/llms.txtdescribes the product, endpoint map, and documentation entry points./openapi.agent.jsoncontains the public API contract./api/v1/mcpexposes the hosted Streamable HTTP MCP server. See MCP server.
Recommended sequence
- Read the Agent quickstart.
- Inspect the target repository's language, framework, server entry points, and existing env conventions.
- Tell the human to create a Berrycrawl key and add
BERRYCRAWL_API_KEY=bc_...to the server environment. - Never ask the human to paste the key into chat or commit it.
- Add the smallest server-side wrapper needed for the requested endpoint.
- Test against
berrycrawl.com; that domain does not consume credits.
The key is the only setup credential an agent needs. Berrycrawl still authenticates every data-plane request with Authorization: Bearer $BERRYCRAWL_API_KEY.
Tool design
When you use the hosted MCP server, expose only the tools the model needs. Common choices are berrycrawl_scrape_url, berrycrawl_map_site, berrycrawl_start_crawl, berrycrawl_get_job, and berrycrawl_get_brand. Validate URLs before invoking them, set explicit limits, and return Berrycrawl's failure reason to the model instead of replacing it with guessed content.
For most agent reading tasks, request Markdown with onlyMainContent: true. Use mapping before crawling when the relevant page is unknown. Use maxAge to reuse recent results within one reasoning loop.