MCP server

Berrycrawl has a hosted MCP server at:

https://api.berrycrawl.com/api/v1/mcp

It uses stateless Streamable HTTP. Authenticate with the same bc_ API key you use for the REST API:

Authorization: Bearer bc_...

Create a key in Dashboard → API keys. The MCP server uses that key's organization, credit balance, rate limits, and usage history. It never asks for a separate MCP credential.

Test with MCP Inspector

The visual inspector is the quickest first test:

npx @modelcontextprotocol/inspector@latest

In the inspector:

1. Select Streamable HTTP.
2. Enter https://api.berrycrawl.com/api/v1/mcp.
3. Set the bearer token to your Berrycrawl API key.
4. Connect, open Tools, and call berrycrawl_get_credit_balance.

You can also list tools from the terminal:

export BERRYCRAWL_API_KEY="bc_..."

npx @modelcontextprotocol/inspector@latest --cli \
  https://api.berrycrawl.com/api/v1/mcp \
  --transport http \
  --method tools/list \
  --header "Authorization: Bearer $BERRYCRAWL_API_KEY"

Call the free Berrycrawl demo through the CLI:

npx @modelcontextprotocol/inspector@latest --cli \
  https://api.berrycrawl.com/api/v1/mcp \
  --transport http \
  --method tools/call \
  --tool-name berrycrawl_scrape_url \
  --tool-arg url=https://berrycrawl.com \
  --tool-arg response_format=json \
  --header "Authorization: Bearer $BERRYCRAWL_API_KEY"

Connect another AI app

Use these values in any app that accepts a remote MCP server:

If the app accepts an MCP JSON file, the server entry has this shape:

{
  "mcpServers": {
    "berrycrawl": {
      "type": "streamable-http",
      "url": "https://api.berrycrawl.com/api/v1/mcp",
      "headers": {
        "Authorization": "Bearer bc_..."
      }
    }
  }
}

Do not commit a file containing the real key. Prefer an app's secret store or environment-variable substitution when it supports one.

TypeScript client

This example uses the official MCP TypeScript SDK:

import { Client } from "@modelcontextprotocol/sdk/client/index.js";
import { StreamableHTTPClientTransport } from "@modelcontextprotocol/sdk/client/streamableHttp.js";

const apiKey = process.env.BERRYCRAWL_API_KEY;
if (!apiKey) throw new Error("BERRYCRAWL_API_KEY is required");

const client = new Client({ name: "my-agent", version: "1.0.0" });
const transport = new StreamableHTTPClientTransport(
  new URL("https://api.berrycrawl.com/api/v1/mcp"),
  {
    requestInit: {
      headers: { Authorization: `Bearer ${apiKey}` },
    },
  },
);

await client.connect(transport);
const tools = await client.listTools();
console.log(tools.tools.map((tool) => tool.name));

const result = await client.callTool({
  name: "berrycrawl_scrape_url",
  arguments: {
    url: "https://berrycrawl.com",
    formats: ["markdown"],
    response_format: "json",
  },
});

console.log(result.structuredContent);
await client.close();

Tools

berrycrawl_search_web never bulk-scrapes its result pages. If an agent needs full page content, it should select only the relevant URLs and call berrycrawl_scrape_url for those pages. This keeps ordinary searches fast and prevents surprise scrape charges.

Every data tool accepts response_format. Use markdown for a model conversation and json when the client will inspect structuredContent.

Screenshot cleanup

berrycrawl_capture_screenshot removes cookie banners, overlays, and chat widgets by default. The cookie cleanup can be switched off independently when the original consent UI needs to remain:

{
  "url": "https://example.com",
  "remove_cookie_banners": false,
  "response_format": "json"
}

The tool also supports device presets, PNG/JPEG/WebP, URL or base64 output, lazy-page scrolling, tall-page stitching, selector capture, hide selectors, and privacy masks. MCP uses snake_case parameter names; the REST API uses camelCase.

Common workflows

Find a page, then read it:

1. Call berrycrawl_map_site with a narrow search filter and a small limit.
2. Select the best URL from the returned links.
3. Call berrycrawl_scrape_url with formats: ["markdown"].

Run a background crawl:

1. Call berrycrawl_start_crawl and save the returned job ID.
2. Poll berrycrawl_get_job until the status is COMPLETED or FAILED.
3. Follow the next-page pointer until all result pages have been read.

Extract structured data:

1. Scrape or search to identify the right source URLs.
2. Call berrycrawl_start_extract with the URLs, a precise prompt, and an optional JSON schema.
3. Poll berrycrawl_get_job with the returned job ID.

Errors and limits

MCP protocol errors are reserved for malformed protocol messages. A tool failure returns isError: true with a short cause and a suggested next step. Internal stack traces, credentials, and provider errors are not returned.

Large tool results are capped to protect the model's context window. When truncated is true, lower the tool limit, narrow the URL set, or request the next crawl page.

The server accepts one JSON response per request and does not keep MCP session state. Your crawl and extraction jobs remain available because job state lives in Berrycrawl, not in the MCP connection.