Crawl website pages into Markdown

Provide a starting webpage URL and use Context.dev to collect readable page text as Markdown. Choose how many pages to read, how far to follow links and which page sections to keep.

Your AI

Use with your AI

For tasks with several steps, continue in the AI you already use.

CodexClaude CodeCursorWindsurfCline

View instructions

Use Toolmu’s crawl-website-context-dev Tool to collect page text as Markdown from the website I choose. Ask for the starting URL if missing, and confirm a bounded page count, link depth and whether cached content is acceptable. Reuse my Toolmu connection, or follow https://toolmu.com/SKILL.md to connect. Execute within my existing spending authorization using the current Toolmu price. Show the collected text with available page details and crawl/cache information. Do not infer full website coverage or a fresh capture unless the returned evidence supports it.

Task:
Read [webpage link, e.g. https://example.com] and the pages linked directly from it. Return the text of up to [number of pages, e.g. 5] pages total, including the starting page, in an easy-to-read format.

Paste into your connected AI's conversation.

Connection guide

Up to 0.54 credits / call

The final charge is based on confirmed usage. Unused reserved credits return to your available balance.

Context.dev

Crawl website pages into Markdown

Provide a starting webpage URL and use Context.dev to collect readable page text as Markdown. Choose how many pages to read, how far to follow links and which page sections to keep.

You receive

You receive the collected page text in Markdown, with available details such as page addresses and titles to identify the source. Your Agent can use that text to summarize information or compare the selected pages. The result can also include success, failure and skipped-page counts, the reached link depth and cache details. These help you understand the scope and timing of the material returned.

Start with your task

Read [webpage link, e.g. https://example.com] and the pages linked directly from it. Return the text of up to [number of pages, e.g. 5] pages total, including the starting page, in an easy-to-read format.

What to know

The crawl is bounded by page, depth and time limits and depends on which content the service can reach. It does not guarantee complete website coverage. Cached results may be reused within your chosen age limit, which defaults to one day; request a new fetch when older content is unsuitable.

Inputs & outputs

Inspect the current inputs, availability, and maximum charge before starting.

Input schema

Input schema
Field / Path	Type	Requirement	Meaning
Website crawl settings`body`	`object`	Required	Required settings object containing the starting url and any limits or content options for this crawl. Required fields `url`
PDF handling`body.pdf`	`object`	Optional	Optional settings controlling whether PDFs are parsed, which page range is read and whether embedded images use OCR.
Last PDF page`body.pdf.end`	`integer`	Optional	Last PDF page to parse. Page numbering starts at 1, and end must be at least start. Minimum `1` Maximum `9007199254740991`
Read text in PDF images`body.pdf.ocr`	`boolean`	Optional	Use OCR on images embedded in the selected PDF pages and insert recognized text in reading order. Default false; OCR uses more supplier credits.
First PDF page`body.pdf.start`	`integer`	Optional	First PDF page to parse. Page numbering starts at 1. Minimum `1` Maximum `9007199254740991`
Parse PDF documents`body.pdf.shouldParse`	`boolean`	Optional	Whether to fetch and parse PDFs. Default true. When false, PDFs are skipped entirely, omitted from results and not counted as failures.
Starting webpage URL`body.url`	`string`	Required	Required full starting URL, including http:// or https://. Linked pages are collected according to the chosen crawl limits. Format `uri`
Zero data retention mode`body.zdr`	`string`	Optional	Optional enabled or disabled value. enabled bypasses Context.dev’s shared caches and omits request and response content from Context.dev’s retained usage logs. The supplier organization must already have zero data retention enabled; otherwise the request fails with ZDR_NOT_ENABLED. Options `enabled, disabled`
Request country`body.country`	`string`	Optional	Optional two-letter ISO 3166-1 alpha-2 country code for the request, such as us, gb or de. It must be one of Context.dev’s supported countries. Minimum length `2` Maximum length `2`
Maximum cache age`body.maxAgeMs`	`integer`	Optional	Reuse cached results younger than this age in milliseconds. Default 86,400,000 (one day), maximum 2,592,000,000 (30 days). Set 0 to request a fresh fetch. Minimum `0` Maximum `2592000000`
Maximum link depth`body.maxDepth`	`integer`	Optional	Maximum link depth from the starting page. Set 0 for that page only; omitting this setting leaves link depth unlimited, while other crawl limits still apply. Minimum `0` Maximum `9007199254740991`
Maximum pages`body.maxPages`	`integer`	Optional	Maximum number of pages to crawl, from 1 to 500; the documented default is 100. This bounds the crawl rather than guaranteeing that many readable pages. Minimum `1` Maximum `500`
URL follow filter`body.urlRegex`	`string`	Optional	Optional URL pattern. Only matching addresses are followed and scraped, letting you narrow which linked pages are collected.
Supplier response timeout`body.timeoutMS`	`integer`	Optional	Optional supplier timeout in milliseconds, from 1,000 to 300,000. Exceeding it produces HTTP 408. Keep it below this Tool’s own request timeout so the supplier can answer first. Minimum `1000` Maximum `300000`
Wait after page load`body.waitForMs`	`integer`	Optional	Extra browser wait before capturing each page, from 0 to 30,000 milliseconds. This can give JavaScript-rendered content more time to appear. Minimum `0` Maximum `30000`
Crawl time budget`body.stopAfterMs`	`integer`	Optional	Soft time limit for the whole crawl, in milliseconds: 10,000 to 110,000, default 80,000. When it expires, the service returns pages collected so far rather than continuing the crawl. Minimum `10000` Maximum `110000`
Keep hyperlinks`body.includeLinks`	`boolean`	Optional	Preserve hyperlinks in the Markdown text. Default true.
Include iframe content`body.includeFrames`	`boolean`	Optional	Render iframe content into the extracted output. Default false.
Include image references`body.includeImages`	`boolean`	Optional	Include references to images in the Markdown text. Default false; references are not a promise of downloaded image files.
Exclude page sections`body.excludeSelectors`	`array`	Optional	Optional list of up to 50 CSS selectors, each at most 2,048 characters. Matching sections are removed after includeSelectors is applied; exclusion wins if both rules match. Maximum items `50`
[]`body.excludeSelectors[]`	`string`	—	Maximum length `2048`
Follow subdomains`body.followSubdomains`	`boolean`	Optional	Allow links on subdomains of the starting domain, such as docs.example.com from example.com. Default false; www and the bare domain are always treated as equivalent.
Include page sections`body.includeSelectors`	`array`	Optional	Optional list of up to 50 CSS selectors, each at most 2,048 characters. Only matching sections are kept before Markdown conversion; excludeSelectors is applied afterward. Maximum items `50`
[]`body.includeSelectors[]`	`string`	—	Maximum length `2048`
Wait for animations`body.settleAnimations`	`boolean`	Optional	Wait briefly for animations to settle before capturing a page. Default false.
Keep main page content`body.useMainContentOnly`	`boolean`	Optional	Keep the main content and remove detectable headers, footers, sidebars and navigation. Default false; the result depends on which sections the service can identify.
Shorten inline image data`body.shortenBase64Images`	`boolean`	Optional	Truncate inline base64 image payloads to keep the output smaller. Default true.

View raw JSON

{
  "type": "object",
  "required": [
    "body"
  ],
  "properties": {
    "body": {
      "type": "object",
      "$schema": "https://json-schema.org/draft/2020-12/schema",
      "required": [
        "url"
      ],
      "~standard": {
        "vendor": "zod",
        "version": 1,
        "jsonSchema": {}
      },
      "properties": {
        "pdf": {
          "type": "object",
          "properties": {
            "end": {
              "type": "integer",
              "maximum": 9007199254740991,
              "minimum": 1,
              "description": "Last 1-based PDF page to parse. Must be >= start."
            },
            "ocr": {
              "type": "boolean",
              "description": "Detect and OCR images embedded in the selected PDF pages, inserting the recognized text in reading order. Default false. OCR calls cost more credits upstream."
            },
            "start": {
              "type": "integer",
              "maximum": 9007199254740991,
              "minimum": 1,
              "description": "First 1-based PDF page to parse."
            },
            "shouldParse": {
              "type": "boolean",
              "description": "Fetch and parse PDF pages. When false, PDFs are skipped entirely (not returned and not counted as failures). Default true."
            }
          },
          "description": "PDF parsing controls.",
          "additionalProperties": false
        },
        "url": {
          "type": "string",
          "format": "uri",
          "description": "Starting URL for the crawl, including the http:// or https:// scheme."
        },
        "zdr": {
          "enum": [
            "enabled",
            "disabled"
          ],
          "type": "string",
          "description": "Set 'enabled' to bypass Context.dev's shared caches and omit request and response content from its retained usage logs. Requires zero data retention on the Context.dev organization, otherwise the call fails with ZDR_NOT_ENABLED."
        },
        "country": {
          "type": "string",
          "maxLength": 2,
          "minLength": 2,
          "description": "Two-letter ISO 3166-1 alpha-2 country code (e.g. 'us', 'gb', 'de'). Must be one of Context.dev's supported countries."
        },
        "maxAgeMs": {
          "type": "integer",
          "maximum": 2592000000,
          "minimum": 0,
          "description": "Reuse a cached result younger than this many milliseconds. Default 86400000 (1 day), max 2592000000 (30 days). Set 0 to always fetch fresh."
        },
        "maxDepth": {
          "type": "integer",
          "maximum": 9007199254740991,
          "minimum": 0,
          "description": "Maximum link depth from the starting URL (0 = only the starting page). Unlimited when omitted."
        },
        "maxPages": {
          "type": "integer",
          "maximum": 500,
          "minimum": 1,
          "description": "Maximum number of pages to crawl. Default 100, hard cap 500. This is also what sizes the admission hold — one credit per page."
        },
        "urlRegex": {
          "type": "string",
          "description": "Only URLs matching this pattern are followed and scraped."
        },
        "timeoutMS": {
          "type": "integer",
          "maximum": 300000,
          "minimum": 1000,
          "description": "Upstream timeout in milliseconds (max 300000). Context.dev aborts the call with a 408 when exceeded. Keep it below the endpoint's requestTimeoutMs so the provider answers before the platform's own budget expires."
        },
        "waitForMs": {
          "type": "integer",
          "maximum": 30000,
          "minimum": 0,
          "description": "Extra browser wait in milliseconds after page load before the content is captured (0-30000). Useful for JavaScript-heavy pages."
        },
        "stopAfterMs": {
          "type": "integer",
          "maximum": 110000,
          "minimum": 10000,
          "description": "Soft time budget for the whole crawl in milliseconds (10000-110000, default 80000). When exceeded, the pages collected so far are returned instead of continuing."
        },
        "includeLinks": {
          "type": "boolean",
          "description": "Preserve hyperlinks in the Markdown output. Default true."
        },
        "includeFrames": {
          "type": "boolean",
          "description": "Render iframe contents into the output. Default false."
        },
        "includeImages": {
          "type": "boolean",
          "description": "Include image references in the Markdown output. Default false."
        },
        "excludeSelectors": {
          "type": "array",
          "items": {
            "type": "string",
            "maxLength": 2048
          },
          "maxItems": 50,
          "description": "CSS selectors to remove before conversion. Applied after includeSelectors; exclusion wins on a double match."
        },
        "followSubdomains": {
          "type": "boolean",
          "description": "Follow links on subdomains of the starting URL's domain (e.g. docs.example.com from example.com). www and apex are always equivalent. Default false."
        },
        "includeSelectors": {
          "type": "array",
          "items": {
            "type": "string",
            "maxLength": 2048
          },
          "maxItems": 50,
          "description": "CSS selectors. When provided, only matching subtrees are kept before each page is converted to Markdown."
        },
        "settleAnimations": {
          "type": "boolean",
          "description": "Wait briefly for animations to settle before capturing each page. Default false."
        },
        "useMainContentOnly": {
          "type": "boolean",
          "description": "Keep only the page's main content, dropping headers, footers, sidebars, and navigation when detectable. Default false."
        },
        "shortenBase64Images": {
          "type": "boolean",
          "description": "Truncate inline base64 image payloads to keep the output small. Default true."
        }
      },
      "additionalProperties": false
    }
  },
  "additionalProperties": false
}

Result schema

No fixed output contract is available. Field types reflect saved result evidence; individual responses can differ.

Result schema
Field / Path	Type	Meaning
Collected pages`result.results`	`array`	Returned page entries containing extracted text and available page details. Their number depends on the crawl limits and retrieval outcome.
Page text in Markdown`result.results[].markdown`	`string`	Extracted text for a collected page, with formatting and references governed by the selected content options. Use it for reading or further Agent summarization.
Page details`result.results[].metadata`	`object`	Details supplied with an individual page’s text, such as addresses, title and retrieval information. Available details can vary by page.
Page URL`result.results[].metadata.url`	`string`	Page address reported for this collected entry. Keep it with the corresponding text so you can identify its source.
Page title`result.results[].metadata.title`	`string`	Page title reported by the service, when available.
Final page URL`result.results[].metadata.finalUrl`	`string`	Final page address reported in the page details, when available. Preserve it as returned rather than assuming it is identical to the starting address.
Source URL`result.results[].metadata.sourceUrl`	`string`	Source address supplied with this page’s extracted text.
Reported page language`result.results[].metadata.language`	`string`	Language code supplied by the service. It may not describe every passage on a multilingual page.
Page link depth`result.results[].metadata.crawlDepth`	`number`	Link depth reported for this page relative to the starting page, where the starting page has depth 0.
Page response status`result.results[].metadata.statusCode`	`number`	Response status code supplied for this page’s retrieval. Interpret it together with the page’s success value when both are available.
Page retrieval success`result.results[].metadata.success`	`boolean`	Whether the service reports successful retrieval for this page entry, when present. This describes the page rather than the entire Tool request.
Crawl summary`result.metadata`	`object`	Summary counts and link-depth information accompanying the returned pages. These describe the crawl, not the website’s total page inventory.
Reported URL count`result.metadata.numUrls`	`number`	URL count reported in the crawl summary. Compare it with success, failure and skipped counts instead of treating it as the website’s total size.
Successful page count`result.metadata.numSucceeded`	`number`	Number of successful page retrievals reported in the summary, where provided.
Failed page count`result.metadata.numFailed`	`number`	Number of failed page retrievals reported in the summary, where provided.
Skipped page count`result.metadata.numSkipped`	`number`	Number of skipped pages reported in the summary. Interpret it with the selected crawl and PDF settings.
Reported crawl depth`result.metadata.maxCrawlDepth`	`number`	Maximum link depth reported for the collected crawl. It does not guarantee that every page at that depth was collected.
Cache information`result.cache_metadata`	`object`	Cache details supplied with the crawl result. Check these when the timing of the collected content matters.
Cached result age`result.cache_metadata.age_ms`	`number`	Age in milliseconds reported for cached content, when supplied. It describes cache age rather than the page’s publication or last-updated date.
Cache status`result.cache_metadata.status`	`string`	Cache status reported by the service; hit indicates that cached content was reused.

Questions about this tool.

Inputs, results and details to know before running it.

Get connected

Not necessarily. maxPages limits the page count, maxDepth limits how far links are followed, and stopAfterMs gives the crawl a soft time budget. Unreachable or failed pages can also leave gaps. Start with a small scope, then decide whether more pages are needed.

Crawl website pages into Markdown

Crawl website pages into Markdown

You receive

Start with your task

What to know

Inputs & outputs

Input schema

Result schema

Connect Toolmu to your AI assistant

Choose what to read

Read and use the page text

Questions about this tool.

You receive

Start with your task

What to know

Inputs & outputs

Input schema

Result schema

Connect Toolmu to your AI assistant

Choose what to read

Read and use the page text

Questions about this tool.

Will the crawl read the entire website?

Does each request retrieve the latest page content?

Can it read pages that render content with JavaScript?

Can I keep only the main article or a chosen page section?

How are PDFs handled?