OptionalallowOptionalcategories?: undefined | CrawlCategory[]Only crawl URLs containing these categories.
OptionalexcludeExclude these domains.
OptionalexcludeExclude these paths.
OptionalextractThe depth of the extractions. It can be "basic" or "advanced".
Optionalformat?: undefined | "markdown" | "text"The format of the respose. It can be "markdown" or "text"
OptionalincludeWhether to include the favicon URL for each result.
OptionalincludeInclude images in the crawl results.
Optionalinstructions?: undefined | stringNatural language instructions to guide the crawler
Optionallimit?: undefined | numberThe maximum number of pages to crawl.
OptionalmaxThe maximum number of pages to crawl per level.
OptionalmaxThe maximum number of hops from the starting URL.
OptionalselectOnly crawl these domains.
OptionalselectOnly crawl URLs containing these paths.
The URL to start crawling from.
Example: "https://example.com"
Allow crawling external domains.