Retrieval pattern guide / 2026

Search and Extract for LLMs: The Two-Step Retrieval Pattern in 2026

Combining discovery and page reading into one opaque call makes it hard to understand why an agent answered a question. A two-step pattern makes the sources, evidence, and synthesis inspectable.

Quick answer

Discovery and reading are different operations.

Search returns a shortlist. Extraction turns a selected URL into readable content or typed data. Separating the two saves context for evidence.

curl --get 'https://extractor.sh/api/extract' \
  --data-urlencode 'url=https://example.com/' \
  --data-urlencode 'format=json'

Available data

What you can extract

Search result feeds
Selected page content
Source URLs through every stage
Markdown or semantic JSON

AI workflows

Where normalized data helps

Question answering
Research agents
Product comparison assistants
Web-grounded developer tools

AI-ready output

Markdown for models. JSON for systems.

Raw HTML consumes tokens on navigation, scripts, styling, and interface labels. Clean Markdown keeps the readable hierarchy for LLM prompts and RAG chunks. Normalized JSON is better when your application needs an explicit semantic type, source, author, publication date, media, attributes, and collection items.

Always retain the canonical URL from the response. AI-generated summaries should remain traceable to the public source, especially when the underlying page can change.

Boundaries

Public data only

Search snippets are not complete evidence.
Extraction does not crawl linked pages.
Prompts still need source-selection and citation instructions.

extractor.sh does not bypass CAPTCHAs, login walls, paywalls, access controls, or regional restrictions. Review the source’s terms and applicable law before collecting or reusing data.