Build a research assistant that searches, reads and cites its sources

Two calls, no search account

An agent that answers from current sources needs two things: a list of relevant pages and the text of the best ones. Web Search returns up to 10 ranked results from a licensed index (Exa or Brave) with title, URL, snippet, highlights and published date, without a search-API account of your own. Web Page to Clean Markdown reads the pages you choose. Neither uses a model; the only model in this pipeline is yours.

Ask for the text you need

Snippets are off by default. Our first test query returned ranked titles and URLs with empty snippet fields; the same query with include_text: true returned snippets of roughly 500 to 700 characters plus highlights. Set it when you want a first-pass relevance check without fetching the page.

Useful filters: site limits results to one domain, date_from keeps results published after a date, provider picks exa, brave or auto, and num_results is up to 10. Not every result has a published date, so do not treat a missing date as old or new.

curl -sS -X POST "https://$RAPIDAPI_HOST/v1/web-search" \
  -H "Content-Type: application/json" \
  -H "X-RapidAPI-Key: $RAPIDAPI_KEY" \
  -H "X-RapidAPI-Host: $RAPIDAPI_HOST" \
  -d '{"query": "Cyber Resilience Act Article 14 reporting 24 hours", "num_results": 3, "include_text": true}' \
  | jq -r '.data.results[] | "\(.rank). \(.title)\n   \(.url)"'

The whole loop in Python

The script searches, reads the top results, and builds a prompt in which every source has a number, a URL and the hash of the text the model was shown. Pass the prompt to the model of your choice and require it to cite sources by number.

import os, sys, requests

HOST = os.environ["RAPIDAPI_HOST"]
HEAD = {"X-RapidAPI-Key": os.environ["RAPIDAPI_KEY"], "X-RapidAPI-Host": HOST}

def call(path, body):
    r = requests.post(f"https://{HOST}/v1/{path}", headers=HEAD, json=body, timeout=90)
    r.raise_for_status()
    return r.json()["data"]

question = sys.argv[1]
hits = call("web-search", {"query": question, "num_results": 5})["results"][:3]

sources = []
for i, hit in enumerate(hits, 1):
    try:
        page = call("page-contents", {"url": hit["url"], "max_chars": 6000})
    except requests.HTTPError:
        continue  # some sites refuse automated fetches; skip them
    sources.append(f"[{i}] {page['final_url']} (sha256 {page['content_sha256'][:12]})\n{page['content']}")

prompt = (
    "Answer using only the numbered sources. Cite each claim as [n]. "
    "If the sources do not answer it, say so.\n\n"
    + "\n\n".join(sources)
    + f"\n\nQuestion: {question}"
)
print(prompt)

Where it falls short

Search results are only as good as the query, so let the model rewrite the question into one or two searches before you call. Pages that depend on JavaScript come back thin, and a page that blocks automated requests returns an error, which the script above skips. Keep the hashes: if an answer is challenged later, you can show which text the model was given.