Both of these keep the knowledge base in sync with something that already exists, on a schedule. Add them once and stop copy-pasting.
Web Sources — scrape pages on a schedule
Use this for pages whose content changes: a pricing page, a policies page, a public help centre you maintain elsewhere.
- Knowledge → Web Sources → Add Web Source.
- Give it a name and list the pages to scrape, one URL per row.
- Set the refresh interval in minutes. The default is 1440 — once a day.
- Save. Use Scrape now if you don't want to wait for the first cycle.
Each page is fetched and its text indexed. Preview captured content shows you exactly what we extracted and how it was chunked — check this if answers from a page look wrong or thin.
API Sources — poll an API on a schedule
Use this when the knowledge lives in a system with an HTTP endpoint and the data is the same for everyone (a catalogue, a list of locations, a schedule).
- Knowledge → API Sources → Add API Source.
- Set the URL and Method, add any Request headers you need for auth, and a Request body if it's a POST.
- Set a Content path if the response is JSON. This is a
gjson path that pulls out the text —
for example
articles.#.body. Leave it blank for plain-text responses or a root-level JSON array. - Click Test before saving. You'll see the status, the timing, the extracted
content, and the raw response — so you can tell a bad content path
(
nothing extracted) from a bad request. - Set the poll interval and save.
Use the Description field. It's for your reference — "returns a JSON array of available flavours" is the kind of note that saves you ten minutes in six months.
Should this be an API Source or a live tool?
Ask whether the answer is the same for every visitor.
- Same for everyone, changes slowly → API Source. It's polled and indexed, so answers are fast and cost nothing extra per question.
- Different per visitor, or must be current to the second (order status, their account, today's stock) → a live tool instead. See Live tools (MCP).
Nothing is being extracted from my API
Click Test and read the raw response. Nine times in ten it's the content path: either it doesn't match the shape of the JSON, or the response isn't JSON at all and the path should be blank. Copy results puts the whole test output on your clipboard if you need to send it to someone.