Example 3 (paginated JSON API)
Reference AI-product-sync parser. Paginated JSON API. fetch_next_chunk returns one page per call; cursor (page number) lives in state['parser']['page']. Demonstrates the natural shape for paginated REST sources where one HTTP call gives you one chunk.
A reference parser for the aiGenerated product-sync provider — and a worked example of a paginated JSON API, where each request returns one page of products and fetch_next_chunk bumps a page cursor in state between calls. Copy it as a starting point and adapt the request parameters and field mapping to your API.
Source shape: paginated-json-api
Runtime helper note. The _fetch_page helper uses a try/except that falls back to fetch_via_proxy() — a globally-injected function that routes the request through Octocom's HTTP proxy (a stable, dedicated egress IP). Use it when the origin blocks the default IP (symptoms: HTTPError 403, HTTPError 429, unexplained urlopen timeouts). Signature:
fetch_via_proxy(url, *, headers=None, timeout=60, method='GET', data=None) -> bytes