pcsarchive

A small local archive API for pcsproxy: when the proxy runs with -store http://127.0.0.1:8082, every procyclingstats.com page you browse is shipped here and kept in a SQLite file — a personal offline copy of PCS that grows as a side effect of normal browsing.

• Original HTML is stored (captured before the proxy rewrites URLs), so the archive is independent of the proxy.
• Deduplicated with history: re-visiting an unchanged page only bumps its last_seen; a changed page gets a new version row, so you can see what a page looked like earlier.
• Bodies are gzip-compressed in the database.
• Pure-Go SQLite (modernc.org/sqlite) — static binaries, no CGO.

Run

go build -o pcsarchive .
./pcsarchive                 # listens on 127.0.0.1:8082, db ./pcsarchive.db
./pcsarchive -listen 127.0.0.1:9000 -db ~/pcs.db

Remote use

To run the archive on another machine than the proxy, set a shared token and listen beyond localhost (PCSARCHIVE_TOKEN env works too, and keeps the token out of ps output):

./pcsarchive -listen 0.0.0.0:8082 -token <token>

With a token set, every /v1/* endpoint requires Authorization: Bearer <token>; only /healthz stays open. Point the proxy at it with -store https://... -store-token <token>. Terminate TLS in front (reverse proxy) — the token and page contents travel in the clear over plain http.

Rankings

Ranking pages (/rankings.php, /rankings/...) are additionally parsed on ingest into structured rows: ranking date and type (PCS's own slugs, e.g. pcs-ranking, pcs-season-ranking — taken from the page itself, not the fetch time), rank, rider, team and points. Rows are upserted on (type, date, rider), so filtered or paginated views merge into one consistent ranking (PCS keeps global rank numbers on those views). Team/nation ranking pages contain no rider rows and are archived unparsed.

curl 'http://127.0.0.1:8082/v1/rankings?type=pcs-ranking&date=2026-07-06&limit=10'
curl 'http://127.0.0.1:8082/v1/rankings?rider=tadej-pogacar'

Riders and race results

Rider profile pages (/rider/<slug>) are parsed into a riders table: name, team, nationality, birthdate, birthplace, weight, height and PCS career points per specialty. Race result pages (/race/...) are parsed into races (name, class, date, distance — keyed on race/year/stage via the page's canonical race_seo field, so URL variants collapse) and race_results: one row per rider per classification (result, gc, points, kom, youth), with rank or status (DNF/DNS/OTL/DSQ), bib, age, team, UCI/PCS points and time. Times are stored as displayed by PCS: absolute for the winner, gaps for the rest, ,, for same time. Rows are upserted, so re-visiting a page refreshes the data. Team time trials have no individual stage table on PCS; their GC and other classifications are still captured.

Stage pages also yield the climbs block and a general-info object:

• climbs is a master list of climbs (a PCS location slug + name, plus length_km, gradient_pct and lat/lon coordinates filled by the external importers; 0 when unknown).
• race_climbs relates a stage to its climbs (climb_id references climbs.id), with each climb's position in the stage plus category and km_point read from the KOM headers — the place to hang timings later.
• The stage's whole general-info block (distance, vertical meters, temperature, classification, profile score, points scales, departure/ arrival, …) is stored as a JSON object in races.info for later use.

Location and climb pages build a climb-route model on top of climbs:

• A /location/<slug> page's "Climbs to this location" table lists the ascent routes of that climb; each is stored in climb_routes (a route slug + descriptive name) related to the climb via climb_id.
• A /climb/<slug> page fills that route's info — its whole detail block (distance, elevation gain, start/top altitude, average %, …) as JSON, excluding the profile score — and links the route to its location. One climb can have several routes (Col du Tourmalet has two).
• The /climb page's "Best climbing times" table is stored in climb_times, one row per rider per timed ascent (rank, race, time, W/kg), foreign-keyed to the climb route and to riders — riders not yet fetched are created as minimal rows (slug + name), which a later /rider page fills in.

A rider page also carries the rider's own results table, so those results are extracted too — the rider's placing (or DNF/DNS/…) and points for each race, keyed like any other race_results row. If the race isn't otherwise in the archive, a sparse race is created (name only) from what the rider page shows: you may know only that someone finished 68th in a race you never opened directly. These rider-derived rows only fill the fields the rider page shows and never overwrite the richer data (team, time, bib, age) a full race page provides for the same result.

Team pages (/team/<slug>) fill three tables: teams (season-keyed slug, name, status WT/PRT/CT/CLUB, abbreviation, license country, bike), a team_riders roster (replaced wholesale on re-ingest), and team_seasons — the season dropdown on every team page lists all years of the same team under each year's own name (Rabobank → Belkin → Jumbo-Visma → Visma), and is stored as a lineage keyed on the earliest season, so renamed seasons stay connected. GET /v1/teams?slug=<s> returns the roster and full lineage; without slug it lists season metadata only.

curl 'http://127.0.0.1:8082/v1/teams?slug=team-visma-lease-a-bike-2026'
curl 'http://127.0.0.1:8082/v1/riders?slug=wout-van-aert'
curl 'http://127.0.0.1:8082/v1/races?race=tour-de-france&year=2026'
curl 'http://127.0.0.1:8082/v1/results?race=tour-de-france&year=2026&stage=stage-1&classification=gc'
curl 'http://127.0.0.1:8082/v1/results?rider=jonas-vingegaard'

API

Method/path	What it does
POST /v1/pages	Ingest a page (used by pcsproxy). JSON: url, status, content_type, fetched_at, body_b64.
GET /v1/pages?url=<u>	Version history of one URL (metadata, newest first).
GET /v1/pages?prefix=<p>&limit=&offset=	Newest version per URL under a prefix.
GET /v1/pages/{id}	Stored body, served with its original Content-Type.
GET /v1/pages/latest?url=<u>	Same, for the newest version of a URL.
GET /v1/rankings?type=&date=&rider=&limit=&offset=	Parsed ranking rows (all filters optional).
GET /v1/rankings/dates	Distinct {type, date} pairs present in rankings, newest date first then type.
GET /v1/riders?slug=&q=&limit=&offset=	Parsed rider profiles; q is a case-insensitive substring match on name.
GET /v1/teams?slug=&year=&q=&limit=&offset=	Team seasons; with slug, includes roster + lineage; q is a case-insensitive substring match on name.
GET /v1/races?race=&year=&q=&limit=&offset=	Parsed race/stage metadata (incl. the info JSON object); q matches on name.
GET /v1/climbs?race=&year=&stage=&limit=&offset=	A stage's climbs (joined with the master climb records).
GET /v1/climb-routes?climb=<slug>&limit=&offset=	A climb's ascent routes, each with its /climb info block.
GET /v1/climb-times?route=<slug>&rider=<slug>&limit=&offset=	Best-times rows for a route, or a rider's timed ascents.
GET /v1/climb-list?q=&slug=&limit=&offset=	Master climbs with route counts and profile (length/gradient/coordinates where known); slug looks one climb up exactly.
GET /v1/climb-measurements?climb=<slug>&limit=&offset=	Every source's numbers for a climb, attributed (source, source_id, source_url).
GET /v1/climb-records?climb=<slug>&limit=&offset=	A climb's timed ascents from all sources in one shape — PCS best times and climbing-records.com yearly records — each row carrying its source.
GET /v1/results?race=&year=&stage=&classification=&rider=&limit=&offset=	Parsed result rows.
GET /v1/stats	Row counts (pages, riders, teams, races, race_results, rankings) plus latest_ranking_date and latest_race_date.
GET/POST /v1/query	Run read-only SQL (sql param); returns {sql, columns, rows}. See below.
GET /v1/schema	Tables and their columns (for the query UI).
GET /healthz	ok

Ad-hoc SQL (/v1/query)

Run arbitrary read-only SQL against the archive and get back columns and rows. Writes and DDL (INSERT/UPDATE/DELETE/DROP/CREATE/…) are rejected — the query runs on a connection with SQLite's query_only pragma, so it physically cannot modify the database. Only a single statement is allowed, and a SELECT/WITH with no LIMIT of its own is capped at 300 rows. A SQL error returns 400 with the message.

curl -G --data-urlencode 'sql=SELECT rider, points FROM rankings
     WHERE type = "pcs-ranking" ORDER BY rank LIMIT 10' \
     http://127.0.0.1:8082/v1/query

Tables: pages, rankings, riders, teams, team_riders, team_seasons, races, race_results. pcsweb exposes this as a Query page with the result rendered as a table.

Browse an archived page directly: http://127.0.0.1:8082/v1/pages/latest?url=/race/tour-de-france/2026

Web interface (pcsweb)

cmd/pcsweb is a small server-rendered web UI (html/template, no JS framework, one embedded CSS file) for browsing the archive: a dashboard, rankings, races/stages with a classification switcher, riders, teams (with season lineage), climbs (with each route's info and best-times leaderboard), the raw archived pages, and a Query page for running read-only SQL (via /v1/query) with a browsable tables-and-columns reference and the result rendered as a table. It talks to pcsarchive only over the HTTP API above, so it can run anywhere pcsarchive is reachable, and it ships as its own container.

It is never meant to be public: every route except /robots.txt requires HTTP Basic Auth, /robots.txt disallows everything, and every response gets X-Robots-Tag: noindex, nofollow and Cache-Control: private.

Configuration is environment-first (flags override):

Variable	Meaning
PCSWEB_LISTEN	Address to listen on (default :8083).
PCSWEB_ARCHIVE_URL	Base URL of the pcsarchive API (required).
PCSWEB_ARCHIVE_TOKEN	Bearer token for the archive API, if it requires one (optional).
PCSWEB_USER / PCSWEB_PASS	Basic Auth credentials (required — pcsweb refuses to start without both).

go build -o pcsweb ./cmd/pcsweb
PCSWEB_ARCHIVE_URL=http://127.0.0.1:8082 PCSWEB_USER=rune PCSWEB_PASS=... ./pcsweb

Or as a container, built from Dockerfile.web (multi-stage, distroless, non-root, static binary):

docker build -f Dockerfile.web -t pcsweb .
docker run -d -p 8083:8083 \
  -e PCSWEB_ARCHIVE_URL=http://archive-host:8082 \
  -e PCSWEB_ARCHIVE_TOKEN=... -e PCSWEB_USER=rune -e PCSWEB_PASS=... pcsweb

CI (.woodpecker/docker.yaml) builds and pushes this image to the Forgejo registry at code.jaduer.dk/rune/pcsweb using woodpeckerci/plugin-docker-buildx. That plugin needs privileged mode to run buildx; if the runner rejects it with "insufficient privileges", add woodpeckerci/plugin-docker-buildx to WOODPECKER_PLUGINS_PRIVILEGED in the Woodpecker server config.

Running the whole stack with Docker Compose

docker-compose.yaml runs all four services off one .env: the pcsproxy reverse proxy, this archive, the pcsweb UI and the climbweb climbs frontend. Each has its own image, built and pushed by CI to the Forgejo registry (code.jaduer.dk/rune/{pcsproxy,pcsarchive,pcsweb,climbweb}); the compose file can also build them locally (the proxy from the sibling ../pcsproxy checkout).

A new service needs a step in .woodpecker/docker.yaml as well as a compose entry. Without one its image never reaches the registry, and docker compose pull fails with "manifest unknown" even though docker compose build works locally.

cp .env.example .env      # edit PCSARCHIVE_TOKEN, PCSWEB_USER, PCSWEB_PASS
docker compose up -d --build

• Browse procyclingstats.com through http://localhost:PROXY_PORT (default 8081). Pages are shipped to the archive over the internal network.
• The web UI is on WEB_PORT (default 8083), behind Basic Auth.
• The archive is also published on ARCHIVE_PORT (default 8082) so a pcsproxy running directly on the host (outside compose) can ship to it too.
• The database persists in the archive-data named volume.

.env holds secrets and is gitignored. The stack talks plain http; put TLS in front (reverse proxy) before exposing any port off the host.

Importing external climb data

Three importers enrich the climbs table from other sources, creating missing climbs on the go (matched to PCS locations by slugified name):

• cmd/climbrecords — climbing-records.com (read via its Blogger JSON feed): climb length and average gradient, plus yearly record times/speeds/riders into climb_records.
• cmd/watts2win — watts2win.eu (segments are treated as climbs): start coordinates (lat/lon), length, average gradient and top altitude, read from the segments map and each segment page.
• cmd/xclimbs — the X account @Na1chaca, which posts one timed effort per post: rider, race and stage, climb profile, time and speed, plus the power-derived metrics no other source here carries — VAM, W/kg and aSLP — into climb_ascents.

The account names several ways up the same mountain, and each becomes its own row in climb_routes under the one climb, slugged the way PCS slugs its own routes:

Posted name	Climb	Route	Whole climb?
Col du Tourmalet	col-du-tourmalet	col-du-tourmalet	yes
Col du Tourmalet via D918	col-du-tourmalet	col-du-tourmalet-d918	yes, by a named road
Col du Tourmalet from Gripp	col-du-tourmalet	col-du-tourmalet-gripp	no, 12.4 km of 17
Col de Marie Blanque - Steep Section	col-de-marie-blanque	col-de-marie-blanque-steep-section	no

A post naming no variant lands on the PCS base route rather than forking a parallel one. Only whole ascents contribute a profile to the master climbs row — a partial's length and gradient describe the segment, not the climb — which is why via is treated differently from from: a named road is the whole climb, a named start is not.

The " - " split is deliberately conservative. Plenty of climbs simply have a dash in their name — the archive holds Lo Port - Mont Caro — so a dashed suffix only counts as a variant when it reads like a section ("Steep Section", "Final 3 km"). Otherwise the name is left whole, because inventing a parent climb nobody rode is worse than leaving a variant unfolded.

climb_ascents is deliberately separate from climb_records: this source reports individual dated efforts rather than one aggregate row per climb and year, and a rider can set several timed ascents of a climb in one season.

Cross-source values never overwrite each other: canonical climbs fields are only filled where still unset, and every source's numbers are also kept verbatim in climb_measurements, attributed to their site (source, source_id, source_url) — so when sites disagree (they do: watts2win's "Jebel Hafeet (without last flat part)" is 9.25 km, climbing-records' full climb is 10.9 km), both values survive and stay traceable. All three importers are idempotent and safe to re-run.

PCS itself is a source like any other: ingesting (or -hydrate-ing) a /climb page materializes its distance/gradient/altitude as a procyclingstats.com measurement, and /v1/climb-records unions the PCS best-times leaderboards with climbing-records.com's yearly records and @Na1chaca's dated efforts into one source-attributed shape — so API consumers see every site's data through the same two endpoints, and pcsweb's climb pages show a "Measurements by source" table, the records with their source, and OpenStreetMap links where a source provided coordinates.

VAM on PCS climb times is derived, not reported. PCS's best-times table is # | Race | Rider | Time | W/kg, with no climbing speed in it, so /v1/climb-times computes VAM as the route's elevation gain over the time and leaves it unset where either is unknown. The formula reproduces the VAM @Na1chaca publishes to the metre on every post checked, so the two sources stay comparable — but a PCS row's VAM is computed from stored numbers rather than something PCS states. Ascents from @Na1chaca carry the VAM the post reports.

Locally

go run ./cmd/climbrecords -db ./pcsarchive.db   # all ~500 posts, a few seconds
go run ./cmd/watts2win   -db ./pcsarchive.db    # all ~530 segments, ~5 min at the default 500ms delay

All honor PCSARCHIVE_DB and take -dry-run (parse and report, write nothing), -limit <n> and -v (log every climb); the two site scrapers also take -base <url>, and cmd/watts2win a -delay between page fetches.

xclimbs and the post-listing problem

cmd/xclimbs reads post text from X's syndication endpoint — the one that backs embedded tweets. It needs no credentials, but it serves one post per request: there is no way to list an account's posts through it (the syndication.twitter.com profile-timeline endpoint answers 429 to unauthenticated callers). So xclimbs takes the posts to import from you:

# post URLs or bare ids, on the command line
go run ./cmd/xclimbs https://x.com/Na1chaca/status/2078876614390059052
# or a file of them, one per line (blank lines and # comments ignored)
go run ./cmd/xclimbs -file ids.txt
go run ./cmd/xclimbs -file -   < ids.txt      # or on stdin

To pick up new posts unattended, -timeline imports whatever the account has posted since the newest post already in climb_ascents. It has two sources, and picks whichever credential it finds:

# 1. Session cookies: free, but only the ~20 most recent posts.
X_COOKIES_FILE=~/.x-cookies go run ./cmd/xclimbs -timeline

# 2. X API bearer token: paginates the whole history.
X_BEARER_TOKEN=...           go run ./cmd/xclimbs -timeline

The cookie path reads the syndication timeline that backs embedded profile widgets. It carries full post text (so no per-post fetch) and needs no API subscription, but a logged-in session is required — unauthenticated callers get 429 — and it serves a fixed window of roughly the last 20 posts with no pagination cursor. That makes it right for keeping an archive current and no use for backfilling one: run it often enough that the account never posts 20 times between runs. For history, use the API token or feed ids by hand.

X_COOKIES_FILE names a file holding the Cookie: header value (the auth_token and ct0 pair is what matters); X_COOKIES takes it inline. Prefer the file — it keeps live account credentials out of shell history, process listings and compose files. Never commit them: they are full account access, not a scoped API key, and they invalidate when you log out.

Posts that are not climb reports (prose, replies) are skipped, not failed. Re-importing a post refreshes its row rather than duplicating it, so a post edited after the fact corrects itself on the next run.

-since overrides where a -timeline run starts, instead of resuming from the newest post already stored. It takes a bare id or a status URL:

go run ./cmd/xclimbs -timeline -since 2075230517499154933
go run ./cmd/xclimbs -timeline -since https://x.com/Na1chaca/status/2075230517499154933
go run ./cmd/xclimbs -timeline -since 0    # everything the source will serve

Use it to re-read a stretch after a parser fix (re-imports refresh rather than duplicate, so this is safe to repeat), or to reach back further than the last run did. Note what it cannot do on the cookie path: -since filters the ~20 posts the syndication timeline returns, it does not make it serve more. An id older than that window yields the same 20 posts, not the history between. Only X_BEARER_TOKEN paginates genuinely far back.

Backfilling from the browser

Without an API token, the way to reach older posts is to let a logged-in browser render them. cmd/xclimbs/collect-ids.js is a console script that scrolls a profile and writes out the post ids it finds:

1. Open https://x.com/Na1chaca logged in.
2. Open DevTools (F12) → Console, paste the contents of cmd/xclimbs/collect-ids.js, press Enter.
3. Leave it alone. It scrolls on its own, logging progress every few seconds, and stops after 45 seconds with nothing new. It downloads Na1chaca-ids.txt when done (also left in window.__ids).
4. Feed it in: go run ./cmd/xclimbs -file Na1chaca-ids.txt

Stop early with window.__stopCollect = true; partial results are kept in window.__ids throughout, so a long run is never lost to a mistake.

Turn off your ad blocker for x.com first. X pages the timeline in over XHR, and a blocker that drops those calls stalls the scroll — the tell is ERR_BLOCKED_BY_CLIENT in the console, and a run that ends after a couple of dozen posts.

It keeps only posts that look like climb reports — a profile line in parentheses, (11,20km; 9,15%; 1025m) — because the account posts plenty of prose and ranking lists, and each id otherwise costs a request and half a second. Reposts and quoted posts are skipped, so the list is the account's own work. Collection is driven by a MutationObserver rather than by the scroll loop, because X recycles post elements out of the DOM as they leave the viewport: a post can be created and evicted between two scroll steps, and reading the finished page finds only the last screenful.

Two limits worth knowing before a long run. X caps how far a profile timeline scrolls, so the oldest posts may simply be unreachable this way. And the selectors track X's current markup — if a run collects 0 posts while the page clearly shows some, the markup moved and the two selectors in harvest() are what need adjusting.

With Docker Compose

The archive image ships all three importers next to the server binary. Run them as one-off containers on the running stack — they share the archive-data volume (the image sets PCSARCHIVE_DB=/data/pcsarchive.db), so they operate on the same database the server is using:

docker compose run --rm --entrypoint /climbrecords archive
docker compose run --rm --entrypoint /watts2win   archive
# preview without writing:
docker compose run --rm --entrypoint /watts2win   archive -dry-run -limit 10

Feeding xclimbs a file of post ids

The archive service mounts only the archive-data volume, so a file sitting on the host is not visible inside the container: -file ids.txt on its own would look for /ids.txt in the image and find nothing. Either pipe the file in on stdin, or bind-mount it. Both work; pick whichever reads better to you:

# stdin: "-" reads the list from the pipe. -T keeps compose from allocating a
# TTY, which is what you want from a script or cron entry.
docker compose run --rm -T --entrypoint /xclimbs archive -file - < ids.txt

# bind mount: the file keeps a name inside the container, which makes the
# invocation self-describing in logs.
docker compose run --rm -v "$PWD/ids.txt:/ids.txt:ro" \
  --entrypoint /xclimbs archive -file /ids.txt

where ids.txt is post URLs or bare ids, one per line; blank lines and # comments are ignored, so the list stays annotatable:

# Tour de France 2026
https://x.com/Na1chaca/status/2078876614390059052
2075230517499154933

# Liège-Bastogne-Liège
2048521583665238296

And for new posts on a schedule (see the two credential options above):

# session cookies, mounted read-only rather than passed as an env var.
# The image runs as nonroot (uid 65532), so the cookie file must be
# world-readable: chmod 644. Do NOT reach for --user "$(id -u)" instead —
# it fixes the read but leaves the container unable to write /data, and the
# run then reports every post "skipped" instead of failing outright.
chmod 644 ~/.x-cookies
docker compose run --rm -v "$HOME/.x-cookies:/x-cookies:ro" \
  -e X_COOKIES_FILE=/x-cookies --entrypoint /xclimbs archive -timeline

# or the API token
docker compose run --rm -e X_BEARER_TOKEN --entrypoint /xclimbs archive -timeline

The stack can stay up while they run: importer and server both wait on SQLite's write lock (busy_timeout), and the importers' writes are short upserts.

climbweb — the climbs-only frontend

cmd/climbweb is a second, narrower UI. pcsweb browses the whole archive; climbweb shows one subject well: the climbs, the routes up them, and every timed ascent, from whichever source reported it.

Two ideas shape it:

• Climbs are ranked by how much is known about them. A climb with one stray time is not worth reading; the API orders by name, which buries everything good, so the index re-orders by result count and draws a data-density bar against the richest climb on the page.
• Every effort names a rider whose other climbs are one click away. A number alone cannot be judged. /rider/<slug> shows one rider across every mountain they have a time on, and /best ranks efforts across all climbs.

Ranking efforts on different climbs only means something with a normalized figure, so /best defaults to aSLP and leaves out efforts that have none rather than ranking them last — unrated is not slow.

CLIMBWEB_ARCHIVE_URL=http://127.0.0.1:8082 go run ./cmd/climbweb -open
# or behind auth, as the compose service runs it:
CLIMBWEB_ARCHIVE_URL=... CLIMBWEB_USER=u CLIMBWEB_PASS=p go run ./cmd/climbweb

Auth is required unless you pass -open: the archive is scraped data, so making it public should be a choice rather than something you fall into by leaving a variable unset. In compose it runs as the climbs service on ${CLIMBS_PORT:-8086}.

API additions it needed

• GET /v1/climb-riders?q= — riders that appear in climb data, ordered by how many timed results they have, with their best aSLP/VAM/W-per-kg.
• GET /v1/climb-records gained rider= and sort= (aslp, vam, watts, recent), and the rows gained climb_name, rider_slug, aslp and date.

rider_slug is what makes a rider comparable across sources: PCS writes "Tadej Pogačar" and the posts write "Tadej Pogacar", and the slug folds the accent so both are one rider. The archive registers slugify() as a SQLite function so that folding happens in the query rather than needing a stored, backfilled column.

MCP server (pcsmcp)

cmd/pcsmcp is an MCP (Model Context Protocol) server over stdio that lets LLM clients — Claude Code, Claude Desktop, anything MCP-capable — query the archive. It exposes four tools backed by the HTTP API, returning the API's JSON verbatim so source attribution stays visible:

Tool	What it does
search_climbs	Search the master climb list by name; returns slug, profile and coordinates.
get_climb	One climb's row + per-source measurements + unified source-attributed records.
search_races	Search stored races/stages (metadata incl. the general-info object).
get_race_results	Result rows by race/year/stage/classification and/or rider.

Configure with -url/-token or PCSMCP_ARCHIVE_URL/PCSMCP_ARCHIVE_TOKEN (the token falls back to PCSARCHIVE_TOKEN). Register it in Claude Code:

go build -o pcsmcp ./cmd/pcsmcp
claude mcp add pcsarchive -e PCSMCP_ARCHIVE_URL=http://127.0.0.1:8082 \
  -e PCSMCP_ARCHIVE_TOKEN=<token> -- /path/to/pcsmcp

Or, against the compose stack, without building anything (the image ships /pcsmcp; the token comes from the stack's .env):

claude mcp add pcsarchive -- docker compose -f /path/to/docker-compose.yaml \
  run --rm -T --entrypoint /pcsmcp archive -url http://archive:8082

Hydrating from the page history

Every raw page is archived, so when a parser is added or changed, the structured tables can be rebuilt from the pages already stored — no re-browsing. -hydrate re-parses every archived page (oldest first, all versions) through the same dispatch the ingest endpoint uses, then exits:

./pcsarchive -db ./pcsarchive.db -hydrate
# in the compose stack (stop the server first to avoid write contention):
docker compose run --rm archive -hydrate

It is idempotent (parsers upsert), so it is safe to re-run.

Notes

• No auth; binds to localhost. Don't expose the listener beyond localhost.
• Ingest bodies are capped at 32 MiB (mirrors the proxy's read limit).
• PCS pages embed per-request values (a token in the cx.procyclingstats.com link, a "Pageload" timing, Cloudflare parameters), so revisiting a page usually stores a new version rather than deduplicating. Bodies gzip to a fraction of their size, so the archive still grows slowly. Normalizing those fields before hashing would improve dedup — future work.

Tests

go test ./...