Open-source release: pipeline code, raw logs, derived data, licences, tests

Adds src/ (scripts, raw measurement logs, derived data, report source),
tests/, LICENSE (MIT, code) and LICENSE-DATA.md (CC BY 4.0 for the survey's
data; third-party terms). The report is updated to the current site; the
v1 and v2 data hand-off files are unchanged.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

README.md

@@ -1,19 +1,71 @@
-# Cable latency survey — data hand-off
+# Cable Latency Survey
 
-Deep-sea cable latency survey: measured latencies between sites, published cable geometry, and the routed path for every pair on four map models.
+Measured round-trip times between cities, compared with the minimum
+propagation time along submarine-cable routes on four models of the Earth's
+surface: a sphere, a North-pole azimuthal equidistant (AE) map, an AE map
+centred on 0°N 0°E ("dual-pole"), and a flat Mercator map. A round-trip time
+below the fibre propagation floor along a model's route is inconsistent with
+that model.
 
-| File | Size | What it is |
+- Report: https://pingstats.aethercosmology.com (also `cable_latency_survey_report.html`, a single file that works offline)
+- Code: MIT (`LICENSE`). Measurements and derived data: CC BY 4.0, with third-party terms (`LICENSE-DATA.md`).
+
+## Files
+
+| Path | Contents |
+|---|---|
+| `cable_latency_survey_data_v2.json` | **Current data hand-off.** 29,341 measurements, 305 sites, every routed path and cable geometry (`[lon, lat]`). The `revision` block lists the changes from v1. |
+| `cable_latency_survey_data.json` | v1: the same measurements without coordinates. Kept for reference. |
+| `cable_latency_survey_report.html` | The report, identical to the website. |
+| `src/` | Pipeline code, raw logs and derived data (below). |
+| `tests/` | Headless-browser checks of the report (`tests/run.sh`). |
+
+### Raw measurements
+
+| Source | File |
+|---|---|
+| Globalping probes, runs 1 and 2 | `src/data/live_20261023T1812Z.json`, `src/data/live_run2.json` (each result with its Globalping measurement ID) |
+| Vultr servers (8, 2026-10-23 21:23–22:30 UTC) | `src/data/vps/<server>/ping.log`, `src/data/vps/<server>/mtr.log` (traceroutes) |
+| Residential connection, Michigan | `src/data/home/ping.log` (location withheld) |
+| RIPE Atlas sea crossings | `src/data/trace.json` (hops, timings, measurement and probe IDs) |
+| WonderNetwork, 2020-07-19/20 | per-pair statistics `src/data/wn_clean.json`; archive downloaded by `src/fetch_raw.sh` |
+
+## Pipeline
+
+Run from `src/`. The measurement steps contact external services and are not
+needed to rebuild the report.
+
+| Step | Script | Output |
 |---|---|---|
-| `cable_latency_survey_data_v2.json` | 39.9 MB | **Current.** 29,341 measurements, 305 sites. Adds `cable_geometry` (608 cables, drawn routes and landing points as `[lon, lat]`) and `cable_routes` (the point-by-point path behind every cable-route distance, each step tagged cable / cable-to-shore / overland). Replaces v1; the `revision` block lists what changed. |
-| `cable_latency_survey_data.json` | 20.4 MB | v1. Same measurements, cable lengths and route names, but no coordinates. Kept for reference. |
-| `cable_latency_survey_report.html` | 2.6 MB | Self-contained report. Open in a browser. |
+| Distance on each model, constants | `models.py` | — |
+| Globalping measurements | `measure.py` | `data/live_*.json` |
+| Vultr servers: create, measure, collect, delete | `vps.py` | `data/vps/` |
+| Vultr logs to measurements | `vps_analyze.py` | `data/vps_cells.json` |
+| WonderNetwork archive: statistics, mislocated servers | `analyze_wn.py` | `data/wn_clean.json` |
+| Published cable lengths on each model | `cables.py` | `data/cables.json` |
+| RIPE Atlas sea crossings | `trace.py` | `data/trace.json` |
+| Cable-and-land routing on each model | `network.py` | `data/network.json` |
+| Best attainable flat arrangements | `flatfit.py` | `data/flatfit.json` |
+| Network attribution from public BGP data | `asmap.py` | `data/asmap.json`, `data/rpki.json` |
+| Shape fit without positions | `pingmap.py` | `data/pingmap.json` |
+| North-pole AE longitude solver (hand-off only) | `solver.py` | `data/solver.json` |
+| Report | `build.py` | `site/index.html`, `web/index.html` |
+| Data hand-off | `export.py` | `handoff/cable_latency_survey_data_v2.json` |
 
-Top-level keys in the v2 JSON: `dataset`, `generated_utc`, `revision`, `constants`, `maps`, `test`, `sources`, `summary`, `sites`, `excluded`, `cable_crossings`, `cables`, `cable_geometry`, `cable_routes`, `flat_map_solver`, `measurements`.
+## Reproduce
 
-Every stored distance in v2 recomputes from the coordinates in the file to within 0.05 km. Overland legs are straight lines and may cross water; that only shortens routes.
-
-Download a single file with the download button on its page, or clone:
-
-```
+```bash
 git clone https://git.aethercosmology.com/toby/cable-latency-survey.git
+cd cable-latency-survey/src
+pip install -r requirements.txt
+./fetch_raw.sh          # TeleGeography, WonderNetwork and BGP inputs (not redistributed)
+python3 build.py        # the report, from the files in data/
+python3 flatfit.py      # flat arrangements
+python3 export.py       # the data hand-off
 ```
+
+`build.py` needs only `data/` and the TeleGeography files; it does not contact
+any measurement service. `fetch_raw.sh` reports whether the TeleGeography data
+it downloads matches the snapshot used (`src/raw/tg.sha256`).
+
+Browser checks: `cd tests && npm install && CH=/path/to/chromium ./run.sh`.