Inside the Newcastle twin: how a synthetic city is built, run and checked

The pipeline

Previously I showed what the twin looks like and how close it gets to the real city. Underneath is a pipeline of scripts. Each stage reads what the stage before produced, and each can be rerun from the raw downloads. Nothing in the chain is edited by hand.

Every file the pipeline writes, 968 of them, is listed in a manifest with its hash, row count, the script that produced it, its source, retrieval date and licence. Files built from OpenStreetMap carry its share-alike licence (ODbL); the rest are CC-BY. A check fails the build when a file changes without its manifest entry changing.

The study area

The twin covers five council areas: Newcastle, Lake Macquarie, Maitland, Cessnock and Port Stephens, 4,086 km² in all, made of 1,500 census areas (SA1s). The map extent is derived from those council boundaries with a 5 km margin, never typed in. That rule came from a mistake: early on, a study area typed into a script as a rectangle left 87 of the 1,500 census areas outside the road network for three stages of the build before anyone noticed. Now no coordinate may appear in the code, and a check counts any that do.

The network

OpenStreetMap is downloaded in ten themed layers (roads, rail, footpaths, buildings, places and so on) over tiles covering the extent, merged, and converted into a MATSim network with pt2matsim, in the local map projection so every distance is in metres.

Walking and cycling have their own network. 40,203 footpaths, cycle paths, stairs and tracks are added as walk and bike links, and each path's own OpenStreetMap tags (foot=, bicycle=, access=) override the default for its class: 4,484 paths were rewritten and 2,252 dropped that way. The network grew from 181,892 to 368,230 links. Speed limits come from the transport department's speed-zone data, and every link carries a gradient from the Copernicus 30-metre elevation model, which slows walkers and cyclists going uphill.

The timetables

Public transport runs from the transport department's open timetable feeds (GTFS). Fifteen feeds - the 2026 timetable, four earlier eras back to 2014 for the light rail question, and the scenario variants - are mapped onto the network in one pass, so every stop sits on a link and every route follows real streets and tracks, with no stop left unmapped.

Mapping runs once per network build, on purpose. The mapper is not reproducible: two mappings of the same inputs place nearly one route in five on different streets. Weekday, Saturday and Sunday timetables and the scenario variants are cut from the one mapped schedule instead of being mapped again, so any two runs being compared share the same routes. On a weekday that schedule has 1,448 bus trips, 332 train trips, 252 light rail trips and 107 ferry crossings, and each vehicle type carries its seated and standing capacity.

Signals and level crossings

The fourteen signalised intersections on the light rail corridor are modelled as signal systems. New South Wales runs adaptive signals (SCATS), and the transport department would not release the operated phase plans or offsets. Rather than guess a fixed timing, the twin runs the adaptive logic SCATS publishes: after each cycle an intersection measures how saturated each approach was, moves its cycle length by 6 seconds towards a target saturation of 0.90, within 30 to 150 seconds, and re-splits the green time between approaches. The offsets that coordinate neighbouring intersections are not adapted, because they were not obtained, and that is recorded rather than filled in.

Freight trains are handled the same way: by what the road network actually experiences. Putting coal trains on the passenger model's tracks would invent interactions nobody observed, so the two level crossings on the network close their boom gates at the times trains pass. The closures come from the mapped passenger timetable plus the freight movements in a published survey of the coal chain: 405 closures on a weekday, written as 2,478 timed changes to the road links.

The people

Households and people are drawn for each census area so that, added up, they reproduce what the census published there: household size, dwelling type and cars per household; age and sex; work, study and income. The result is 612,667 synthetic residents against the 611,915 the 2021 census counted, with the household-size distribution within 0.6 percentage points of the census in every band.

Two things are measured rather than assumed. Driver licences come from the transport department's licence statistics divided by the resident population, for each council area and age band: 78% of 18 to 24 year olds hold one, 94% of 25 to 34 year olds, all of those aged 35 to 44, and 51% of those over 85. And households own the cars the census says they own, as named vehicles: 81,384 households, a third, have more licensed drivers than cars, so their drivers compete for them during the day. Each person's income, from the census band, sets how much a dollar matters to them when they choose.

Their days

Each person gets a day built from home-based tours: out to work or study and back, a stop at the shops, taking a child to school, a work trip in the middle of the day. The number of trips is solved so that the population makes 3.473 trips per person per day, the travel survey's figure; it realises 3.470. Destinations are real buildings and places from OpenStreetMap, chosen with a distance model whose decay is solved so that trip lengths match the survey for each purpose and home council. People without a car choose nearer destinations: their decay per kilometre is 1.25 times faster, solved so that both groups together match the survey.

Some trips are tied to other people. A child's trip to school is bound to a parent with a licence and a car, and a lift is bound to a driver who owns one, so a passenger in the simulation can only ride with a real driver who is making that trip. What nobody is given is a mode.

The simulation

The engine is MATSim. Everyone's plans are executed together on the network, in a queue simulation where every mode takes up space: walkers and cyclists on paths and roads, cars, taxis and trucks in traffic, buses, trains, trams and the ferry on their timetables, stopping for signals and boom gates. Nothing teleports.

Each executed plan gets a score. Time spent at activities earns 6 utility points an hour; time spent travelling costs, with values of time from the literature by purpose ($18.60 an hour for a commute); waiting and walking to a stop cost more than riding, and each transfer costs the equivalent of 8 minutes, a value swept from 3 to 15. Money counts too: Opal fares from the published fare schedule, and parking priced by how many jobs are nearby, up to $3.20 an hour, at 7,710 parking facilities.

Between rounds, people revise their plans: 70% of revisions pick among remembered plans by score, 15% try a new route, 10% a new mode for a tour, 5% a new departure time. After 80% of the 250 rounds, only remembered plans are used, so the last rounds show where the population settled.

Several behaviours needed code of their own, written as additions to MATSim in Java. A passenger getting a lift boards the driver's actual car, and the driver detours to pick them up. Taxis are a finite fleet - 800 at full scale - so a request can be refused and the traveller has to choose again. Household cars are shared, so a driver may find the car already out. Crowded buses and trains feel worse, cycling in heavy traffic feels worse, and hills slow walkers and cyclists. The constant each mode carries in the choice model is kept at its prior value, so that a mode which misses its target shows a missing behaviour instead of being tuned to fit.

A run simulates a quarter of the population, drawn by household so families travel together, with road capacity, seats and standing room scaled to match.

Checking a run

Each of the twelve modes has a target on the basis its real figure is published on: shares of residents' trips for car, lift, walk, bike, motorbike, taxi and bus, from the travel survey and, for taxis, industry trip counts; weekday Opal boardings at the train stations, on the light rail and at the ferry wharves; road counts for trucks; and the timetable and coal-chain survey for freight trains. Every 100 rounds the run reads all twelve, and a mode 20% or more off its target is the signal to stop and find its cause. Of 210 validation targets, 143 are held back and have never been read by the calibration code; they are opened once, at the end.

Runs are grouped into families: any change that can move the results starts a new family, and runs are never compared across one. Within a family, differences are read as bands. MATSim runs are not bit-reproducible - three runs of one build, same seed, diverge by the second round because parallel threads draw random numbers in a different order - though their mode shares came out the same.

Running it

Everything runs on one Windows workstation with 64 GB of memory: 16 threads for the simulation, a 48 GB Java heap for a quarter-sample run, and about 28 hours for 250 rounds. Before a full run, a four-round probe of about 45 minutes measures what the change costs, and the full run starts only with an approved, measured cost.

The workstation was the least reliable part. Runs were killed by a forced Windows Update restart at round 237 of 250, by a disk controller crash, and by a shutdown from the Start menu. Each became a rule in the run launcher: it now refuses to start unless updates are paused past the run's expected end, it launches runs detached from any session, and it can resume a dead run from its last checkpoint.

The record

Every value a run can be sensitive to - 600 of them - is declared in a registry, with its units and where it came from. Values that are assumed or taken from the literature carry a range to sweep them over, or a stated rule for holding them fixed. A check counts any number a script decides for itself, and it stands at zero. Every decision is written into a dated record that is never rewritten, only added to.

What is still missing

• The walk to a parked car. The model charges nothing for it, so people drive 59% of trips under a kilometre.
• Public transport routes for everyone. Almost half of the attempts to plan a public transport journey find no route.
• Choices for people without a car. On a trip no driver is bound to and no transit serves, they have walking, cycling or a taxi, and they cycle and take taxis far more than real people do.
• The run-to-run band. How far readings of the same model spread across seeds is declared but not yet measured.
• The signal offsets that coordinate intersections, which were not released.

The next post adds a second city on the same framework, and describes how both were built with AI coding agents.