Back to the project in the portfolio

Anonymising production data

A scheduled GitHub Actions trigger runs, on the admin instance, a dump of the production database, its anonymisation, then its restore into the staging database. The two databases are only ever joined by a tunnel: nothing transits outside the instance.

Generic diagram: instances, repositories, channels and modules are named by function. The architecture is shown for illustration, independently of any organisation.

Pipeline step External service Cleartext data Anonymised data Admin instance
01 — dump, anonymisation, restore

Production → staging pipeline

The point that matters: cleartext data only exists in the top half of the diagram, inside the instance. What crosses over to staging is already anonymised.

GitHub Actions scheduled trigger admin instance 1 · extraction Dump script pg_dump Production data cleartext Anonymisation anonymisation script anonymised dump 2 · restore Anonymised data ready for staging Restore script pg_restore No cleartext data ever leaves the instance: the dump is anonymised in place before the restore. RDS production cleartext data tunnelled connection RDS staging anonymised data tunnelled connection
The two tunnels are never open for the same data: the left one only carries the raw dump into the instance, the right one only carries anonymised data out to staging.

All architecture diagrams