Moving customer reporting off the transactional database
Customer-facing indicators were timing out as analytical queries competed with live application traffic. I authored the architecture decision to separate the workloads, revised it to account for warehouse latency, and helped another engineer establish the service foundation. Rollout is in progress; end-state latency has not been measured.
Context
Customer-facing inspection indicators were timing out. Their analytical queries ran on the same transactional database that served live application traffic.
Problem
Moving the queries to an analytical warehouse would isolate the workload, but a warehouse has a latency floor. Serving a synchronous screen from it could replace one timeout with a different kind of slowness.
Decision
I authored an architecture decision that compared four combinations of service ownership and database choice. When I noticed that my first recommendation underplayed warehouse latency, I revised it to require a serving layer. The decision also included cost, tenant-isolation, security, and privacy risks, and was reviewed by seven stakeholders.
04Implementation
The service foundation was shared with another engineer. I contributed to the walking skeleton and data layer; the platform team continued to own the existing ingestion pipeline. Later product work was delivered by the broader team.
Impact
The architecture separates analytical reads from the transactional workload and is being rolled out to address the timeouts. The service foundation and curated, tenant-isolated warehouse tables are in place. No before-and-after indicator latency has been captured.