Data Engineering

A per-request warehouse join was scanning 658.8 MB against a hard 953.7 MiB per-job ceiling, on a table growing with every new customer. I measured it, moved the join into the pipeline that already ran it, deleted the cache that was hiding the problem, then audited my own migration and found 79% of production rows never backfilled, plus 9,299 alerts that had never appeared in the product at all.