The $1.4M/Year Bill Nobody Was Watching

Context

A 20-store RFID loss-prevention proof-of-concept at a major retailer: RFID readers embedded in stores tracked tagged products, streaming events through GCP Pub/Sub into a real-time pipeline. I was lead full-stack engineer on the integration of RFID tracking, video analytics, and the front-end applications.

Before writing code, I did the homework: a 17-page impact analysis covering shrink at this specific retailer — what was commonly stolen, how it was stolen, profiles of the people doing it, and the false-recognition caveats of the technology involved. Two findings shaped everything after. First, theft was dominated by rings — the same people stealing the same things repeatedly — which meant the real value was in building cases, not firing alerts. Second, done correctly, the program was worth $500–700M in recovered shrink.

The False-Flag Problem

The detection algorithm ran on-prem, in the readers, and it had a fatal mismatch with the hardware. Zebra RFID antennas — even the four-dish units reading in four directions — can only report about 200 labels at a time. A typical aisle holds well over a thousand tagged items. The algorithm compared the list of labels read one second against the list from the previous second and flagged anything that vanished — but with 5× more items than the antenna can see at once, items "vanish" constantly. The result was hundreds of false flags per second, per aisle. Of every 10,000 events streaming from stores into Pub/Sub, roughly 10 were real.

Following the Money

Nobody asked me to look at the bill — I got curious about what all those false flags cost. I built a GCP usage calculator in Excel to validate my suspicion, modeling the event volume against Pub/Sub and pipeline pricing. The answer: ~$119K/month, a ~$1.4M/year run rate for 20 stores, before any rollout — nearly all of it spent transporting noise.

The calculator also exposed the architectural insight: loss prevention doesn't stop theft in the moment — nothing they can do interrupts someone walking out with an item. They build cases. Real-time alerting was expensive urgency with no operational value.

The Fix: Windowed Differentials

I redesigned detection around time windows instead of second-over-second diffs:

1. Dedupe reads into one-minute SKU sets. Within a minute, the antenna gets ~60 chances to see each label — a set absorbs the 200-label sampling limit instead of fighting it.
2. Compare minute over minute. A SKU missing from the next minute's set has been missed 60 consecutive times — now that's signal.
3. Confirm for five minutes, then send one batch event. If the SKUs are still missing, the event ships — carrying the timestamp of the minute each item first went missing, so CCTV footage could be pulled for the right moment.

flowchart TB
    subgraph Before ["Before — second-over-second diff"]
        direction LR
        R1["RFID reads<br/>(200-label cap per antenna)"] --> D1["Diff vs previous<br/>second"] --> F1["Hundreds of false flags<br/>per second, per aisle"] --> P1["~10 valid per 10,000<br/>events to Pub/Sub"]
    end
    subgraph After ["After — windowed differential"]
        direction LR
        R2["Reads deduped into<br/>1-minute SKU sets"] --> D2["Minute-over-minute<br/>differential"] --> W["Missing 60 reads<br/>= likely gone"] --> C["Still missing at 5 min:<br/>one batch event,<br/>first-missing timestamp"]
    end
    Before ~~~ After

Detection quality went up — fewer, truer flags with accurate timestamps — while the event volume collapsed at the source. The bill dropped ~99%, to ~$1.3K/month.

From Detection to Case-Building

Downstream, I built the loss-prevention dashboard: likely-stolen items with the timestamps of when each product left its aisle and when it left the store, linked to the exact CCTV footage worth reviewing. Camera systems logged the faces of people exiting at the time of a theft, so repeat incidents could be correlated — a matching face across thefts flagged the individual as high-risk pending human confirmation of the pattern.

My proposal took the case-building arc to its conclusion: once a pattern was human-confirmed and the individual banned, their re-entry could trigger an immediate alert to police — turning a system that documented losses into one that ended repeat-offender runs, the core of the $500–700M opportunity.

flowchart TB
    A["RFID event:<br/>item left aisle → left store<br/>(first-missing timestamp)"] --> B["Pull CCTV footage<br/>at exact timestamp"]
    B --> C["Log faces of exits<br/>at time of theft"]
    C --> D{"Face matches a<br/>prior incident?"}
    D -->|no| E["Retain incident log only"]
    D -->|yes| F["Flag individual<br/>as high-risk"]
    F --> G{"Pattern confirmed<br/>by human review?"}
    G -->|no| E
    G -->|yes| H["Store biometric record<br/>· ban from store"]
    H --> I["Re-entry detected"]
    I --> J["Immediate police alert —<br/>before the culprit leaves"]
    style G stroke-width:3px

Outcome

• ~$1.4M/year saved, unprompted — from treating the system's economics as part of the system
• False-flag noise replaced by fewer, truer detections with case-ready timestamps
• Architecture hardened from POC toward production, with a sized roadmap from detection to case-building