Advertising Event Data: Define the Contract Before Building the Pipeline

Engineering notes with synthetic examples. Version scope and references are included in each article.

When onboarding an advertising data source, the easiest milestone to deliver is “the JSON now matches our common format.” The harder question is whether the transformed record still represents the same business event. A complete set of fields does not make data usable. The first deliverable should be a data contract that can be enforced and reconciled.

This article uses a fictional click-reporting example. It does not describe any company’s internal protocol.

What does one record represent?

Suppose a user clicks an ad once, but a response timeout causes the client to deliver the report three times. The ingestion layer receives three requests for one business event. Generating a fresh UUID on every arrival makes those deliveries look like three different events downstream.

Start by distinguishing four identities:

Field	Question it answers	Suitable for event deduplication?
request_id	Which ad request?	Usually not; one request may contain multiple ad opportunities
impression_id	Which opportunity or impression, as defined by the protocol?	Cannot directly replace a click identifier
event_id	Which business event?	Yes, if its uniqueness scope is explicit
delivery_id	Which delivery attempt?	No; retries should retain the original event_id

OpenRTB gives requests, impression objects, and bids their own identities. The auction itself operates at several levels of granularity; one identifier cannot serve as a universal primary key for every downstream event. IAB OpenRTB 2.6

For this design, the deduplication key is (source, event_type, event_id), while delivery_id tracks transport attempts. If a producer has no stable event ID, agree on a deterministic generation rule and document its limits. Concatenating “user + ad + timestamp rounded to a second” can merge two legitimate actions within the same second. That is not a safe default to introduce silently.

Preserve at least two time axes

Keep both the business occurrence time, event_time, and the platform receipt time, ingest_time. Processing timestamps can be added, but must not overwrite the original times.

{
  "source": "demo-media",
  "event_type": "click",
  "event_id": "evt-demo-001",
  "event_time": "2026-10-01T10:00:00Z",
  "ingest_time": "2026-10-01T10:00:08Z",
  "schema_version": "3",
  "mapping_version": "12"
}

These values are illustrative. ingest_time - event_time describes arrival delay, but a negative result should not automatically cause a record to be discarded: the client clock may be wrong. Measure negative delays, very late arrivals, and missing timestamps separately. Then apply the contract’s quarantine or correction policy. Store a corrected timestamp in a separate field so that later changes in historical reports remain explainable.

A common event envelope can borrow CloudEvents’ consistent representation of metadata. Business meaning still belongs in the application contract: a standard cannot decide what counts as a valid click. CloudEvents

Represent unknown values instead of guessing

“Amount missing” and “amount equals zero” are different. So are “user unknown” and “user is an empty string.” Mapping rules should define types, units, null semantics, and handling for unrecognized enum values. For example, represent money as an integer in the currency’s smallest unit and carry the currency separately. Preserve an unfamiliar enum’s original value and mark it unknown instead of mapping it to success.

Versioning also needs separate dimensions: a schema describes the target structure, a mapping describes the source-to-target transformation, and a rule describes a business decision. A mapping that confuses seconds with milliseconds can corrupt the pipeline without changing the schema. Attach all three relevant versions to the output so that replay can reproduce the original result.

I prefer four error categories: unparseable input, missing required identity, invalid business values, and acceptable but incomplete information. The first three go to a quarantine stream. The last may continue with an explicit quality flag. Quarantine needs an owner, a retention policy, and a replay path after repair; it must not become an unmonitored sink for errors.

Acceptance criteria beyond a success rate

Build a fixture set covering normal events, duplicate deliveries, out-of-order arrivals, missing IDs, unknown enum values, events crossing a day boundary, oversized fields, and invalid encoding. Then verify the following:

Replaying one event twice produces one business outcome, while the delivery audit retains both attempts.
Every input belongs to exactly one category: accepted, intentionally filtered, or quarantined. Their counts add up to the input count.
Each target field can be traced to a source field and mapping version. Defaulted values have their own counters.
Running old and new mappings on the same fixtures produces differences grouped by field and reason.

Monitor delivery volume, valid events, deduplicated events, and successfully joined events separately. A single “success rate” can hide missing IDs behind successful parsing and make retry traffic look like business growth.

The useful artifact is a reviewable contract, fixture set, and error taxonomy—not just a conversion function. Those artifacts make the validation process reusable for the next source.

References

IAB Tech Lab: OpenRTB 2.6 , for the boundaries between requests, impression objects, and bids. The event contract here is an independent design example.
CloudEvents , for the purpose of consistent event metadata.