The collection pipeline

The collection pipeline

Data flows through several stages from your site to a live report. Understanding this pipeline helps you debug missing or delayed data.

The five stages

Stage 1: Your site fires a hit

Your JavaScript (Web SDK or AppMeasurement) collects data and sends it to Adobe's Edge Network.

// Web SDK
alloy("sendEvent", {
  xdm: {
    eventType: "web.webpagedetails.pageViews",
    web: {
      webPageDetails: {
        name: "Contact Form Page",
        pageViews: { value: 1 }
      }
    }
  }
});

The hit carries: page name, props, eVars, events, referrer, timestamp, visitor ID.

Stage 2: Edge receives and routes

Adobe's Edge Network receives the hit in milliseconds. It:

• Validates the request (is this a real visitor? is the report suite ID valid?)
• Stitches it to a visitor ID (ECID or user ID)
• Routes it to the correct report suite

Latency: ~100-500ms

Stage 3: Processing rules and VISTA

Before data lands in the report suite, it can be modified by:

• Processing rules (UI-configured transformations)
• VISTA rules (server-side code changes)

Example: a processing rule might lowercase all page names or extract domain from referrer.

Latency: ~1 second

Stage 4: Data validation and filtering

Adobe checks:

• Is this a bot? (if bot filtering is enabled, hit is discarded)
• Is this IP excluded? (if yes, hit is discarded)
• Are all values within limits? (strings > 255 bytes are truncated)

Latency: ~2-5 seconds

Stage 5: Report suite database

Data lands in the report suite database and becomes queryable in Analysis Workspace and real-time reports.

Latency: 5-30 minutes (reports can take up to 24 hours to fully process)

Data loss points

Your data can be lost at several stages:

• Client-side: Page never loads, JavaScript never fires → no hit sent
• Edge validation: Invalid report suite ID or visitor ID → hit rejected
• Bot filtering: Hit marked as bot → discarded before report suite
• IP exclusion: Visitor IP is excluded → discarded
• Processing rules: Rule deletes the hit
• VISTA rules: Rule deletes the hit

Real-time vs. finalized data

Real-time reports show data as it arrives at Edge (~30 seconds latency).

Standard reports show data after processing, validation, and deduplication (~1-24 hours).

This is why your real-time report count can differ from your finalized report count hours later.

The Example Super pipeline

1. Site visitor fills contact form on Example Super site
2. JavaScript fires trackEvent() with form data
3. Hit sent to Adobe Edge in Sydney
4. Edge validates and routes to "Example Super" report suite
5. Processing rules check: is page name uppercase? If not, make it.
6. Bot filter runs: is User-Agent a known bot? No → continue
7. Data lands in report suite database
8. Analyst queries "Contact Form Submissions" in Workspace 10 minutes later → data is there