Building Verve's Audience Builder with Spark and Sampling

In 2017 I built the Audience Builder at Verve. It answered two questions anyone planning a campaign asks. How many people are in this audience? And how many ads can we actually show them?

Start With a Place

Every audience started with a location. The Audience Builder plugged into a POI library, so you could search for any point of interest. Then you defined the audience: people who went to that location a certain number of times.
The tool gave you two numbers back. Reach was how many unique devices were in that audience. Availability was how many ad requests those devices sent us. That's the total impressions you could actually serve.
The demo shows the scale. Ford Dealerships, November 2016 through February 2017: almost 2 million unique devices and more than a billion available impressions.

Fast Numbers First, Real Numbers Second

Nobody wants to wait on a full scan of every ad request just to see whether an audience is worth building.
So the Audience Builder worked in two passes. First, it quickly sampled a large database. Directional numbers, fast enough for the UI. Change the location or the visit count and the estimate moved right away.
Second, a Spark job did the real work. It pulled every actual IDFA (Apple's advertising identifier) that matched the audience. Those were the IDs we targeted, and the audience was sent to our ad server.

Directional Beats Exact

You don't need the exact answer to make a decision. You need a good-enough answer now, and the exact one when it counts. Sample for the UI. Run the batch job for the truth.
How long does your team wait on a query before it can decide anything? Email jonathan@rubygrowthlabs.com with "Fast estimates" and tell me what you're measuring.