Core to almost every measurement metric for online and mobile media networks are counts of “unique users” or “unique visitors”. It is vital that digital service providers can accurately track and measure how their massive visitor populations interact with the services provided, and the measure of choice is unique users over various dimensions (e.g., demographics, behavior, geography and time intervals).
This is a hard problem, and unsurprisingly even leading web analytics offerings like Omniture impose severe limits on the number of unique users and page views that they can report on, with the dreaded “database uniques exceeded” error message that analysts have grown to hate. In addition, these systems face the problems of unacceptable reporting latencies because of their batch processing paradigms and/or quality issues due to reliance on sample data. It’s common to hear our customers complain that they just cannot rationalize their Omniture and Google Analytics numbers. Of course, there was also the recent controversy with comScore and Nielsen reporting very different unique user numbers for Hulu during the month of April (over 40 million versus 8.9 million !!).
If you (1) have too much data, (2) would like to move from 12-24 hour old data to real-time, and (3) need to blend web analytics with additional data sources for a more comprehensive view, you should consider implementing the Truviso Continuous Analytics solution.
Continuous Analytics version 3.2 provides critical new functionality built specifically to solve the uniques problem. The unique-user tracking feature is designed for efficient analysis of tens of billions of unique values (that’s right, billions!), with an expectation of tens to hundreds of millions of actual users for any given category or dimension measured. This information is available instantly in real-time without full table scans, self-joins or 12-24 hour processing delays as is the norm for batch-oriented systems.
The feature itself is based on a novel insight – if the user-id can be mapped to a dense space (we provide connectors to handle cases where this is not true) then it is possible to efficiently represent and maintain the sets of users for each attribute, time dimension or category. Truviso has invented an innovative SQL-based adaptive data structure that is easily used in standard SQL queries – this structure yields high compression and can represent both sparse and dense sets very effectively. In addition, it allows for highly optimized manipulation of the sets – including the ability to add elements in a cumulative fashion, combine sets in an additive fashion for roll-ups, and compare sets to measure similarity.
The Truviso unique-user tracking feature transforms the challenging unique count problem into something that can be processed additively, resulting in enormous data processing productivity gains. It enables streaming on-the-fly unique-user computations, and affords easy roll-up of fine-granularity unique counts to coarser measures. Finally, its efficient set representation allows for a fast and easy implementation of higher-value correlation queries through the standard SQL query language. Some examples of these higher value queries include comparing the actual users who saw a given campaign across different time periods to enable better targeting, or measuring the similarity of the audience of two prime-time shows.
Ben Lorica at O’Reilly Radar posted a great article discussing how the Truviso approach to uniques allows publishers and marketers to adjust campaigns in real-time with A/B split bucket testing and referral analysis. With Truviso, marketers and business managers can really get a better understanding of loyalty and engagement, and relate that directly to a person — not just as a “new” or “returning” visitor statistic.
In the coming weeks we’ll be sharing more information on this feature, including example uses cases as well as mind-blowing performance results.
Features and Benefits