Like many technologists, I find customer interactions to be one of the most fulfilling parts of my job. The Internet world in particular is extremely liberating – customer problems are so unique that none of the standard assumptions hold any more. It turns out that for these folks, their business doesn’t fit the traditional model of a data warehouse that stores all the data from the beginning of time.

What really matters, what drives the business are the events of the last 60 to 120 days, and that is because these businesses operate at a blinding pace. Their user populations are constantly in flux, growing and changing in various ways. Heck, oftentimes their very business models change within a year. In other words, for these organizations recent performance is a better indicator of future results as opposed to the cautionary maxim of “past performance does not guarantee future results” that our friends in Wall Street probably wish they followed more closely!

It’s the recent data that’s most valuable, that’s most required and actively sought out by business managers making decisions on the front line day in and day out. Of course, it’s necessary to keep all historical data for various reasons such as compliance, accounting, and long-term trend analysis by the high priests of data.

These Internet companies, especially the trail-blazers, have realized that their enterprise data warehouses (EDWs) are needlessly over-provisioned. Many organizations are taking Bill Inmon’s advice and storing their most recent transactional data on an operational data store.

Truviso takes the ODS to the next level by providing live production analytics on top of transactional reporting, thereby complementing the EDW, and offloading the most challenging and useful workloads to a location where those workloads can be immediately used to improve operations. This model lets the EDW be used for its strengths, such as open-ended “blue sky” analysis incorporating years worth of historical data.

A critical aspect of Truviso Continuous Analytics version 3.2 that was architected for ODS environments is rolling data management – a feature necessary to efficiently move old data out of the system on a continuous, controlled basis. The actual mechanics are based on infrastructure that can be used to organize detailed as well as summary data into separate partitions (typically on a daily basis) thus letting older partitions be instantly groomed. In a typical implementation, the system is configured to periodically offload data onto a different system such as a data warehouse or a more cost-effective storage array, or even a Hadoop cluster. This reduces the cost of storage and maintenance, cuts hardware costs, and speeds up query processing by retaining only “recent” data in the operational analytics system.

Tell me your stories – I’d love to know what fraction of your analytics workloads focus on recent data.