Data warehousing = publishing

He stated that data warehousing specialists do not "make" the data, but are rather purveyors of used data.

"Part of the data warehouse mission means you collect input from a variety of sources, edit them for quality and consistency, and publish the data on a regular basis," he explained.

According to Kimball, it was first discovered in the 1980s that data could be repackaged and redistributed, with as much as 5MB being collated at a time.

In the 1990s, customer profitably began to play a larger role and volumes of between 40GB and 50GB were already being redistributed.

The 2000s saw data warehousing emerging as the ideal platform for business intelligence. "Data warehousing is the shiny, perfect, performance platform for BI," he said.

"In 2010, BI will become the platform by which to run a company as data is in the spinal cords of all users."

Kimball highlighted numerous data warehousing constraints, and using his electrical engineering background, showed delegates how these can be overcome using engineering principles.

Estelle de Beer, BI practice manager at Sybase`s BI Practice, also spoke at the event. She explained how a few years ago users and their needs started changing at the same time data started growing.

"I was introduced to a product called Sybase IQ nine years ago, and did research on it to find out how it was different to other business intelligence engines. I found out that a lot of different global leaders from around the world were making phenomenal claims about it," she said.

Some of these claims included The Bureau of Transportation seeing query speeds that were 20 50 times faster and time to add a column reduced from four hours to 15 minutes. Barclay`s Bank pronounced that Sybase IQ enabled an average of 70% increase in performance as well as 30% 40% data compression capabilities.

De Beer also quoted analyst firm Bloor as saying: "We would not expect any longer learning curve for Sybase IQ than any other database product, due to the fact that it is based on the SQL language."

"Don`t make the assumption that what you`ve got is always good enough. Challenge those thoughts and see what else is out there," De Beer concluded.

The event also included a panel discussion entitled "Lessons from the frontline: the do`s and don`t of making BI work."

Alan Cowley, CEO of Sybase SA chaired the discussion. "BI might be a maturing technology but it is far from hassle-free. Tedious technology issues including the need for comprehensive data cleansing and integrating incompatible systems are still a big part of nearly all BI projects. This is a candid discussion with IT executives as they map out their strategies for navigating around user resistance and resentment, creating quick ROI wins and managing over-enthusiastic vendors," he explained.

The panel included Clint Vosloo, director of BI In The Box, Matilda Nell, manager of reporting applications at Vodacom SA, Kevin Griffiths, executive: BI of Telkom, as well as Kimball and De Beer.

A separate Q & A session with Kimball included questions about whether BI lives up to expectations, ROI and deployment issues, and the BI lifecycle.

The afternoon session consisted of a three hour workshop by Kimball on the classic best practices for building effective data warehouses. He demonstrated how the most important data warehouse subject areas are implemented using the techniques of dimensional modelling.

The basic concepts of dimensional modelling were outlined, including slowly changing dimensions, which show how changes are tracked in time; and conformed dimensions, which solves the integration problem.