Top Of The News - Apps & Analytics:

Study: Demanding Users, High Data Volumes Bad For Data Warehouse

At the Data Warehouse Institute (TDWI) World Conference, SAND Technology Inc released the results of an independent research study by Dynamic Markets Ltd entitled "The Crumbling Foundations of Data Warehouses." The research reveals that data warehouse managers are coming under increased pressure as users' demands grow at a time when large volumes of data are seriously decreasing the performance of data warehouses.

According to the survey of data warehouse managers in the Top 2000 UK companies, 88 percent of respondents think that the original architecture of their data warehouse will have to change to accommodate the business demands being placed on it. These demands come from two sources: the requirements of users within the business, and the pressures resulting from the way in which the data is stored within the architecture.

Summary results show that fully 97 percent of data warehouse managers feel that their users are becoming more demanding, and over three quarters (86 percent) are having to restrict user queries in order to alleviate the strain on both the technology and their team. At the top of the list of areas in which users are being demanding is in requirements for query performance, with almost two thirds (62 percent) of users asking for faster retrieval of data, followed by the growing complexity of requests (51 percent), and mounting interest in exploring older data (41 percent).

In addition to user pressures, the volume of data stored, and the lack of alignment between data value and storage methods, is adversely affecting data warehouse performance, leading more than 60 percent of respondents to expect they will increase the manpower allocated to their data warehouse in the next three to five years. The challenge of tiering the data according to business value in order to ease the pressures of retrieval requirements still resonates loudly with the sample: almost half (48 percent) think that only a quarter of the total data stored is queried on a regular basis, yet less than 20 percent tier their storage according to business value.

Commenting on the results of this research, Robert Thompson, vice president of Marketing at SAND Technology said: "It is alarming to see the impact of the pressure from both users and increasing data volumes on the data warehouse. More than two thirds of the respondents admitted that these pressures are negatively affecting performance, and that the traditional "one-size-fits-all" architecture is struggling to keep up with the demands being placed upon it. SAND believes the solution lies in an intelligent data warehouse design that acknowledges the changing business value of data and applies resources accordingly. Information Lifecycle Management (ILM) principles are the core of best practices in other data intensive applications and, we believe, can offer the solution to declining data warehouse performance."