Is a  Data Warehouse infrastructure,  a prerequisite for efficient BI ?

There is a widespread belief that the investment in an expensive DWH infrastructure will provide a strong BI competitive advantage. An attempt to identify issues that cannot be solved by a DWH infrastructure, is made.
 

Which are the arguments in favor of a comprehensive Data Warehouse (DW) (some call it EDW) ?

• A systematic approach toward data gathering & processing to produce business information
• Combined & effective analysis of various data sources, in a systematic way
• A single version of truth, derived via rigorous processing of data sources (according to a Sept 2008 Aberdeen report, 74% of businesses declared that ‘their data sources are not clean’ and this is a root cause for ‘multiple versions of truth’,. In the same report 59% of ‘best-in class’ businesses seek the ‘single version of truth’ for more than 2 years.)
• The ability to drill down on information rich dimensions, supported by the data warehouse and the ability to segment information in meaningful ways that may reveal valuable insights
• High level of automation in data processing & report generation, with positive effects in speed & quality (improvement in time to decision).

Which are the arguments against a comprehensive DW ?  

• Strategy, Business goals which drive BI needs are changing dynamically (one period you  focus on customer retention & value building, next you focus on channel cost cutting).
• Building a complete EDW requires a high budget & long-term efforts. Commonly,  Businesses require fast results from their projects (that is why the term quick-win is used frequently by Consultants). Therefore, they do not opt for building costly infrastructures. They frequently opt to analyze the specific data needed in any convenient structure, in order to reach quick an actionable insight.
• Certain subject areas do not have common information (in example CRM vs HR or CRM vs Financial Performance) with others.
• Many data marts are incompatible because they capture data at a different level of detail. Therefore connecting them in a DWH bus architecture, is neither feasible nor meaningful.
• The inherited perception of BI projects & infrastructures, as being hard to deliver to their end-user (the business analyst). Failure to manage expectations may lead to a major disappointment, if the investment is substantial.
• The poor quality of information, frequently affecting (if not minimizing), the value of findings and the level of trust. (a big issue, which is often triggered by legacy systems & data sources). The lack of systematic and end-to-end data stewardship in the information value chain, may affect the value of a DW infrastructure.
• Moreover, certain ΒΙ projects are predominantly qualitative (i.e. competitive positioning analysis) or their quantitative part is based mainly on unstructured data (i.e. info gathered from financial statements or sectoral reports).
• In periods of financial crisis cost cutting, smaller & focused investments are popular.
• Investments in infrastructures should at least be balanced by investments in analyst skills (as web analytics guru, Avinash Kaushik, asserts).
• Alternative reporting ways (reporting modules within operational systems (CRM/ ERP), isolated departmental on-site BI applications, reports received via email/ mail), are traditionally used and may cover, comprehensively or partially, the BI needs.

Information gathered by the market or the customer may have an incompatible structure compared to that of the DW:
• Competitor performance data may not be at the same level of detail at which internal performance data is structured & stored
• Customer survey data may be anonymous, thus they cannot be linked to the DW customer records
• Customer behavioral data on the Web channel may also be anonymous
 

So what is the answer, to the question above (placed on the title).

 Urgent business issues affecting the bottom line figures, require quick solutions. A high priority analytics project may require various data sources which may not be available in the existing DW. What should management do? Wait for the DW to develop the relevant subject area (which might take 6-12 months)? Commonly, such projects are run without DW support. An independent database or data mart may be built, by extracting data directly from operational databases, to support the analytics project.

 On the other hand, the systematic building of a BI culture & infrastructure, with small prioritized steps (and controlled cash outflows), is a necessary step to build sustainable competitive advantage. All relevant investments should be weighed against expected improvements in customer satisfaction (according to the above mentioned Aberdeen report, ‘best in class’ businesses (vis-à-vis their BI maturity) achieved an average customer satisfaction increase of 3.4 %, nine times greater than average businesses, during the last 24 months), employee productivity, cost containment, though quantification of business benefit is not an easy task.