Friends Don’t Let Friends Overpay for BI

Business Intelligence projects are famous for low success rates, high costs and time overruns. The economics of BI are visibly broken, and have been for years. Yet BI remains the #1 technology priority according to Gartner. We could paraphrase Lee Iacocca and say: People want economical Business Intelligence solutions and they will pay ANY price to get it.

Nobody argues with the need for more Business Intelligence; BI is one of the few remaining IT initiatives that can make companies more competitive. But only the largest companies can live with the costs or the high failure rates. BI is a luxury.

I believe that the bad economics of BI are rooted in the IT department/BI vendor duopoly on BI infrastructure. This post focuses on IT’s inability to deliver efficient BI projects; I will write about the BI industry in my next blog:

There are three fundamental reasons why IT departments in their current form fail to deliver economical BI solutions:

1) They don’t understand elastic scale

IT departments are good at scaling: adding more and more hardware and software but scaling makes sense for tasks that are highly predictable. Given the ad hoc nature of BI we not only need to increase the compute power when we need it for a complex queries but we also need to be able to decrease the compute power when it’s not needed to keep the costs down. Elastic is more important than scalable. And this precisely why internal BI solutions will always be either too expensive or too slow for complex queries…

2) They try to control BI with a single version of the truth

While the volatility of business environment is increasing the IT departments are trying to button up the business knowledge (data, metadata, processes) into a top-down, inflexible and lengthy process that should produce a single version of truth. The problem is that the underlying business is changing so rapidly that by the time this is done the resulting analysis and reports are not correct anymore and the BI project becomes shelfware.

3) They cannot measure success of BI

“If you can’t measure it, it’s not worth doing!” is one of the selling point of BI but it is difficult to measure the success of BI projects. IT delivers on initiatives that are quantifiable (throughput, response time, performance, data sizes) and since the data size is one of the few easily measured aspects of BI it is the only metric where IT can claim success. This is why we often read about terabyte and petabyte datawarehouses. But it is a small portion of the BI market (2%) and they happen to be places where data goes to die.

Comments

  1. Good article. I would add that there aren’t only proprietary solutions on the BI market, but also open source.

    I think Open source software could be more economical than proprietary tools, just looking, for example, at the licence cost. But there is a lot more to talk about. And these savings are important in these times of IT budget restrictions.

    Thanks for the article.

  2. All of the reasons cited are certainly true, but the overarching reason BI hasn’t been successful is that it failed to deliver something useful and relevant to the work that people do. Prior to the rise of data warehousing, analytical systems were sponsored by and supported by business units, not IT. Because data warehousing required the participation of IT to both host a large database and provide access to source systems that weren’t often very stable, the entire enterprise became an IT project. As a result, IT enlisted the services of the large consulting firms, all of whom turned the whole mess into a giant IT PROJECT. That’s where everything went off the rails and we saw the introduction of foolish ideas like “get the data to the right person at the right time so they can make better decisions,” or the single version of the truth dogma.

    The BI software vendors were happy to sell big contracts to IT instead of a handful of seats to the finance department. The analyst firms and industry “influencers” were all in collusion, wittingly or not.

    This whole drama needs a jolt of reality. BI needs to return to people, not databases. As Roman correctly points out, the mega-warehouses represent a tiny fraction of the population and prove nothing. And what’s worse is that the industry seems to be careening from one next big thing to another – SaaS, unstructured data, operational BI – to name a few. What is needed is some rich understanding of how people work and how they use information, how decisions are made (or not) and what we can do to provide something useful, not “fish stories” about the size of our databases.

    -Neil Raden
    Hired Brains

  3. Радиодетали в г.Ростове-на-Дону http://andromaril.ucoz.ru +79185540950 Владимир

Trackbacks

  1. […] Friends Don’t Let Friends Overpay for BI | Roman Stanek Business Intelligence projects are famous for low success rates, high costs and time overruns. The economics of BI are visibly broken, and have been for years. Yet BI remains the #1 technology priority according to Gartner. We could paraphrase Lee Iacocca and say: People want economical Business Intelligence solutions and they will pay ANY price to get it. […]

  2. […] Note: Last week, Good Data Founder and President Roman Stanek posted a commentary entitled Friends Don’t Let Friends Overpay at his blog. I spoke with Stanek […]

  3. […] Roman’s blog – Friends Don’t Let Friends Overpay for BI […]

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s