Key Performance Indicator Disadvantages and Controversy
Have you ever heard for the mathematical equation that caused the banks to crash?
It was the holy grail of investors. The Black-Scholes equation, brainchild of economists Fischer Black and Myron Scholes, provided a rational way to price a financial contract when it still had time to run. It was like buying or selling a bet on a horse, halfway through the race. It opened up a new world of ever more complex investments, blossoming into a gigantic global industry. But when the sub-prime mortgage market turned sour, the darling of the financial markets became the Black Hole equation, sucking money out of the universe in an unending stream.
This is the major point made in 1973 together with oil production peek (2006) which started current crise starting at 2008. When things go seriously wrong, there’s nothing more natural than to seek the causes, reasons, and possibly also the culprits, because finally – to learn from mistakes.
Black-Scholes equation has launched a massive economic growth. To 2007. The international financial system, derivatives traded annually worth about quadrillion dollars, as much as 10 times more than the value of all goods that were produced in the last century the whole world. The downside was that the company had to hire talented mathematical analysts, or quanta, to develop a similar formula by which the assessed value and risks of new instruments. But they forgot to ask how many answers will be reliable if market conditions change.
The difference between model and reality
But the problem was the fact that mathematical models have always operated with the simplifications and assumptions which may not always coincide with reality. When the market is stable, it is valid, however, when it comes to major disturbance models and reality begin to seriously diverge. For example the Black-Sholes equation is based on the evaluation model arbitration of securities in which the constant volatility of the market. This assumption is common in financial theory, however, often does not apply to real market. The equation also assumes that there are no transaction costs, there are no restrictions on sales and that money can always borrow the familiar and secure a fixed interest rate. In this sense, the reality is considerably different.
By studying ecological systems, it can be shown that instability is common in economic models, mainly because of the poor design of the financial system. The facility to transfer billions at the click of a mouse may allow ever-quicker profits, but it also makes shocks propagate faster.
Was an equation to blame for the financial crash, then? Yes and no. Black-Scholes may have contributed to the crash, but only because it was abused. In any case, the equation was just one ingredient in a rich stew of financial irresponsibility, political ineptitude, perverse incentives and lax regulation.
Despite its supposed expertise, the financial sector performs no better than random guesswork. The stock market has spent 20 years going nowhere. The system is too complex to be run on error-strewn hunches and gut feelings, but current mathematical models don’t represent reality adequately. The entire system is poorly understood and dangerously unstable. The world economy desperately needs a radical overhaul and that requires more mathematics, not less. It may be rocket science, but magic it’s not.
Do you automatically assume data and information stability from your Information Systems?
Do you automatically assume stability of information interpretation?
Do you automatically assume mathematical formulas are implemented in information relations?
These are 3 major Key Performance Indicators assumptions that are misleading the whole KPI concept.
Key Performance Indicators as Business Derivates
KPIs are controversial, they do assume that information and data are integrated and are of high quality. With more and more generation of unstructured data in the Information Systems, quality of KPIs is also highly volatile and questionable.
I’ve tried to make survey of key performance indicators disadvantages, but resources were quite poor. Can’t be that with implementation of whatever chosen KPIs metrics, company gets advanced strategic weapon to guide company… KPI’s are installed and everything starts to look pink.
Is this really so simple, design and model winning KPI sets and they will allow effective business performance tracking? If necessary let them be highly advanced…
Why should KPIs have any kind of problem? If there is a change in production system and if data quality is effective there should not be any problem because KPI value is going to be automatically updated.
Two biggest KPI disadvantages in complex Information Systems
#1 Poor relation stability between different KPI groups
Let’s assume that single KPI update is working properly and that update of its value is stable, question is, what happens with different KPI groups – and that are related.
What shall happen with financial indicators if CRM system is updated with new products and that have new structures? Corresponding change in financial indicators might full of errors, and usually is. Impact of new products in CRM and into structures of financial systems might lead too poor quality information feed for financial indicators. Gross Ad (GA) of new product line for example may cause error in reporting of old products, GA of old products.
There might be many reasons for such mistakes like IT developers did additional modifications on old structures, modified business logic deep in code, there are new exceptions that are counted differently, dummies are being used, changes on tables in production systems that are used for data warehouse were changed and communication to DWH team was forgotten, master data management error, ETL error… and so on and so on.
Tracking costs and product profitability is getting major financial battlefield for controllers. As a controller I’ve seen so many disproportional financial and business indicators. Business indicators and their changes often do not correspond to financial figures and their changes. Differences are hard to explain and analysis on them are quite demanding. Average calculation cycle is usually one year, just to maintain aspects of business and financial indicators together for products.
#2 Dynamic and Complex Information Systems
There are many non standard processes and they are happening too often. These reasons worsen quality of fragile information and performance indicators relations, their mutual transparency. In complex systems there are too many variables that can not be controlled and that can guarantee quality of feed toward Key performance Indicators. Instability is furthermore used in KPI’s matrix and models… how can these models guarantee information quality to decision makers?
It is not possible to build house without solid basis. Same is with Key Performance Indicators, their main problems lay in information basis feed, not them. These problems are not easy to solve, actually, with information request tsunami in almost every industry, situation with quality and stability of Information Systems is only worsening.
If basic problems are not solved, how can then KPI models be operative and basis for decisions?
This is the reason why KPIs are so similar to financial products made upon The Black-Scholes equation. They both use information unstable environment as basis for new products and decisions.