Artificial Intelligence – Parts Per Billion

Logarithmic Harmony in Natural Chaos

Mathematics is one powerful tool to make sense out of randomness but bear in mind that not every randomness could be handled effectively with the mathematical tools we have at our disposal today. One of such tools called Benford’s Law proves that nature works in logarithmic growth and not in linear growth. The Benford’s law helps us to make sense of the natural randomness generated around us all the time. This is also one of the first-hand tools used by forensic accountants to detect possible financial frauds. It is one phenomenal part of mathematics which finds patterns in sheer chaos of the randomness of our existence.

Benford’s Law for natural datasets and financial fraud detection

People can find patterns in all kinds of random events. It is called apophenia. It is the tendency we humans have to find meaning in disconnected information.
Dan Chaon, American Novelist

Is There Any Meaning in Randomness?

We all understand that life without numbers is meaningless. Every single moment gazillions and gazillions of numbers are getting generated. Even when I am typing this and when you are reading this – some mathematical processing is happening in bits of the computer to make it happen. If we try to grasp/understand the quantity of numbers that are getting generated continuously, even the lifetime equivalent to the age of our Universe (13.7 billion) will fall short.

Mathematics can be attributed to an art of finding patterns based on certain set of reasoning. You have certain observations which are always true and you use these truths to establish the bigger truths. Psychologically we humans are tuned to pattern recognition, patterns bring in that predictability, predictability brings in safety because one has knowledge of future to certain extent which guarantees the higher chances of survival. So, larger understanding of mathematics in a way ensures better chances of survival per say. This is oversimplification, but you get the point.

Right from understanding the patterns in the cycles of days and nights, summers, and winters till the patterns in movements of the celestial bodies, the vibration of atoms, we have had many breakthroughs in the “pattern recognition”. If one is successful enough to develop a structured and objective reasoning behind such patterns, then predicting the fate of any process happening (and would be happening) which follows that pattern is a piece of cake. Thus, the power to see the patterns in the randomness is kind of a superpower that we humans possess. It’s like a crude version of mini-time machine.

Randomness inherently means that it is difficult to make any sense of the given condition, we cannot predict it effectively. Mathematics is one powerful tool to make sense out of randomness but bear in mind that not every randomness could be handled effectively with the mathematical tools we have at our disposal today. Mathematics is still evolving and will continue to evolve and there is not end to this evolution – we will never know everything that is there to know. (it’s not a feeling rather it is proved by Gödel’s incompleteness theorem.)

You must also appreciate that to see the patterns in any given randomness, one needs to create a totally different perspective. Once this perspective is developed then it no longer remains random. So, every randomness is random until we don’t have a different perspective about it.

So, is there any way to have a perspective on the gazillions of the numbers getting generated around us during transactions, interactions, transformations?

The answer is Yes! Definitely, there is a pattern in this randomness!!

Today we will be seeing that pattern in detail.

Natural Series – Real Life Data

Take your account statement for an example. You will see all your transactions, debit amount, credit amount, current balance in the account. There is no way to make sense out of how the numbers that are generated, the only logic behind those numbers in account statement is that you paid someone certain amount and someone paid you certain amount. It is just net balance of those transactions. You had certain urgency someday that is why you spent certain amount on that day, you once had craving for that cake hence you bought that cake, you were rooting for that concert ticket hence you paid for that ticket, on one bad day you faced certain emergency and had to pay the bills to sort things out. Similarly, you did your job/ work hence you got compensated for those tasks – someone paid you for that, you saved some funds in deposits and hence that interest was paid to you, you sold some stocks hence that value was paid to you.

The reason to explain this example to such details is to clarify that even though you have control over your funds, you actually cannot control every penny in your account to that exact number that you desire. This is an example of natural data series. Even though you have full control over your transactions, how you account will turn out is driven by certain fundamental rules of debit/ credit and interest. The interactions of these accounting phenomenon are so intertwined that ultimately it becomes difficult to predict down to every last penny.

Rainfall all around the Earth is very difficult to predict to its highest precision due to many intermingling and unpredictable events in nature. So, by default finding trend in the average rainfall happened in given set of places is difficult. But we deep down know that if we know certain things about rainfall in given regions we can make better predictions about other regions in a better way, because there are certain fundamental predictable laws which govern the rainfall.

The GDP of the nations (if reported transparently) is also very difficult to pin down to exact number, we always have an estimate, because there are many factors which affect that final number, same goes for the population, we can only predict how it would grow but it is difficult to pin point the number.

These are all examples of real life data points which are generated randomly during natural activities, natural transactions. We know the reason for these numbers but as the factors involved are so many it is very difficult to find the pattern in this randomness.

I Lied – There is A Pattern in The Natural Randomness!

What if I told you that there is certain trend and reference to the randomness of the numbers generated “naturally”? Be cautious – I am not saying that I can predict the market trend of certain stocks; I am saying that the numbers generated in any natural processes have preference – the pattern is not predictive rather it only reveals when you have certain bunch of data already at hand – it is retrospective.

Even though it is retrospective, it can help us to identify what was manipulated, whether someone tried to tamper with the natural flow of the process, whether there was a mechanical/ instrument bias in data generation, whether there was any human bias in the data generation?

Logarithm and Newcomb

Simon Newcomb (1835-1909) a Canadian-American astronomer once realized that his colleagues are using the initial pages of log table more than the other pages. The starting pages of log tables were more soiled, used than the later pages.

Log tables were instrumental in number crunching before the invention of any type of calculators. The log tables start with 10 and end in 99.

Newcomb felt that the people using log tables for their calculations have more 1’s in their datasets repetitively in early digits that is why the initial pages where the numbers start with 1 are used more. He also knew that the numbers used in such astronomical calculations are the numbers available naturally. These numbers are not generated out randomly, they signify certain quantities attributed to the things available in nature (like diameter of a planet, distance between stars, intensity of light, radius of curvature of certain planet’s orbit). These were not some “cooked up” numbers, even though they were random but they had natural reason to exist in a way.

He published an article about this but it went unnoticed as there was no way to justify this in a mathematical way. His publication lacked that mathematical rigor to justify his intuition.

Newcomb wrote:

“That the ten digits do not occur with equal frequency must be evident to anyone making much use of logarithmic tables, and noticing how much faster the first one wears out than the last ones.”

On superficial inquiry, anyone would feel that this observation is biased. It seemed counterintuitive, also Newcomb just reported the observation and did not explain in detail why it would happen. So, this observation went underground with the flow of time.

Frank Benford and The Law of Anomalous Numbers

Question – for a big enough dataset, how frequently any number would appear in first place? What is the probability of numbers from 1 to 9 to be the leading digit in given dataset?

Intuitively, one would think that any number can happen to be in the leading place for given dataset. If the dataset becomes large enough, all nine numbers will have equal chance to be in first place.

Frank Benford during his tenure in General Electric as a physicist made same observation about the log table as did Newcomb before him. But this time Frank traced back the experiments and hence the datasets from these experiments for which the log table was used and also some other data sets from magazines. He compiled some 20,000 data points from completely unrelated experiments and found one unique pattern!

He realized that even though our intuition says that any number from 1 to 9 could appear as the leading digit with equal chance, “natural data” does not accept that equal chance. The term “Natural data” refers to the data representing any quantifiable attribution of real phenomenon, object around us, it is not a random number created purposefully or mechanically; it has some origin in nature however random it may seem.

Frank Benford thus discovered an anomaly in natural datasets that their leading digit is more 1 or two than the remaining ones (3,4,5,6,7,8,9). In simple words, you will see 1 as leading digit more often in the natural datasets than the rest of the numbers. As we go on with other numbers the chances that other numbers will be frequent in leading position are very less.

In simple words, any naturally occurring entity will have more frequent 1’s in its leading digits that the rest numbers.

Here is the sample of the datasets Frank Benford used to find this pattern:

Dataset used by Frank Benford in his 1938 paper “The Law of Anomalous Numbers”

So, according to Benford’s observations for any given “natural dataset” the chance of 1 being the leading digit (the first digit of the number) is almost 30%. 30% of the digits in given natural dataset will start with 1 and as we go on the chances of other numbers to appear frequent drop drastically. Meaning that very few number in given natural data set will start with 7,8,9.

Thus, the statement of Benford’s law is given as:

The frequency of the first digit in a populations’ numbers decreases with the increasing value of the number in the first digit.

Simply explained, as we go on from 1 to 9 as first digit in given dataset, the possibility of their reappearance goes on reducing.

1 will be the most repeated as the first number then 2 will be frequent but not more than 1 and the frequency of reappearance will reduce and flatten out till 9. 9 will rarely be seen as the leading digit.

The reason why this behavior is called as Benford’s Law (and not Newcomb’s Law) is due to the mathematical equation that Benford established.

Where, P(d) is the probability that a number starts with digit d. Digit d could be anything 1,2,3,4,5,6,8 or 9.

If we see the real-life examples, you will instantly realize how counterintuitive this law is and still nature chooses to follow it.

Here are some examples:

I have also attached an excel sheet for complete datasets and to demonstrate how simply one can calculate and verify Benford’s law.

Population of countries in the world –

The dataset contains population of 234 regions in the world. And you will see that 1 appears the most as first digit in this dataset. Most of the population numbers start with 1 (70 times out of 234) and rarely with 9 (9 times out of 234)

Country-wise average precipitation –

The dataset contains average rainfall from 146 countries in the world. Again, same pattern emerges.

Country wise Gross Domestic Product –

The dataset contains 177 countries’ GDP in USD. See the probability yourself:

Country-wise CO₂emissions:

The data contains 177 entries

Country wise Covid cases:

Here is one more interesting example:

The quarterly revenue of Microsoft since its listing also shows pattern of Benford’s Law!

To generalize we can find the trend of all these data points by averaging as follows:

This is exactly how Benford avearaged his data points to establish a generalized equation.

Theoretical Benford fit is calculated using the Benford equation expressed earlier.

So here is the relationship graphically:

Now, you will appreciate the beauty of Benford’s law and despite seeming counterintuitive, it proves how seemingly random natural dataset has preferences.

Benford’s Law in Fraud Detection

In his 1938 paper “The Law of Anomalous Numbers” Frank Benford beautifully showed the pattern that natural datasets prefer but he did not identify any uses of this phenomena.

1970 – Hal Varian, a Professor in University of California Berkely School of Information explained that this law could be used to detect possible fraud in any presented socioeconomic information.

1988 – Ted Hill, an American mathematician found out that people cannot cook up some numbers and still stick to the Benford’s Law.

When people try to cook up some numbers in big data sets, they reflect certain biases to certain numbers, however random number they may put in the entries there is a reflection of their preference to certain numbers. Forensic accountants are well aware of this fact.

The scene where Christian pinpoints the finance fraud [Warner Bros. – The Accountant (2016)]

1992 – Mark Nigrini, a South African chartered accountant published how Benford’s law could be used for fraud detection in his thesis.

Benford’s Law is allowed as a proof to demonstrate accounts fraud in US courts at all levels and is also used internationally to prove finance frauds.

It is very important to point the human factor, psychological factor of a person who is committing such numbers fraud. People do not naturally assume that some digits occur more frequently while cooking up numbers. Even when we would start generating random numbers in our mind, our subconscious preference to certain numbers gives a pattern. Larger the data size more it will lean to Benford’s behavior and easier will be the fraud detection.

Now, I pose one question here!

If the fraudster understands that there is such thing like Benford’s Law, then wouldn’t he cook up numbers which seem to follow the Benford’s Law? (Don’t doubt my intentions, I am just like a cop thinking like thieves to anticipate their next move!!!)

So, the answer to this doubt is hopeful!

The data generated in account statements is so huge and has multiple magnitudes that it is very difficult for a human mind to cook up numbers artificially and evade from detection.

Also, forensic accountants have showed that Benford’s Law is a partially negative rule; this means that if the law is not followed then it is possible that the dataset was tampered/ manipulated but conversely if the data set fits exactly / snuggly with the Benford’s law then also there is a chance that the data was tampered. Someone made sure that the cooked-up data would fit the Benford’s Law to avoid doubts!

Limitations of Benford’s Law

You must appreciate that nature has its ways to prefer certain digits in its creations. Random numbers generated by computer do not follow Benford’s Law thereby showing their artificiality.

Wherever there is natural dataset, the Benford’ Law will hold true.

1961 – Roger Pinkham established one important observation for any natural dataset thereby Benford’s Law. Pinkham said that for any law to demonstrate the behavior of natural dataset, it must be independent of scale. Meaning that any law showing nature’s pattern must be scale invariant.

In really simple words, if I change the units of given natural dataset, the Benford law will still hold true. If given account transactions in US Dollars for which Benford’s Law is holding true, the same money expressed in Indian Rupees will still abide to the Benford’s Law. Converting Dollars to Rupees is scaling the dataset. That is exactly why Benford’s Law is really robust!

After understanding all these features of Benford’s Law, one must think it like a weapon which holds enormous power! So, let us have some clarity on where it fails.

Benford’s Law is reflected in large datasets. Few entries in a data series will rarely show Benford’s Law. Not just large dataset but the bigger order of magnitude must also be there to be able to apply Benford’s Law effectively.
The data must describe same object. Meaning that the dataset should be of one feature like debit only dataset, credit only dataset, number of unemployed people per 1000 people in population. Mixture of datapoints will not reflect fit to Benford’s Law.
There should not be inherently defined upper and lower bound to the dataset. For example, 1 million datapoints of height of people will not follow Benford’s Law, because human heights do not vary drastically, very few people are exceptionally tall or short. This, also means that any dataset which follows Normal Distribution (Bell Curve behavior) will not follow Benford’s Law.
The numbers should not be defined with certain conscious rules like mobile numbers which compulsorily start with 7,8, or 9; like number plates restricted 4, 8,12 digits only.
Benford’s Law will never pinpoint where exactly fraud has happened. There will always be need for in depth investigation to locate the event and location of the fraud. Benford’s Law only ensures that the big picture is holding true.

Hence, the examples I presented earlier to show the beauty of Benford’s Law are purposefully selected to not have these limitations. These datasets have not bounds, the order of magnitude of data is big, range is really wide compared to the number of observations.

Now, if I try to implement the Benford’s Law to the yearly revenue of Microsoft it reflects something like this:

Don’t freak out as the data does not fully stick to the Benford’s Law, rather notice that for the same time window if my number of datapoints are reduced, the dataset tends to deviate from Benford’ Law theoretically. Please also note that 1 is still appearing as the leading digit very frequently, so good news for MICROSOFT stock holders!!!

In same way, if you see the data points for global average temperatures (in Kelvin) country-wise it will not fit the Benford’s Law; because there is no drastic variation in average temperatures in any given region.

See there are 205 datapoints – big enough, but the temperatures are bound to a narrow range. Order of magnitude is small. Notice that it doesn’t matter if I express temperature in degree Celsius of in Kelvins as Benford’s Law is independent of scale.

Nature Builds Through Compounded Growth, Not Through Linear Growth!

Once you get the hold of Benford’s law, you will appreciate how nature decides its ways of working and creating. The Logarithmic law given by Frank Benford is a special case of compounded growth (formula of compound interest). Even though we are taught growth of numbers in a periodic and linear ways we are masked from the logarithmic nature of the reality. Frank Benford in the conclusion of his 1937 paper mentions that our perception of light, sound is always in logarithmic scale. (any sound engineer or any lighting engineer know this by default) The growth of human population, growth of bacteria, spread of Covid follow this exponential growth. The Fibonacci sequence is an exponential growth series which is observed to be at the heart of nature’s creation. That is why any artificial data set won’t fully stick to logarithmic growth behavior. (You can use this against machine warfare in future!) This also strengthens the belief that nature thinks in mathematics. Despite seemingly random chaos, it holds certain predictive pattern in its heart. Benford’s Law thus is an epitome of nature’s artistic ability to hold harmony in chaos!

You can download this excel file to understand how Benford’s law can be validated in simple excel sheet:

benford-partsperbillion Download

References and further reading:

Cover image – Wassily Kandinsky’s Yellow Point 1924
The Law of Anomalous Numbers, Frank Benford, (1938), Proceedings of the American Philosophical Society
On the Distribution of First Significant Digits, RS Pinkham (1961), The Annals of Mathematical Statistics
What Is Benford’s Law? Why This Unexpected Pattern of Numbers Is Everywhere, Jack Murtagh, Scientific American
Using Excel and Benford’s Law to detect fraud, J. Carlton Collins, CPA, Journal of Accountancy
Benford’s Law, Adrian Jamain, DJ Hand, Maryse Bйeguin, (2001), Imperial College London
data source – Microsoft revenue – stockanalysis.com
data source – Population – worldometers.info
data source – Covid cases – tradingeconomics.com
data source – GDP- worldometers.info
data source – CO2 emissions – worldometers.info
data source – unemployment – tradingeconomics.com
data source – temperature – tradingeconomics.com
data source – precipitation – tradingeconomics.com

Riding on the ‘Hype Wave’ of Technological Breakthroughs

Countless breakthroughs are happening around the globe everyday but very few hold the potential to change the world and the future. Due to our cognitive limitations and biases, we wrongly estimate the impact of the emerging innovations in near future and long-term future. Amara’s Law and ideas of Hype Cycle can provide insights into how a technology evolves over time from its emergence and how to spot the technology which truly holds the potential to change the future course of the humanity. Understanding the phases in the development of an innovation and its coherence to reality can help entrepreneurs, researchers, investors, policymakers and even a common man to have practical expectations from any technological breakthrough.

How to gauge the trend and acceptance of any emerging technology?

The Wheel and The Fire

One of the key differentiating factors that created a totally separate path for the evolution of humans from the apes and other species is the invention of tools. Right from the invention of the wheel and the fire to the invention of steam engines to the invention of the computers, smartphones to the invention of artificial intelligence – our tools to interact with the world around us keep on getting more and more sophisticated thereby uplifting our lifestyle. This is way different than how other species live, interact with the world around them and exist on the earth – our home, their home.

Any sufficiently advanced technology is equivalent to magic
Arthur C Clarke

Imagine if you traveled back in time and showed ancient Egyptians a smartphone? You may explain it to them as a tablet sitting in hands made up of the components built from sand which could show you the extremely detailed real-time pictures from the location far from their locations or you can send your voice and receive the voice from the other side. Given that Egyptians created some of the astounding engineering marvels that world has ever seen, even after that a smartphone will be equivalent of a pure magic for them.

Now coming back to the future- present times, if you are told that a company has built a device which can teleport you instantly to other planet, what would be your reaction? Though seemingly magical, it is just a fiction for us due to practical reasons. We haven’t even teleported an insect from point A to point B till date.

The Think Tanks For The Future

So, there is an limit to realize the practicality of every technology. Many technological breakthroughs are happening all around the world every day but very few of them actually change the course of the humanity. That is why it is very important to identify which technology really holds the true potential. The earlier one realizes potential of the technology, the faster and bigger will they have the grasp over the world, politics, economy and society thereby maybe the whole humanity.

Many think tanks around the worlds are always invested in the prediction of the future scenarios in global politics, war fares, technological breakthroughs, epidemics. These people are called as ‘the futurists’ who are striving to predict the long-term future obviously for the sake of readiness, survival and sometimes dominance. These futurists have certain tools in their kit which can help us to understand and point out which next technological revolution will change the course of the humanity in near future. Although we are not experts of all technologies and breakthroughs happening in the world daily, these tools can help us to understand which technology can actually benefit and uplift our daily lives. These tools can help an innovation manager to point out the technology which can create differentiation to his product in the market, these tools can help an entrepreneur to select the technology to boost his/her business or startup. This can also help a common man to gauge whether a technology claimed by a company can actually bring a difference in his life.

There are these questions we keep on asking ourselves when we are dealing with new technologies coming to our doorsteps, into our hands –

Why Apple brings the technologies in their phones really late when every other device manufacturer has already made it mainstream and sometimes obsolete?

Why AI won’t actually take over the world in near future? Why EV’s may face a cold death? Why flying cars are still not practical and common product as predicted in back to the future? Why teleportation is not a reality and only a magical part of the today’s Science fiction?

What is the future of the innovations and breakthroughs we are making every day? Is there any way to predict the future value of certain emerging technology and built a product around it to create a fruitful venture/ business? How to be confident while investing in any technology based on its current condition or stage?

Amara’s Law from the futurist Roy Amara gives a deep insight into the phases every technology goes through and it also helps us to make any decision for given technology.

Influence Of Technological Breakthroughs On Humanity And Its Future

Roy Charles Amara was the president of Institute for the future (ITFT) an American non-profit think tank which works for the better prediction of long-term future. The prospects of ITFT include the exploration for the possible futures for the USA and the world, to figure out the preferences/desirability of these futures and to increase the chances to bring that future possibility into reality by finding out ways that support it.

Roy Amara gave a very important insight into how we as the human beings perceive future of any technological breakthrough and most of the times the masses are wrong about these technological breakthroughs.

Amara’s Law

“We overestimate the power and effects of the technology for a short period of time, while we underestimate the power of technology for a long period of time.”
Roy Charles Amara

In simple words, our expectations (positive or negative) from recent technological breakthroughs are always high and we are very skeptical about already existing technologies to be revolutionary and mainstream in the future.

Take for example the LHC experiments which were started in CERN to better understand the subatomic particles. Some scientists predicted that there may be chances that some small black holes may form in this experiment which would engulf the whole earth into it. Same fear was lingering around the Trinity test, where many scientists thought that the Atomic Bomb test would initiate a chain reaction which could ionize the whole atmosphere of the earth thereby leading to the end of the humanity.

But, look what happened? Nothing dangerous happened during the LHC runs, the runs confirmed the presence of Higgs Boson. Although atom bomb proved to be really fatal and formidable invention of humanity, the Trinity test at Los Alamos didn’t ionize the earth’s atmosphere.

In our times, people are speculating that the AI will take over humanity and will rule over the world by enslaving everyone. Now look what ChatGPT responds when you ask some fundamental philosophical questions? (Although it is excellent in certain tasks but there is still long way to go in order take over the humanity!) There are some examples where the AI image generator could not create proper images of human hands as the orientation of fingers relative to each other is “confusing” for the AI image engine. It is also known that certain biases can be created in the AI engine based on the sample training data provided, so there is still a long way for the AI to catch up with the humanity and there is no doubt that AI will totally revolutionize our lives but, in the ways, we are yet to imagine or grasp. (although AI has already revolutionized some parts of our lives already)

Amara’s Law points out a cognitive limitation in us where it is really difficult for us to predict the non-linear behavior of technology in the coming future. We human beings are very great in predicting the linear incremental behavior, somewhat ideal and constant incremental behavior of things around us. The moment we infuse multiple variables and some non-ideality in these predictions, we make wrong decisions based on our survival instincts. We “overhype” the technology’s potentials. For given technology people may think that it will revolutionize the ways of doing things and uplift the society, some think that certain technology will take people’s jobs and push society into dystopia. Look what is happening with cryptocurrency and NFT – the technologies which were supposed to revolutionize the complete world economy. Although, blockchain is there to remain forever as an excellent invention it will change the world in totally different way than people actually predicted.

The AI which is supposed to take over the humanity

Amara’s Law is famously explained by the S-shaped curve to represent the difference between anticipated impact and actual impact of the technological breakthroughs.

Gartner Hype Cycle

Gartner Hype Cycle also throws light on some interesting concepts on the actual non-linear impact of technologies over the time. This idea was developed by an American consultancy firm Gartner Inc. named after Gideon Gartner who is called as ‘the father of the modern analyst industry’. Gartner Hype Cycle establishes certain phases in the implementation, growth and acceptance of any technology.

These phases are given as follows:

Innovation/ Technology trigger – A new technology is presented to the world which creates intrigue, sometimes fear in the minds of the masses. The competitors panic sometimes for the probable upcoming uncertainties in business
Peak of inflated expectations – As the technology is something new, there are very few experts to truly understand it. The hype build around it due to the insufficient knowledge of the media and communicators. This builds unrealistic expectations among the masses.
Trough of disillusionment – There comes a time when this hyped technology starts getting implemented into real life where practical limitations keep piling up. Not only practical but also some economical, social problems start peeking this pile of problems. The expectations were already high and when such failures start becoming apparent to the mass users, the technology enters the rock bottom, the cold death
Slope of enlightenment – After remaining in the abyss of failures there comes a time when the exact technology finds a better purpose for implementations; it’s newer generation become more people relevant more practical which people accept properly, where the society is evolved enough to accept it as their way of life. From here on this technology enters ‘the plateau of productivity’. The true value and proper points to implement the technology are identified and widely accepted in this plateau.

Many experts in the industry critic Gartner hype cycle as it does not provide any instructions or actions to control these behaviors for any emerging or disruptive technological revolution. One can safely say that Gartner Hype Cycle gives a generalized view on acceptance of new technology.

There is also an Extended Gartner Hype Cycle where after the plateau of productivity, the technology loses its value to the reduced returns to the business over time which further ends into the obsolescence – “the cliff of obsolescence”.

Key Takeaways From The Hype Cycle

When a policy maker, an entrepreneur, a manager understands the Gartner Hype Cycle, it will definitely help them to make informed decisions which can reduce risks and maybe save many lives.

Being patient and not getting tempted to ride on the hype wave is the first important response.

Updating the knowledge and current trends regularly will help in creating fruitful strategies against the hype.

Change is the only thing that remains forever and embrace this. Adaptations to stay ahead in the practical technology is the optimum move for any leader/ policymaker.

Understanding the long-term viewpoint from a sustainable perspective with the closeness to reality/ practicality immediately breaks the illusion of hype.

It is really important for a policymaker, a leader and even a common man to understand that creating a breakthrough does not guarantee the success in the practicality of the innovation/ technology. The innovation even though called as breakthrough has to be practical, relevant and realistic.

The Gartner Hype Cycle can explain why some tech companies wait for the technology to evolve and establish in order to deliver complete consumer experience. The hype cycle also explains why many startups who have found breakthroughs initially, fail to deliver at the end as the hype wave builds unnecessary expectations among the investors. ‘Edison’ by ‘Theranos Inc.’ founded by Elizabeth Holmes is one such example. The company was expected to create revolution in the medical diagnostic industry and was touted to be ‘the iPhone’ of the medical and healthcare industry. Look what has happened after that!

Internet and GPS (Global Positioning System) are the examples which were supposed to remain military secrets for years eventually have become the walk and talk of everyday lives and influence every part of our life now. (There was a time called dot com bubble which reiterates the hype of internet companies!)

Once you understand the Amara’s Law and Gartner Hype Cycle you can clearly see how any new technology launched in the market will behave in near future. It is not just about creating disruptive innovation to the market; it is also about solving realistic problems and understanding the key pain points of the masses.

Roy Amara’s Futurist Legacy for Predicting Breakthroughs

Any innovation which will truly impact the future should be studied for three main parameters/ premises.

The possible – as the breakthroughs are practically ‘the trend breakers’, the study for their possibility should involve unconventional approaches that defy formalization/ structured-ness. There should also be some human element of intuition which gives a personal touch to such innovations. For example, the Science Fiction authored by a well-versed scientist/ artist who understands its practical limitations for today but anticipates that it will get solved in near future. (The motion capture technology evolved during the creation of James Cameron’s Avatar is one such good example)
The probable – defining probability requires to understand ‘what is connected to what?’ What action will increase the chances of certain event? Thus, the process of quantifying the probability of the success of the breakthrough innovation immediately establishes the chain of reasoning to its future projection.
The preferable – Even though the innovation is possible and most probable but if it is not required by the time and the society then it may surely face the cold death. So, prefer-ability anticipates the societal, economical, and humanistic aspects to accept the innovation. If the innovation has no net positive utility, then it won’t come out as the most probable future. (For example, even though we know that the Grade 5 Titanium is one of the lightest and the strongest materials in the mechanical world, we also know that people can’t afford the cars made out of it for daily use as the costs of manufacturing will be exceptionally high which ultimately will get transferred to end users who won’t pay for it – even when it comes to saving their lives for the amount they invest.)

This makes it very clear that an innovation needs to be realistically possible, the most probable and most preferable in order to be called as a breakthrough which holds the potential to change the course of humanity.

These are the exact reasons why people over-anticipated the trip to the Mars when Elon Musk expressed his SpaceX proposition. Now that we are seeing how difficult it is to create a rocket, how many resources, how many allied innovations need to happen, how many relevant financial, behavioral mindsets need to evolve in order to send few humans (alive!) to Mars then we are getting hold of the practicality behind sending humans to settle on Mars.

This also explains why the flying cars shown in Back to the Future are not a common reality or way of life today, it still will take time or maybe it won’t happen in future due to some other breakthroughs (like teleportation!). Back to the future successfully predicted 3D projections, Video calling, digital currency, smartwatch which were possible due to practicality and relevance.

Once you understand the Amara’s Law you can grasp that creating many innovations is not important to change the course of future, a single innovation which is practical, relevant and realistic is sufficient enough to change the course of the humanity. One can also do so by making the innovations implementable in real life,not every breakthrough guarantees immediate revolution.

“Our nations rely on innovation to improve productivity and fuel economic growth. But to be competitive, nations and organizations do not necessarily have to excel at originating innovation—they have to be able to apply innovation successfully.”
Mastering the Hype Cycle by Jakie Fenn and Mark Raskino