The shift to the 'algorithm economy'

The concept of the smart city is to integrate information and technology developments to improve the quality of life of its citizens. Smart city initiatives have been around since the dawn of the digital revolution. While there are several smart city models implemented across the world, Singapore has been one of the most ambitious in adapting this program to push the city into the digital age. The country’s “smart nation” program was launched in November 2014, turning the island into a “living laboratory” testing smart solutions to address urban issues involving entities in the private and public sectors in Singapore.

Vast amount of data is generated and collected in a highly digitalised economy and over time, the term big data is becoming increasingly irrelevant, with all data now considered as big data. This exponential growth in unstructured data has skyrocketed in complexity (and the processing power needed to cope with it) to levels hard to imagine even a few years ago.

Storage and processing concerns, however, are far from the only issues and, when it comes to the big data revolution, we need to appreciate that it’s not just volume and complexity that are revolutionary. It’s our growing ability to understand and make use of this massive resource, which is proving the most disruptive and has been transforming our whole approach to IT.

It is not just about size

The real driving force behind the big data revolution lies in advances in our ability to combine advanced statistical models and new compute technologies to better understand and benefit from data resources. More specifically, the ability to generate rules, or algorithms, to look for patterns in data and solve problems in a fraction of the time and effort required using conventional computing methods.

Just as apps have revolutionised the way we, as humans, interact with computers, now algorithms are facilitating a quantum leap in machine learning, enabling us to do something really useful with our big data resources.

As a test bed for autonomous and data-driven systems, Singapore in particular has the potential to move towards the algorithm economy as it shifts towards the “fourth industrial revolution”. Using big data, the country has developed innovations such as self-driving taxis, airbus drones delivering packages and eventually autonomous street cleaners and self-driving buses as they move towards transforming into a smart nation. These advancements reflect the country’s investment in technological disruptions and its ability to advance towards an Algorithm Economy.

Algorithms are what drive Google’s search engine, power Netflix recommendations, voice driven assistants, driverless cars, next day deliveries, high speed trading and an ever-growing number of services and technologies that we already take for granted.

Crucially, it’s not just the data that’s of value in these examples, or the amount being processed, vast though it may be. It's the intelligence that algorithms are able to provide that matters, enabling machines to make sense of the data and learn how to use it. Moreover, it’s this value that companies will be looking to monetise, with Gartner predicting what it calls the “algorithm economy” as the next big thing in big data, as algorithms come to be developed, traded and exploited just like mobile apps over the coming years.

Machine learning as part of the process to digital transformation

The inevitable flipside to the algorithm economy will be even greater pressure on data storage and processing resources, with CIOs increasingly turning to public cloud services to soak up this demand. Many organisations will, however, prefer to keep their algorithms, as well as business-critical data, behind the corporate firewall and it’s here that, paradoxically, algorithms could have their biggest impact, by empowering the enterprise to dip in and out of the public cloud on their own terms.

Think of it as “physician heals thyself”. While the rise of the algorithm economy will compound data storage and processing issues, algorithms will also be essential when it comes to solving those issues by leveraging big data insights to better manage the big data resources they come from.

We begin to see the shift towards an algorithm economy in Singapore due to the dawn of cloud process technologies. We see this particularly in the banking industry, wherein a vast amount of customer data is collated via various banking transactions.3 As the cloud collates more data, the algorithm economy simplifies data processing, hence, unloading the burden of having to continuously develop and maintain their IT solutions in-house. These usually come in the form of machine learning tools able to automatically balance storage demands and compute workloads across a mix of public and private platforms. Balancing these workloads in response to subtle changes in demand profiles is out of the question; it has to be done by machines.

So, if you haven’t already, expect to become more familiar with algorithms and machine learning as essential to the ongoing digital transformation process. The algorithms are coming, and to an infrastructure near you, sooner than you might think.