Analytics and machine learning at the frontiers of finance

In the banking and finance sector where trust and confidence are central to the business, information security, reliability, and resilience are critical.

Given its central position in the world of global finance, HSBC understands this very well. With its 150-year history, over $2.4 trillion in assets, 37 million customers, and 4,000-strong presence across 70 countries, the banking leader is an important financial institution in a heavily-regulated industry. “We have to make sure our customers feel confident and trust in us to be the custodian of their assets,” stated Darryl West, Group Chief Information Officer at HSBC, at the recently held Google Cloud Next conference.

West, whose team is currently at the center of transformation happening within the organization, said that HSBC seeks to achieve this by leveraging Google Cloud’s data analytics and machine learning tools to derive insights out of its growing volumes of data.

“Apart from our $2.4 trillion dollars of assets on our balance sheet, we have at the core of the company a massive asset in [the form of] our data. And what’s been happening in the last three years is a massive growth in the size of our data assets,” shared West, pointing out that data at HSBC has grown tremendously from 56 petabytes in 2014 to over 100 petabytes as of early 2017. “Our customers are adopting digital channels more aggressively and we’re collecting more data about how our customers interact with us. As a bank, we need to work with partners to enable us to understand what’s happening and draw out insights in order for us to run a better business and create some amazing customer experiences,” said West.

The road to cloud-first

As with most large enterprises, HSBC has historically worked with core systems running on infrastructure that has been around for decades. The systems, while robust and scalable, do not have the database structures to allow the bank to run data analytics and machine learning. “The history of enterprises like us is that we typically extract data into legacy platforms, which worked well for many years but in recent times have become expensive to run and difficult to use,” relates West.

Three years ago, HSBC took the plunge “to really embed itself into the evolving Hadoop ecosystem.” West shared that this transition from legacy platforms has been “a tough road for us. All of this has been done on-premise and we’ve had to provision and build out large infrastructure and data centers, as well as hire talented people that understand these new technologies.”

In the last year, however, HSBC eventually came to the conclusion that it would be best for the business to adopt a cloud-first strategy. It started working with Google Cloud six months ago—“We’re still at the very early stages of the relationship,” said West.

Cloud use cases

HSBC’s foray into Google Cloud began with initial use cases that involved addressing business problems that involved massive data sets and required “very intense computing capabilities in short bursts.”

One such initiative involved Anti-Money Laundering where HSBC started running analytics to look for patterns in the data to identify nefarious activities within its customer base. “These patterns we identify are then escalated to government and their crime agencies, who then track down the bad guys. This is an application that not only requires massive data sets and great computing capability, but also machine learning capability to be able to identify patterns,” said West.

Such requirements apply to Finance and Risk where billions of transactions need to be aggregated to manage HSBC’s finances and risk at a global level. “Valuation services in our trading business is a major play in global financial markets. We need to be able to run complex multi-caller simulations on a regular basis to better understand our trading positions and our risk,” reported West.

Critical enablers

For these applications to work smoothly, West said critical enablers need to be in place.

Being a heavily regulated business, some of the first enablers should cover risk and compliance, and information security. “Information security is a massive issue particularly for a bank—we need to keep our data safe, as well as provide very resilient, very safe, and performance-intensive services to our customers. We also need to be very agile as the requirements of our customers change tremendously very quickly,” he explained.

Data protection and data residency rules within the 70 countries HSBC operates need to be respected as well. “There’s a lot of rules about what data can sit where and what data can be shared so we’re working closely with Google on this construct,” said West.

Building the integration layer is vital. “All our core data sits on these legacy platforms and we need to make sure we can integrate all these technology complexities to work with the Google Cloud platform,” said West. “Data preparation is yet another enabler—you have to invest a lot of time and effort in your data architecture to understand where your crown jewels are, where the copies are, and get a good understanding of the preparation of getting the data ready to be able to interface effectively with the cloud partner,” he added. 

Perhaps the biggest enabler of HSBC’s digital transformation, according to West, is “driving a cultural shift within the business to make this really work. You have to adopt an agile methodology, a dev-ops mindset, and you need to be able to recruit and retain talented people who understand how to use these new technologies and work in this way.”

When it came down to why the bank leader picked Google as its partner towards being a cloud-first company, it also boiled down to culture. “If you’re going to be innovative and bold and take on big challenges, you need a partner that thinks the same way. What we’re doing is complex and difficult, so the chemistry and culture of the organizations is very important. We want to like who we’re hanging out with in the years ahead,” he ended.