What’s next for big data analytics? Machine learning

Cloudera, the data management and analytics leader, whose mission is to help companies ask bigger questions, has done exactly that. But, the journey is far from over especially in the region of Asia Pacific.  Cloudera’s co-founder and CTO Amr Awadallah said, “Asia is our fastest growing region in terms of revenue. There is just so much latent demand.”

He does admit that for the most part, it is a matter of companies not knowing what they don’t know. On one end of the spectrum, there are Cloudera customers like DBS Bank want to be a data-first organisation and group CTO David Gledhill shared about DBS’ 2-day offsite for the management team to brainstorm up ways to get there.

At the other end of the spectrum, there are the companies that still think the old way is the right way.

Amr said, “We are still at the beginning of this journey. So many CxOs don’t understand the importance of big data analytics (BDA).

“That’s where all of our efforts in marketing and education and training have shifted to. “

The second challenge or resistance towards mass take-up sadly has to do with skillsets; there is a shortage of people who know how to work with data management platforms like Cloudera’s or have knowledge of data science and how to leverage data to gain actionable intelligence.

“MDEC has clear goals to generate over 20,000 data professionals to work with this platform, by working with institutes of higher learning on educational programmes to develop the required skills,” Amr also shared, adding that Cloudera is doing its part by providing free training materials and subsidising cost of training for companies.

Malaysia and Asia

The Malaysian Digital Economy Corporation (MDEC) wants to do more than just generate more data professionals. Earlier this year, MDEC launched ADAX or the Asean regional data analytics exchange platform that is intended to make Malaysia the go-to Asean hub for BDA by 2020.

This exchange is meant to be a vendor-neutral environment for data-driven startups to learn, exchange, showcase and also for companies to avail themselves of the expertise, experience and knowledge that would ideally converge at the hub’s physically location in Bangsar South.

Amr lent Cloudera’s experience of BDA use cases commenting that they tend to fall into four categories – to gain insights about customers, improve products, prevent security and fraud and modern data architecture.

Having worked with a number of startups that build apps on top of Cloudera’s platform, Amr also observed that industries like telcos, finance and insurance, credit cards, payments and governments have demand for BDA and recommended for startups to build solutions in these areas.

He also observed an Indian-based startup that builds cloud services for data processing. “In essence, they simplify implementation of cloud for other companies, and act like an AWS but locally in India.

“That is a winning formula,” Amr stated, explaining that many countries are looking at local cloud options because they do not want the regulatory hassle of dealing with US-based companies that are ‘far removed’ from Asia, in more ways than one.

Amassing mass

Amr drew attention to fact that Cloudera is growing at a rate that surpasses any other enterprise software company in history. “In a few years, this region will reach critical mass. And the leader tends to hold 70-percent of the market.”

Besides all the latent demand that Cloudera aims to address over the next few years, is the next step in the BDA evolution – machine learning.

“Over 10-percent of our over 1000 customers are doing what I like to call ‘magic use cases’ – learning from all the events and predicting the next behaviour, sometimes even better than the experts,” said Amr who also predicted that in over five years, some jobs may actually be replaced by artificial intelligence.

Medical diagnosis, for example, is largely about pattern matching symptoms to health issues.