By Cat Yong
Because of the volume and velocity requirements of Big Data, the only way to process it effectively, is with in-memory technology, opined Robin Gilthorpe, CEO of Big Data and in-memory management vendor, Terracotta.
This is a critical technology to look at, especially when you consider that a majority; maybe as much as 60%; of organisations admit that they can’t use all of the data they have – how are they supposed to separate the wheat from the chaff?
 |
| Robin Gilthorpe |
Gilthorpe added, “SAP has done a huge favour in terms of popularising this stuff (in-memory), but they have a specific goal. Big Data is not only analytics, but also transactional and operational data.”
Central to Terracotta’s Big Memory in-memory solution, is DRAM technology or dynamic random access memory, which Gilthorpe began to put into perspective. “Enterprise-class spinning disks could have access times of 3 miliseconds. DRAM in the enterprise is 50 nanoseconds. Then you realise, that’s like the difference between 1 second and 17 hours!
“Freeing up that amount of clock cycle, gives you an enormous amount of room to do things better. For example, evaluate risk more carefully, consider more options for what you would put in front of the online consumer with their very next click, with that very next interaction.”
Where should data reside?
Despite this, data will still have to find its natural ‘home’ – not all of it will reside in in-memory, as it is simply not practical. Gilthorpe said, “For many people, you will see a sort of NAND world subsume a lot of the data that is currently on spinning disks and act as a kind of halfway house. But for really critical data, you will find it on DRAM.
“And the fact that now we can offer reliability, accessibility, scalability, the failover capabilities and so on from DRAM, makes that (DRAM) progressively more appealing.”
Economic factors also come into play, and the ability to have multiple places to put data, means organisations can manage cost and performance portfolios more tightly than ever before.
“I don’t think data warehousing will go away entirely,” Gilthorpe cautioned although in the same breath he also emphasised the criticality of the real-time element.
“There is a lot of data that is required for the smooth operation of business or the effective management of the customer experience, that doesn’t need to go all the way back to the data warehouse, and is in fact harmful when it happens.
“Particularly when you consider that a lot of the data you capture around customer experience is unstructured; it doesn’t naturally fit into a SQL structure and has value intimately linked to the time factor. So making that long journey back (to the data warehouse) is incredibly damaging,” he explained.
Immediate benefits of in-memory
In 2011 Terracotta was acquired by BPM solutions experts, Software AG, and Gilthorpe claimed that their parent company’s product line-up has solutions that ensure data resides in the most appropriate place from a response time, interoperability and cost of storage perspective.
The CEO also discovered 2 extremely popular approaches to using Terracotta technologies, and one of them included simply being a performance enhancer.
“Maybe you have built up a large application server infrastructure, and like a lot of Java environments, maybe you started off designing for a minnow and ended up as a whale with that architecture.”
The end result is very expensive operations and poor response times, multiple failutres and a boat load of other challenges. Gilthorpe claims that Terracotta is able to mitigate these with a 1000-fold improvement in performance.
Another use case scenario is in large hosted environments which tend to be good at expediting transactions, but is expensive when it comes to queries. Gilthorpe illustrated, “We have a bunch of customers who said, ‘Why use expensive cycles; especially when you got e-commerce on the front-end; to satisfy things that are just information requests? Why not front-end that (info requests) in caching, save the mainframe or whatever cycles that are in the back-end AND also improve customer experience?’
“You free up money, you make time and space to create new offerings… we see this increasingly in areas such as risk, such as gaming, such as customer interaction, and frankly, across a broad range of industries!”
