Estimated reading time: 7 minutes
Chris Jackson, Cloud Evangelist, Teradata talks about data governance, best practices, and complying to regulations like GDPR.
EITN: What is your definition of data governance? What are top industry best practices to implement data governance?
Chris Jackson: The purpose of data governance is to ensure that data is well understood, proactively monitored and managed to be in useable condition needed to support valuable action. This implies a host of things, including creating policies and stewardship procedures, maintaining a catalog, establishing and implementing good quality data and data management practices such as protecting security and privacy.
The top priority in implementing data governance is understanding the extent to which it can quickly and efficiently ensure your data readiness in meeting the needs of important business objectives.
A good data governance program can ensure that the level of confidence and trust in the data and it is fitness for purpose in decision making, business or analytic processes.
In essence, data governance needs a pragmatic middle ground between providing “polite” advice and taking on excessively large data delivery projects that are disconnected from the major business initiatives of the company.
In order to enact best industry practices, organisations must do the following:
- Identify top-priority, funded business initiatives and the data needed.
- Appoint stakeholders to take direct responsibility, in partnership with IT and a federated network of data domain owners and stewards to establish a foundation for implementing highly reusable and cross-functional data to support initiatives.
- Provide a means to reach common definitions of data and metrics and then to standardise them across system and organizational boundaries.
- Governed data should be published and accessible to downstream users and applications.
- Ensure data resources are flexible, scalable, and accessible on-premise and in the cloud.
- Manage data products that are internally and externally shared and re-used.
- Work with end users and application developers to understand common data management struggles and address them within data delivery projects.
- Establish “sandbox” environments with easy-to-use tools that support experimental data provisioning combined with easy access to authorised production data.
While some organizations have successfully institutionalized data governance practices and invested in supporting technology and the stewards who actively curate, most organizations have historically under-invested in data governance.
As the volume of data requiring governance explodes (including both internally and externally sourced datasets), organizations need to exercise a “light touch” but use comprehensive framework and supporting infrastructure tools such as encryption/tokenization, master data management, data quality, data lineage, and data catalogs. If an organization doesn’t yet have robust solutions in place, they should have a roadmap for deploying them.
The value of most governance initiatives is realized when:
- Teams move faster because data is curated, well-understood and consistent
- Confidence in results of data and analytics increases leading to better and faster decision-making
- Impact assessments can be completed rapidly for proposed changes to systems or during M&A/divestments
- The risk of breaches (and associated fines) falls because data is protected at the attribute level (not just files or disks) and access is limited by role
EITN: Does Teradata have a role in helping organisations govern data? If yes, please share how. How do you interoperate with multi-clouds and comply with regulations like GDPR?
Chris Jackson: Teradata serves customers across many industry segments, included regulated and non-regulated industries. In addition to our Master Data Management (MDM) capabilities, in which we provide organisations with a central framework that offers complete support for a workflow and process-driven data governance environment, we also help customers on their journey towards compliance with regulations like GDPR in today’s multi-cloud environments.
We do this by helping them understand they can only begin this journey if they have answers to the following questions about their sensitive data—does it exist, where does it reside, how valuable is it to the company, and who has access to it?
We realised that on the GDPR journey, many organisations do not actually have the answers to these questions. They do not know where all the personal data lives within their repositories, databases, and archives, and whether they are on premises or in the cloud.
Teradata also provides a service called DataDNA which offers an automated framework to collect and connect data assets, their usage and lineage across platforms. Whenever systems are impacted by planned upgrades or digital transformation, we can use DataDNA to perform detailed impact assessments based on data lineage and transformations from source system to every impact system and downstream consumer.
The risks are high, because many organisations are not properly investing in data governance in their digital transformation journey. A global survey by Teradata reveals that 50% of IT decision-makers worldwide (vs 52% of Singapore respondents) reported that their organisations had no plans to increase data governance investments at this stage; 89% felt their organisations should still prioritize data governance / responsibility initiatives more.
The research report, commissioned by Teradata and conducted by independent research firm Vanson Bourne in August 2021, explores global sentiment around data-driven transformation investments this year and beyond, and reveals where organisations are placing their bets to set themselves up for success in the years to come.
That’s the reality across sectors today.
In getting to compliance, organisations also have to deal with internal costs, effort, time, and risks involved in trying to detect all personal data manually, which is a tall order and thoroughly inefficient. It’s also the reason why many organisations are not enacting proper data governance—it’s often seen as an expense, not a business value.
To help organisations embrace data governance, we try to help them see how they are better served when they have a framework, tools and data stewards in place, again, highlighting its role in supporting valuable action, and minimise their exposure to any kind of regulatory risks.
EITN: Can you share the role Teradata plays in helping organisations glean more insights from data in real-time? Can you interoperate with graph analytics technologies? Please share about your tech and business partnerships/roadmap for the next 12 to 24 months.
Chris Jackson: Our platform, Teradata Vantage is a connected multi-cloud data platform for enterprise analytics that unifies everything—data lakes, data warehouses, analytics, and new data sources and data types that delivers unlimited intelligence for organisations. It gives users in any department across the organisation easy access answers to any question against integrated and connected cross-functional data in real-time. Teradata Vantage can analyze data in place and in the format it is already stored across an optimized data fabric and join that data together to answer queries or feed a downstream AI algorithm.
We help customers approach this not simply by gathering large data volumes, but through machine learning and AI, we are providing them with the ability to scale machine learning vertically by training models on more than millions of observations and scoring them against hundreds of million observations multiple times a day. Teradata Vantage also enables customers to scale horizontally by training millions of predictive models to support “hyper-segmentation” use cases and scoring them daily in demanding production settings.
With the introduction of Vantage Streams on AWS, we can ingest millions of records per second directly into Vantage from streaming sources such as Confluent Kafka and AWS Kinesis with very low latencies. Analytics can deliver solutions from Vantage such as Next Best Offer recommendations and real-time fraud detection leveraging behavior analysis and historical data patterns.
In 2022, we anticipate seeing the continued adoption of digital solutions as organisations look into enhancing the three key pillars that support their digitalisation journey—cloud infrastructures, data management, and emergent technologies. As organisations continue to operate remotely and embrace cloud services, modernising cloud architecture will help prevent data silos, maximise analytics value, and drive real-time answers at scale, and give them the agility and flexibility required to future-proof themselves in this unpredictable economic landscape. And it’s only through good data governance that provides the regulations and policies that guide the proper handling of data that will enable organisations to achieve the full potential and value of these technologies.
