The Power of Domain-Oriented, Federated Data Ownership
In today’s hyper-competitive environment, companies are increasingly opting for data lakes that enable real-time stream and batch data availability. Organizations are approaching this with the intent to ingest, enrich, transform, and serve data via a centralized platform.
However, for most organizations, this architecture contains several shortcomings:
- Data is siloed, restricted to those teams and use cases, limiting the ability to share.
- Lack of a holistic view, and the ability to discover data across the organization, limiting the ability to experiment and innovate.
- A centralized repository is limited to the personas and associated use cases for which it can serve.
- Such architectures are challenged with the ability to secure and govern data due to the central team’s limited understanding of the incoming data sources.
- Inevitably, these architectures are costly, inflexible, and slow, limiting their ability to support an ever-changing business.
Chaotic lakes of data result in impatient and disconnected data consumers in addition to backlogged data teams. This results in data drift and the further creation of data silos, as users look to gain access to data and insights through alternative means .
Our data mesh services offer a paradigm shift for domains and business areas, with the capability to perform enhanced analytics without the need of a centralized data architecture. This system puts control in the hands of your business, enabling it to be responsible to manage its own data quality, governance, security, and availability.
Trianz Data Mesh Maturity Model
Trianz created the Data Mesh Maturity Model to assist clients in assessing and planning their data mesh deployment. This provides a clear pathway to think big but start small, allowing the organization to incrementally adopt and learn at its own pace, based on its maturity, culture, and capability.
Experimentation. Utilizing data federation, business personas have immediate access to data to explore, understand, and gain insight. This stage connects multiple separate data siloes together to create a single panel of glass to said data.
Stage 1 is characterized by data utilization, but a lack of data exploitation. The business uses data as-is within its silo but lacks the ability to identify insights across domains to extract additional insight and value. Analytics is ad-hoc due to siloed data architecture, creating barriers to insight generation. Visibility of data meaning (single source of truth) due to limited data governance capabilities leads to data trust challenges.
Typically, the data strategy for each silo is owned and developed by IT with input from business stakeholders. It is more difficult for a true business-focused data approach, as each silo has different governance rules and access methods aligned with their needs. This results in organizational and cultural barriers to data access and insight generation.
At Stage 2, businesses must start to experiment with the elements of data ownership, where the silo is responsible or “owns” that data. In other words, it is responsible for the quality, security, governance, consistency, and availability of data within its domain.
At this stage, the business begins to understand that the data strategy is critical to its business strategy, and therefore starts to take more ownership of that strategy. To support such strategies, data agility becomes a critical focus. To support such agility, data connection and sharing across domains becomes imperative. However, this stage still involves centralized control, with centralized teams performing a majority of the activities around data operations.
The business has formed the general guardrails for data governance and access. The next step is to work towards federating this governance and access control framework so the business can mature to federated ownership and the development of data products.
Domains are actively taking ownership and stewardship of the development and sharing of data products. The concepts and practices of federated data ownership start to mature. As data products are established, these products are shared across domains and are used to create new customer experiences, generate new insights, explore new opportunities, and increase operational efficiency.
Stage 4 results in lower time-to-market for analytics as well as lower costs, in addition to enabling self-provisioning capabilities and reducing data team workloads while increasing output.
Governance is baked in and automated in the development of data products and data access, with flexible governance and security frameworks enabling data-as-a-Service models, as well as DataOps. The user experience is greatly improved and IT becomes a true enabler for the organization to be data driven.
This stage is where businesses reach full data mesh maturity. All data is treated as a product, with governance and ownership federated and managed by the originating data owner—with the flexibility of centralized guardrails. Data and Analytics (D&A) is a central pillar of the wider business strategy, with data guiding the majority of important business decisions. Insights are everyday decision making and touch points, leading to greater awareness and agility.
The organization is able to rapidly access new data sources at will, integrate, and create or enhance data products to accelerate the data advantage. Automated governance and security capabilities, coupled with absolute transparency greatly reduces risk. Domains are truly accountable and responsible for the data products within their domain. This reduces ambiguity by establishing a data trust through increased data quality, while also providing clear ownership to rapidly resolve issues.
