The core idea of modern data management is to improve the generation of data-driven business insight based on two main drivers:
- Bring agility to the data analytics process, often through self-service initiatives that reduce IT bottlenecks
- Make any piece of information available to analytical processes
However, any organization owns a long list of applications and systems focused on different aspects of the business, from supply chain management to marketing automation and a variety of data systems, like data warehouses, data marts, and data lakes.
This complex data landscape must be managed efficiently to achieve the above mentioned goals. Topics like agile data integration, data discovery, security, and governance are at the forefront of any modern data strategy.
To effectively implement those requirements, enterprises have turned to two different approaches:
- Centralized data strategies based on physical data consolidation and
- Logical data strategies based on consolidated view of data across disparate systems.
Challenges of a centralized approach
One size never fits all:
Operational RDBMs, noSQL, graphs, key-value pairs, data lake engines, enterprise data warehouse, etc., exist to address different data requirements. There is no single data system that provides capabilities for all data needs.
Take significant time and effort:
Intensive data replication scripts must be designed for each new data need.
Consolidation may be prohibited:
Regional data privacy regulations often limit data consolidation.
Does not reduce data silos:
For 30+ years, we have tried consolidating everything into a data warehouse. Cloud alone will not change the outcome.
Logical data management
Logical data architecture and management enable access to multiple, diverse data sources while appearing as one “logical” data source to users. It is about unifying data that are stored and managed across multiple data management systems, including traditional data sources like databases, enterprise data warehouses, data lakes, etc., and other data sources like applications, big data files, web services, and the cloud to meet every analytics use case.
Logical data management allows practices like data discovery, access, security, integration, and sharing to be performed through a logical (or virtual) representation instead of directly on each physical source system. Logical data management practices enable the consistent implementation of policies and practices to manage, integrate and use an organization’s data, regardless of each source system's nature, location, and capabilities.
The market has realized that those data integration tools that do not balance “collect”- with “connect”-based data management architecture strategies will always result in data silos and/or poorly integrated infrastructures.”
The Required Capability for Logical Data Management
To successfully implement a logical data architecture and management approach, organizations need only one capability:
Data virtualization is a data integration and data management technology that leverages metadata to enable organizations to access all enterprise data in real-time, and to discover, catalog, provision, combine, share, and govern data to meet a wide variety of use cases..