Data Preload and Caching
A good Data Virtualization and Data Federation solution will manage the idiosyncrasies of the data sources to provide the right virtualized data services layers at the right time to the applications and projects across the enterprise.
All data sources have certain attributes that a good virtual data integration tool should manage according to the project requirements. For instance, data source latency (how long is it going to take to access this source?), data source reliability (is the source going to respond correctly every single time it is called?) and cost (how many dollars does it cost to access this source?) are some such attributes. Also some data sources may have constantly-changing data, some others have static data. Being able to intelligently manage the data sources is an integral part of the recipe for success in virtual data integration.
To achieve this, Denodo has an advanced cache system and a Task Scheduler, that lets users define the type of access to each data source:
- Real-time data access
- Cached data access
- Combination of the above
With real-time data access, Denodo accesses the original data repositories directly every time virtual data is fetched. For cached data access, Denodo preloads the federated data to a new repository.
Preloading the federated data to a new repository (usually the Denodo cache) will make virtual data access faster since Denodo will access this new repository instead of the original one. Denodo also allows a consolidated approach that combines the above two types of access.
An application built on the Denodo Data Services Platform will therefore adequately balance between the data sources that are preloaded and the ones accessed in real time.
