Denodo Demo: Mastering the Data Product Lifecycle

WEBVTT

1
00:00:05.620 --> 00:00:07.360
Narrator: [Upbeat tech music intro] Data has never been more accessible.

2
00:00:08.300 --> 00:00:11.860
Narrator: Yet turning it into something the business and AI can trust and use

3
00:00:12.040 --> 00:00:12.940
Narrator: remains a challenge.

4
00:00:14.220 --> 00:00:18.000
Narrator: Data products help organizations deliver trusted, reusable data for

5
00:00:18.040 --> 00:00:19.740
Narrator: business and technical teams alike.

6
00:00:21.020 --> 00:00:24.240
Narrator: As organizations push to enable data self-service and scale

7
00:00:24.300 --> 00:00:27.730
Narrator: AI, they need a consistent way to deliver data with clear

8
00:00:27.800 --> 00:00:29.820
Narrator: meaning, context, and governance.

9
00:00:30.660 --> 00:00:32.741
Narrator: That is where data products play a critical role.

10
00:00:33.760 --> 00:00:37.720
Narrator: Without a unified approach, however, users and AI agents often

11
00:00:37.780 --> 00:00:41.620
Narrator: face fragmented access, inconsistent definitions, and limited

12
00:00:41.640 --> 00:00:42.180
Narrator: governance.

13
00:00:43.320 --> 00:00:47.050
Narrator: Denodo addresses this with a unified semantic layer, centralized

14
00:00:47.120 --> 00:00:50.120
Narrator: governance, and real-time access to distributed data.

15
00:00:51.140 --> 00:00:54.500
Narrator: In this video, we walk through the data product lifecycle from the

16
00:00:54.520 --> 00:00:58.300
Narrator: perspectives of three key roles: the business user, the data

17
00:00:58.320 --> 00:01:02.290
Narrator: product owner, and the developer, showing how Denodo simplifies

18
00:01:02.320 --> 00:01:04.371
Narrator: each stage to accelerate time to value.

19
00:01:06.400 --> 00:01:09.670
Eliana Nordstrom: [Screen showing Denodo Data Marketplace] I'm Eliana Nordstrom,
a marketing analytics manager.

20
00:01:10.580 --> 00:01:14.560
Eliana Nordstrom: My company's online course portal needs a more engaging user experience
to boost enrollment and guide users to the right courses.

21
00:01:14.620 --> 00:01:17.500
Eliana Nordstrom: To answer this, I turn to the Denodo Data Marketplace.

22
00:01:18.300 --> 00:01:21.460
Eliana Nordstrom: I search for a certified data product that could provide the insight I need,

23
00:01:22.280 --> 00:01:25.960
Eliana Nordstrom: specifically information on user web events.

24
00:01:26.510 --> 00:01:29.720
Eliana Nordstrom: To find it, I chat directly with the Denodo assistant, and

25
00:01:30.740 --> 00:01:34.690
Eliana Nordstrom: I quickly locate a promising data product called Customer Web Events.

26
00:01:34.740 --> 00:01:38.420
Eliana Nordstrom: From here, I explore its definition, the properties it contains,

27
00:01:38.480 --> 00:01:39.020
Eliana Nordstrom: how it's been built, and where the data comes from.

28
00:01:43.400 --> 00:01:45.759
Eliana Nordstrom: Now, I preview the data itself to get a sense of its content.

29
00:01:46.560 --> 00:01:50.480
Eliana Nordstrom: Everything looks useful, but I notice two key gaps for my analysis.

30
00:01:50.500 --> 00:01:51.740
Eliana Nordstrom: First, there's no customer profiling information,

31
00:01:57.300 --> 00:02:01.060
Eliana Nordstrom: details that could help me segment users and understand behavior patterns.

32
00:02:02.540 --> 00:02:05.940
Eliana Nordstrom: Second, the dataset only covers the past six months.

33
00:02:05.960 --> 00:02:06.620
Eliana Nordstrom: For the analysis I'm planning, I need a longer historical view,

34
00:02:10.140 --> 00:02:13.090
Eliana Nordstrom: ideally the past two years.

35
00:02:13.880 --> 00:02:17.720
Eliana Nordstrom: Recognizing these limitations, I submit a change request in the marketplace.

36
00:02:22.280 --> 00:02:25.640
Eliana Nordstrom: I specify the two enhancements I need. First, extend the data

37
00:02:25.640 --> 00:02:29.900
Eliana Nordstrom: coverage period from six months to two years.

38
00:02:30.700 --> 00:02:32.300
Eliana Nordstrom: Second, add customer profiling fields to enrich the dataset.

39
00:02:37.400 --> 00:02:41.160
Eliana Nordstrom: With the request submitted, the process is set in motion to refine this data

40
00:02:41.220 --> 00:02:44.800
Eliana Nordstrom: product so it can deliver the insights I need to improve the course

41
00:02:45.580 --> 00:02:49.500
Eliana Nordstrom: selection experience for the users.

42
00:02:50.340 --> 00:02:53.820
Alexis Cheng: [Notification alert] Hi, this is Alexis Cheng, the data product owner
responsible for customer web events.

43
00:02:53.860 --> 00:02:55.840
Alexis Cheng: I receive an email notification and head to the Denodo Data Marketplace

44
00:02:57.940 --> 00:03:01.880
Alexis Cheng: to review the request from Eliana.

45
00:03:01.900 --> 00:03:05.660
Alexis Cheng: I examine the details and mark the ticket as in progress.

46
00:03:05.680 --> 00:03:07.520
Alexis Cheng: I then send Eliana a quick message letting her know I'm starting work.

47
00:03:10.900 --> 00:03:14.880
Alexis Cheng: The next step is to make these changes in the Denodo Design Studio,

48
00:03:14.900 --> 00:03:15.320
Alexis Cheng: where all data product models are built and maintained.

49
00:03:18.530 --> 00:03:22.460
Alexis Cheng: I begin by creating a new workspace. This is a dedicated feature branch
connected to Git.

50
00:03:22.500 --> 00:03:24.100
Alexis Cheng: This branch enables my team to develop and test changes in isolation

51
00:03:27.740 --> 00:03:31.120
Alexis Cheng: without affecting other developments.

52
00:03:38.700 --> 00:03:42.560
Alexis Cheng: With the workspace ready, I switch into it and open the customer web events

53
00:03:51.520 --> 00:03:55.500
Alexis Cheng: data product definition.

54
00:03:55.580 --> 00:03:58.100
Alexis Cheng: Here, I add the new fields needed to include the customer profiling

55
00:04:00.540 --> 00:04:02.600
Alexis Cheng: Eliana is requesting.

56
00:04:04.060 --> 00:04:06.860
Alexis Cheng: I leverage AI capabilities to generate clear, consistent descriptions for

57
00:04:08.220 --> 00:04:11.300
Alexis Cheng: these fields and to automatically classify the data.

58
00:04:11.360 --> 00:04:14.000
Alexis Cheng: [UI showing automated field classification] During classification, the system

59
00:04:24.460 --> 00:04:28.360
Alexis Cheng: detects a field containing email information and applies a PII email tag.

60
00:04:28.440 --> 00:04:29.640
Alexis Cheng: This tag is linked to a global security policy, which ensures that sensitive

61
00:04:42.340 --> 00:04:45.840
Alexis Cheng: information will be masked for any roles that should not see it.

62
00:04:46.300 --> 00:04:47.520
Alexis Cheng: I quickly navigate to the policy to confirm the masking rules,

63
00:04:53.660 --> 00:04:57.650
Alexis Cheng: reinforcing the product's compliance with governance standards.

64
00:04:57.680 --> 00:05:00.469
Alexis Cheng: With the updated definition complete, essentially the data contract for the

65
00:10:00.000 --> 00:05:13.700
Alexis Cheng: published view, I commit the changes to the Git repository and push them

66
00:05:13.720 --> 00:05:17.320
Alexis Cheng: to the feature branch.

67
00:05:23.140 --> 00:05:26.340
Alexis Cheng: The modifications are now ready for the next phase of development, where they'll

68
00:05:26.380 --> 00:05:30.290
Alexis Cheng: be implemented and tested before moving into production.

69
00:05:30.300 --> 00:05:33.940
Suresh: With the updated definition in place, the changes now move to me, Suresh.

70
00:05:33.960 --> 00:05:37.690
Suresh: As the data product developer, I will implement them in the workspace

71
00:05:37.700 --> 00:05:38.630
Suresh: Alexis created.

72
00:05:41.620 --> 00:05:45.520
Suresh: The historical web events data is stored in an S3 bucket formatted as an Iceberg file.

73
00:05:45.580 --> 00:05:49.480
Suresh: I therefore leverage the Denodo Lakehouse Accelerator.

74
00:05:49.560 --> 00:05:50.840
Suresh: With it, I can explore and onboard the data assets straight from the S3 path.

75
00:05:54.020 --> 00:05:57.840
Suresh: The Lakehouse Accelerator enables me to query, manipulate, and integrate any

76
00:05:57.880 --> 00:06:00.800
Suresh: dataset in the lake just like any other source.

77
00:06:06.796 --> 00:06:10.356
Suresh: Once completed, the Iceberg data becomes a base view in the virtual model.

78
00:06:10.676 --> 00:10:14.536
Suresh: I then create a derived view on top of it, adding semantic clarity, renaming it

79
00:06:14.576 --> 00:06:16.136
Suresh: Web Events Historical to reflect its contents.

80
00:06:20.276 --> 00:06:23.696
Suresh: Using Denodo's GenAI integration, I quickly generate descriptive field names

81
00:06:24.256 --> 00:06:27.836
Suresh: and metadata. This accelerates documentation and ensures the dataset is

82
00:06:27.956 --> 00:06:31.416
Suresh: business-friendly for future users.

83
00:06:31.476 --> 00:06:35.256
Suresh: With the historical data ready, I move to combine it with the existing current

84
00:06:35.996 --> 00:06:39.836
Suresh: web events dataset, which is stored in BigQuery.

85
00:06:39.836 --> 00:06:43.816
Suresh: Using a graphical union view, I simply drag and drop both datasets together.

86
00:06:49.796 --> 00:06:53.756
Suresh: Denodo automatically aligns the structures, and the new integrated view now

87
00:06:53.856 --> 00:06:54.176
Suresh: contains both historical and current event data, seamlessly blending in

88
00:07:08.676 --> 00:07:12.416
Suresh: real-time sources from the data lake and the cloud warehouse.

89
00:07:12.426 --> 00:07:16.416
Suresh: The next step is to add customer profiling data, which resides in a Postgres

90
00:07:16.456 --> 00:07:17.416
Suresh: database in the marketing domain.

91
00:07:28.676 --> 00:07:32.236
Suresh: This source has not yet been connected to Denodo, so I set up the new connection

92
00:07:32.256 --> 00:07:34.216
Suresh: using credentials stored in our corporate credentials vault.

93
00:07:43.336 --> 00:07:47.096
Suresh: Then describe source location and assign it to the correct folder in the

94
00:07:47.156 --> 00:07:49.096
Suresh: metadata tree.

95
00:08:01.136 --> 00:08:04.966
Suresh: And finally, introspect the Postgres catalog to bring the relevant asset,

96
00:08:05.076 --> 00:08:07.956
Suresh: which is, in this case, customer profile.

97
00:08:15.816 --> 00:08:19.416
Suresh: I bring the profile structure into the virtual model and again use AI to

98
00:08:19.456 --> 00:08:20.516
Suresh: standardize and enhance the field names for clarity.

99
00:08:30.356 --> 00:08:34.036
Suresh: With both datasets, web events and customer profiles now available, I create

100
00:08:34.076 --> 00:08:37.336
Suresh: a join view. GenAI assists by automatically determining the correct join

101
00:08:38.196 --> 00:08:42.096
Suresh: condition between the datasets.

102
00:08:42.136 --> 00:08:43.047
Suresh: The resulting customer web events integrated view combines the user behavior data

103
00:08:48.376 --> 00:08:52.116
Suresh: with rich profile attributes, meeting the requirements from Eliana's original

104
00:08:52.156 --> 00:08:54.426
Suresh: request and as defined by Alexis in the interface view or data model contract.

105
00:09:00.926 --> 00:09:04.806
Suresh: Satisfied that the new definition meets all criteria, I replace the old

106
00:09:04.836 --> 00:09:08.416
Suresh: implementation with this updated version.

107
00:09:09.036 --> 00:12:12.676
Suresh: The status of the data product changes from a warning state to fully implemented.

108
00:09:12.796 --> 00:09:14.256
Suresh: With the development work complete, I commit and push the changes to the

109
00:09:21.436 --> 00:09:25.316
Suresh: feature branch in Git.

110
00:09:25.326 --> 00:09:28.936
Suresh: The final step is a merge request to bring these changes from the feature branch

111
00:09:36.356 --> 00:09:40.096
Suresh: into the main branch, ensuring the updated data product is ready for

112
00:09:40.216 --> 00:09:44.096
Suresh: deployment to production.

113
00:09:59.016 --> 00:10:02.546
Alexis Cheng: [Screen transitions to Solution Manager] Hi, this is Alexis again. With the

114
00:10:03.256 --> 00:10:05.836
Alexis Cheng: new data product successfully implemented in the development environment,

115
00:10:08.956 --> 00:10:12.416
Alexis Cheng: the final step is to promote it into production, making it available for the

116
00:10:12.496 --> 00:10:14.136
Alexis Cheng: business to use.

117
00:10:25.796 --> 00:10:29.716
Alexis Cheng: Using the Denodo Solution Manager, I will deploy the updated customer web

118
00:10:29.796 --> 00:10:33.016
Alexis Cheng: events data product into the production environment.

119
00:10:33.676 --> 00:10:37.616
Alexis Cheng: I begin by creating a new revision. This revision is a complete package

120
00:10:37.656 --> 00:10:40.976
Alexis Cheng: of the changes made, containing all the VQL code and definitions that

121
00:10:41.016 --> 00:10:41.936
Alexis Cheng: make up the updated product.

122
00:10:45.796 --> 00:10:49.716
Alexis Cheng: I select the full data product with all its dependencies automatically included,

123
00:10:49.726 --> 00:10:53.376
Alexis Cheng: and assign the revision a descriptive name.

124
00:10:57.816 --> 00:11:01.176
Alexis Cheng: Once the revision is saved, it appears in the list of available revisions.

125
00:11:01.236 --> 00:11:02.976
Alexis Cheng: I validate it against the production environment to confirm everything is ready.

126
00:11:08.776 --> 00:11:12.736
Alexis Cheng: The checks pass, so I proceed with the deployment. With a simple click, the

127
00:11:12.776 --> 00:11:14.316
Alexis Cheng: revision is deployed into production.

128
00:11:34.836 --> 00:11:38.736
Alexis Cheng: Behind the scenes, Solution Manager handles all the details, moving the code

129
00:11:38.776 --> 00:11:42.666
Alexis Cheng: into the production Denodo cluster.

130
00:11:42.716 --> 00:11:45.836
Alexis Cheng: I can monitor the progress from the deployments menu, watching as the status

131
00:12:07.516 --> 00:12:11.480
Alexis Cheng: changes from deploying to completed.

132
00:12:11.560 --> 00:12:15.439
Alexis Cheng: Now the updated customer web events product is live and ready for consumption.

133
00:12:15.620 --> 00:12:19.540
Alexis Cheng: The last step for me is to notify the business that the request has been

134
00:12:21.160 --> 00:12:24.780
Alexis Cheng: fulfilled. So first, I mark the ticket as completed.

135
00:12:24.820 --> 00:12:27.970
Alexis Cheng: And then send Eliana a confirmation message letting her know the enhanced

136
00:12:31.000 --> 00:12:32.700
Alexis Cheng: data product is now available for her to use.

137
00:13:37.740 --> 00:13:40.780
Eliana Nordstrom: I am back in the Denodo Data Marketplace to check on the request I submitted

138
00:13:47.300 --> 00:13:50.980
Eliana Nordstrom: earlier. I can see that the ticket has been completed and the enhancements

139
00:13:51.040 --> 00:13:53.040
Eliana Nordstrom: I asked for are now in place.

140
00:14:02.420 --> 00:14:03.880
Eliana Nordstrom: I open the updated customer web events data product to confirm everything

141
00:14:07.920 --> 00:14:11.780
Eliana Nordstrom: meets my requirements. Right away, I notice the new customer profiling

142
00:14:16.020 --> 00:14:19.920
Eliana Nordstrom: fields I had requested have been added.

143
00:14:21.400 --> 00:14:23.660
Eliana Nordstrom: Sampling the data, I also see that some sensitive information is masked

144
00:14:28.480 --> 00:14:32.400
Eliana Nordstrom: to me. Exploring the data lineage, I spot the changes behind the scenes.

145
00:14:32.440 --> 00:14:35.380
Eliana Nordstrom: Current event data from BigQuery, historical data from the S3 data lake,

146
00:14:45.340 --> 00:14:49.220
Eliana Nordstrom: and the newly added profile data are all integrated into a single product.

147
00:14:49.260 --> 00:14:49.680
Eliana Nordstrom: When querying, this data is coming in real time from these data sources with

148
00:14:52.480 --> 00:14:56.420
Eliana Nordstrom: zero copy. I check the coverage period and confirm that the dataset now

149
00:14:56.480 --> 00:14:57.430
Eliana Nordstrom: spans the past two to three years.

150
00:15:04.420 --> 00:15:08.330
Eliana Nordstrom: With the updated product ready, I use the option to open it directly in

151
00:15:08.380 --> 00:15:11.930
Eliana Nordstrom: Power BI. From here, I can start building the reports and dashboards that

152
00:15:12.020 --> 00:15:15.480
Eliana Nordstrom: will help me uncover the insights I need.

153
00:15:22.280 --> 00:15:25.420
Narrator: Data products are not only valuable for analytics, but also crucial for scaling

154
00:15:25.440 --> 00:15:27.740
Narrator: agentic AI. However, as agents are built in isolation, they tend to proliferate

155
00:15:47.280 --> 00:15:50.740
Narrator: without coordination, resulting in fragmented access and disjointed semantics.

156
00:15:51.300 --> 00:15:55.280
Narrator: Denodo addresses this by offering live, unified, and governed data access

157
00:15:55.360 --> 00:15:59.100
Narrator: through a zero copy approach. By providing data as reusable governed data

158
00:15:59.160 --> 00:16:03.100
Narrator: products, Denodo separates AI builders from underlying data complexities.

159
00:16:03.220 --> 00:16:05.520
Narrator: AI scales effectively when intelligent applications can leverage trusted data

160
00:16:10.420 --> 00:16:14.280
Narrator: products built on a unified semantic foundation. Contact Denodo to explore

161
00:16:14.290 --> 00:16:18.120
Narrator: the right approach for your organization. [Upbeat music fades out]