The Data Observability Pyramid
Data Quality
Monitoring data accuracy and data values
Data Profiling
Monitoring data distribution and segmentation
Data Observability
Monitoring data at rest for issues related to missing or stale data
Operational Observability
Monitoring data in motion for broken pipelines, including job failures, latency, not started jobs and data movement
Data Pipeline Traceability
Cross-platform lineage of data pipelines across the data stack including jobs and datasets