Walk through a simple data ecosystem and how to build in testing at each failure point. Then we'll focus on setting up automated alerts.
Valorem Reply Data Scientist, Joseph Hobbs will be presenting a session at SQL Saturday KC- Automating Data Integrity Checks.
Automating Data Integrity Checks- SQL | PBI | Flow
Duration: 75 minutes
Track: BI Information Delivery
Everyone's had their data pipeline break. The difficulty is often pinpointing which of the many failure points caused the issue. Did the source system not provide data? Was it the ETL? Did the reporting layer fail? Maybe the data really is just weird this month? This talk is an intermediate level solution to that pain point. We'll walk through a simple data ecosystem and how to build in testing at each failure point. Then we'll focus on setting up automated alerts.