Data Validation and profiling

1. Completeness
   1. definition: Ensures all required data is present and no essential parts are missing
   2. checks: missing values, null, % of populated fields
   3. importance: Missing data can lead to inaccurate analyses and insights
2. Consistency
   1. definition: Ensures data values are consistent across datasets and do not contradict each other
   2. checks: cross-field validation, comparing data from different sources or period
   3. importance: Inconsistent data can cause confusion and result in incorrect conclusions
3. Accuracy
   1. definition: Ensures data values are correct, reliable and represents what it is supposed to
   2. checks: Comparing with trusted sources, validation against known standards or rules
   3. importance: Inaccurate data can lead to false insights and poor decision-making
4. Integrity
   1. definition: Ensures data maintains its correctness and consistency over its lifecycle and across systems
   2. checks: Referential integrity, relationship validation
   3. importance: Ensures relationships between data elements are preserved, and data remains trustworthy over time