Data
Table of contents
- Purpose
- Data contracts
- Schema and validation
- Artifacts
- Lineage
- Versioning
- Privacy and governance
- Related guides
Purpose
The data layer provides reproducible, validated inputs for retention analysis, modeling, monitoring, and dashboard evidence.
Data contracts
Each dataset should document:
text
source
grain
primary identifiers
target definition
feature window
observation window
allowed nullability
expected types
refresh behaviorSchema and validation
Validation should cover:
text
required columns
types
ranges
category domains
duplicate keys
missingness
temporal consistency
target leakageArtifacts
Versioned analytical artifacts should be reproducible from code and source data.
Avoid committing sensitive raw data.
Lineage
Every dashboard result should be traceable to:
text
dataset version
transformation
model version
evaluation version
explanation evidenceVersioning
Use immutable dataset and artifact identifiers whenever practical.
Monitoring and retraining workflows must know which data version produced a model or report.
Privacy and governance
Retention data may involve high-impact employment context. Apply data minimization, access control, appropriate retention periods, and human review.