01 / Data & training
Which pipelines may train on which data?
Training jobs read datasets, use compute and write checkpoints. The application asks before each dataset read, job start and checkpoint export, passing sensitivity, tenant, consent and region as attributes.
- Subjects
- Training pipelines as entities; ML engineers and their teams.
- Resources and actions
- Datasets:
read. Training jobs:start. Checkpoints:export. - Enforcement point
- The data loader before it opens a dataset, the job scheduler before it allocates compute, and the artifact store before an export.
[service.ml-training]
[policy]
# Pipelines read only consented data from their own region
grant role training-pipeline read expr:^/datasets/ if consent == true && data_region == job_region
# Restricted datasets never enter training
deny role training-pipeline read expr:^/datasets/restricted/
grant role ml-engineer start /jobs/training if gpu_hours <= 64
grant role ml-engineer export expr:^/checkpoints/
[rolepolicy]
grant entity pipeline-churn training-pipeline
grant group ml-team ml-engineer
Consent and region checked per dataset, restricted data denied outright, and a GPU-hour limit on job start.