Data Mesh Governance / Policies / Isolation / Project Structure
For consistency, we want a uniform structure and naming of our BigQuery projects.
The structure must fit to BigQuery’s strict 3-level-hierarchy:
BigQuery has some naming restrictions: Project IDs must be 6-30 characters, contain letters, numbers, and hyphens and are globally unique, cannot be in use or have previously been used. Datasets and table names can contain up to 1024 characters, numbers and underscores.
We agree on a set of conventions for our BigQuery projects, datasets and tables:
If applicable, more datasets can be defined by adding a suffix, separated by an underscore, e.g.
The BigQuery project structure can be set up through a self-service web-app, when a new data product is created.
A dbt hook can be implemented that makes sure that all models use the defined prefixes.