Medallion architecture
The medallion architecture is a framework for describing a set of data transformations within a data lakehouse or data warehouse. In the medallion architecture, data moves through several steps:
- Bronze: ingestion of raw, unfiltered data - which may come from operational databases, from logs, from sensors, or other sources - into bronze data tables
- Silver: initial processing of raw data, including filtering, cleansing, and deduping, as well as combinations of data from multiple tables - into silver data tables
- Gold: further processing of data to produce results that speed up business-specific queries and dashboards - stored in gold tables
There are many advantages to the medallion architecture, which include the governance advantages of maintaining an original copy of raw data; additional governance advantages from having the results of intermediate processing steps; the performance advantage of re-using tables from one process for repeated or new processes; the ability for open source projects and proprietary products to add value at specific steps in the process; and enhanced comprehensibility of the architecture to all stakeholders.
The medallion architecture may incur additional costs due to the storage of multiple copies of data as it goes through various transformations, but this may be offset by governance advantages and potential performance advantages from having the copies available.
Related terms: ingest; query processing
On the Onehouse website:
Stay in the know
Be the first to hear about news and product updates