Advantage
-
decentralize the data team (don’t need to have data engineer and analyst always work together for each project)
-
reduce script silos
-
Remove the need of avoiding writing boilerplate DML and DDL
-
Test and doc at the same time
-
dynamic referencing/ (Write DRYer code by leveraging macros, hooks, and package management) parameterization
-
DAG for modelling the whole transformation process
-
maintain idempotency
-
adopt versioning/ CI/CD practice, integrated with Git
-
dry code (but be aware of readability! Too many macro/ small component are very difficult to undertsand )
Incoperated with pythonic template language Jinja - foundation of referening and macro
scalable