Advantage

  • decentralize the data team (don’t need to have data engineer and analyst always work together for each project)

    centralized-vs-decentralized-3.png

    centralized-vs-decentralized-4.png

  • reduce script silos

  • Remove the need of avoiding writing boilerplate DML and DDL

  • Test and doc at the same time

  • dynamic referencing/ (Write DRYer code by leveraging macros, hooks, and package management) parameterization

  • DAG for modelling the whole transformation process

  • maintain idempotency

  • adopt versioning/ CI/CD practice, integrated with Git

  • dry code (but be aware of readability! Too many macro/ small component are very difficult to undertsand )

Incoperated with pythonic template language Jinja - foundation of referening and macro

scalable