Packages and dependencies

As the core functionalities in ReOrc's data modeling leverages dbt, external dbt packages and project dependencies are also supported.

What are dbt packages?

Definition

dbt package refers to the encapsulation of dbt resources into a folder structure. Like code libraries in software engineering, dbt packages are reusable collections of resources (sources, models, and macros) that can be distributed and reused across multiple projects. These packages are often created by other teams or the community and provide useful functionalities, best practices, or industry-specific logic.

Benefits

• Code Reusability: Packages let you reuse tested models, macros, and transformations across multiple projects, saving development time.

• Standardization: Apply consistent logic and best practices across teams by using shared packages.

• Collaboration: Teams can share their work as packages, enabling other teams to leverage existing solutions without reinventing the wheel.

• Community Resources: Open-source packages provide industry-standard tools and utilities maintained by the community, reducing the need for custom development.

What are dbt project dependencies?

Definition

Leveraging the flexibility and modularity of packages, each dbt project can also be treated as a package. This means one project can rely on models, resources, or data from another. This setup allows multiple teams to collaborate and share data models while maintaining separate projects.

Dependencies between projects can be dynamically resolved, meaning you can reference models from another project without having to include their source code directly in your project.

Benefits

• Shared Logic: Teams can reuse models across projects, promoting consistency and reducing redundancy.

• Dynamic Resolution: Dependencies are resolved dynamically, meaning you don’t need to pull in the source code from other projects, making the setup cleaner and more manageable.

• Clear Boundaries: Each team maintains ownership over their project, while still allowing other teams to benefit from their work.

ReOrc simplifies the process of package import and dependency management through the Packages and Dependencies menus.

Manage packages

To access the Packages menu, from the top navigation bar of your project, click on Packages.

The menu displays a curated list of packages that you can install to your project.

Install a package

You can search for a package by name or filter by provider, and toggle on the Install option. The package's resources will then be available to use in your project.

You can then use the package's resources, such as macros in your models: Use macros from a package.

To view the full list of packages and their descriptions, refer to the Package Hub - a registry for dbt packages, maintained by the dbt Community: Package Hub.

Uninstall a package

To uninstall a package, simply navigate to the package row in the Packages menu and toggle off the Install option.

Note that after uninstalling, all the references to the package's resources will likely fail.

Manage dependencies

To access the Dependencies menu, from the top navigation bar of your project, click on Dependencies.

The Dependencies menu lists out all projects in your organization with relevant information, including the analytics database in use and the project owner.

Connect a dependent project

To connect another project in your organization as a dependent project, locate the project in the list and toggle on the Connect option.

All resources with the Public access from the dependent project, such as sources, models, macros, will be accessible from the current project. See: Metadata types.

Remove a dependent project

To remove a dependent project, locate the project in Dependencies and toggle off the Connect option.

Note that all resources in the current project that depend on the removed project will likely fail.