Pipeline health
Last updated
Last updated
The Pipeline Health dashboard provides a project-wide visualization of all pipelines, allowing you to monitor and manage pipelines and jobs efficiently.
You can view historical data, and filter pipelines by run status, schedule states, and tags. Inspecting each entry to view details of each pipeline, and perform key actions like triggering, editing, or activating / deactivating the pipelines and jobs
This guide walks you through all components of the Pipeline Health dashboard.
To access the dashboard, in your project, click on Health tracking.
The Pipeline distribution chart represents the total number of pipelines with active jobs distributed across pipeline different categories, and the Pipeline health trend displays the total number of pipeline jobs with and their statuses, within a specified timeframe.
Below pipeline charts is the list of all pipelines and associated jobs.
Here you can use the top search bar to quickly search for pipeline from the list. The sort and filter functions help you navigate and group the pipelines by a combination of tags, owners, and run statuses.
Each pipeline item displays the list of associated jobs and relevant information, including:
Job name
Environment: development or production.
Last run time
Schedule
Recent runs: the last 14 runs; the height of each bar represents the run duration and its color indicates the status (green for successful and red for failure).
Status toggle: activate or deactivate the job.
To view the details of a job, click on the job item's name or click the action button and select View job details.
By default, ReOrc opens the latest job run details in the graph view. Each node represents a task, with run status and last run time.
The graph view provides a visual representation of task dependencies and relationships. It is best used to understand workflow structure and task dependencies at a glance.
A job can have multiple runs over a period of time. To filter, you can use the Date and Run ID filter functions. Using the search bar, you can highlight a task or a group of tasks (nodes).
After locating a specific run, you can view the details of a task by clicking on the corresponding node. Task details display two key information in two tabs:
Query: the complied query of the model.
Run logs: details about the materialization process and results of data tests.
Switch to the Grid tab to view job details in grid view.
The grid view displays all tasks in the job. Each column represents a job run and the column height indicates the execution duration of the run. Each square represents an individual task run, with color-coded status.
Grid view displays tasks in a table format with detailed status and timing. This view is best used for tracking historical runs and comparing task performance across multiple runs.
Using the search bar, you can select a specific group of tasks to focus on. The filter tool allows you to:
Filter by runs: select a range of runs (last 10, 30, 50 runs, or a customized number).
Filter by date: display all runs in a specific date range.
Filter by status: Completed, Failed, In progress, No status.
Similar to the graph view, you can click on a square to see the corresponding task details.
