ReOrc docs
Get ReOrc
English
English
  • About ReOrc
  • Set up and deployment
    • Set up organization
    • Install ReOrc agent
  • Getting started
    • 1. Set up a connection
      • BigQuery setup
    • 2. Create a project
    • 3. Create data models
    • 4. Build models in console
    • 5. Set up a pipeline
  • Connections
    • Destinations
      • Google Service Account
    • Integrations
      • Slack
  • Data modeling
    • Overview
    • Sources
    • Models
      • Model schema
      • Model configurations
    • Jinja templating
      • Variables
      • Macros
    • Materialization
    • Data lineage
    • Data tests
      • Built-in generic tests
      • Custom generic tests
      • Singular tests
  • Semantic modeling
    • Overview
    • Data Modelling vs Semantic Layer
    • Cube
      • Custom Dimension
      • Custom Measure
        • Aggregation Function
        • SQL functions and operators
        • Calculating Period-over-Period Changes
      • Relationship
    • View
      • Primary Dimension
      • Add Shared Fields
    • Shared Fields
    • Integration
      • Guandata Integration
      • Looker Studio
  • Pipeline
    • Overview
    • Modeling pipeline
    • Advanced pipeline
    • Job
  • Health tracking
    • Pipeline health
    • Data quality
  • Data governance
    • Data protection
  • Asset management
    • Console
    • Metadata
    • Version history
    • Packages and dependencies
  • DATA SERVICE
    • Overview
    • Create & edit Data Service
    • Data preview & download
    • Data sharing API
    • Access control
  • AI-powered
    • Rein AI Copilot
  • Settings
    • Organization settings
    • Project settings
    • Profile settings
    • Roles and permissions
  • Platform Specific
    • Doris/SelectDB
Powered by GitBook
On this page
  • Requirements
  • Set up ReOrc Agent
  • Actions on an agent
  • Run the agent in the background
  • Upgrade the agent
  1. Set up and deployment

Install ReOrc agent

PreviousSet up organizationNextGetting started

Last updated 15 days ago

Data Plane of ReOrc consists of multiple worker hosts to run the data jobs orchestrated from the Control Plane.

Each worker contains a service called ReOrc Agent that communicates with Control Plane and manages execution environments on the host. Some rules for setting workers are as follows:

  • Multiple workers can be set up for horizontal scalability.

  • Workers can be installed in different places across the Internet.

  • Workers that are meant for a single environment must be able to communicate with each other. Therefore, it is recommended that workers for the same environment be located in the same local network, for maximum connection stability and minimum latency.

This guide shows you how to set up ReOrc agent.

Requirements

Your worker hosts should meet these requirements.

Hardware:

Depending on the usage, your setup can follow these configuration ranges to optimize the performance with ReOrc.

Number of workers
CPU (cores / worker)
Memory (GB / worker)
Disk (GBs/worker)

Minimum

1

2

4

20

Recommended

2+

4

16

100

Software:

  • Operating System: Linux (RHEL and Ubuntu are tested).

  • Python: 3.9+, we recommend using a newer version and virtual environment if feasible.

  • : 20.10.0+, with Docker Compose installed.

Make sure the user runs the ReOrc agent has access to docker cli. This command might be helpful. You may need to logout and re-login to the terminal session after modifying the user's group.

sudo usermod -aG docker $USER 

Set up ReOrc Agent

Follow these steps:

  1. On the left sidebar of ReOrc, click on the Organization icon and select Environment & Workers.

  2. Click Deploy Agent and follow the on-screen instructions to deploy the agent.

    Alternatively, follow the steps below.

Install ReOrc agent in CLI
  1. On the worker host: Install the agent package.

pip install -U reorc-agent

You can verify the installation by running the reorc version.

  1. On the worker host: Register the host to the Control Plane

reorc login
  1. On the Environments & workers page, click Deploy Agent, copy the API key, and paste it to the host.

  2. On the worker host: Start the agent; it will keep sending heartbeats to the Control Plane.

reorc agent start
  1. Install ReOrc agent.

    It is essential to maintain persistent functionality and connection of the agent.

  1. Refresh the page to see the newly deployed host. The status should be Online.

  2. Allocate the agent to an environment.

    Once the agent is running with the Online status, click action > Join Environment and select the environment to allocate. When the agent successfully joins the environment, you should see the Active status.

Note:

  • Both dev and prod must have at least one agent running.

Actions on an agent

Once the agent is up and running, there are some key actions that you can perform with the service. Click on the action button of an agent:

Here you can:

  • Upgrade Service: Upgrade the worker service to the latest version.

  • Restart Service: restart the agent service; the agent will still join the environment after restarting.

  • Stop Service: stop the agent service; the agent will become inactive.

  • Leave Environment: remove the agent service from the current environment.

Run the agent in the background

systemd
  1. Install the recurve service with the following command line, and the service will be started automatically after it has been installed.

recurve service install
  1. Use the following command to manage the service

systemctl start recurve-agent
systemctl stop recurve-agent
systemctl status recurve-agent

Upgrade the agent

Keeping the reorc-agent updated is always recommended.

To upgrade the agent , follow these steps:

  1. Stop the agent service.

    Depending on the way you manage the service, you may need to run one of these commands to stop the process:

# if managed by Systemd
sudo systemctl stop reorc-agent

# if managed by Supervisor
supervisorctl stop reorc-agent
  1. Upgrade and re-login.

# Upgrade to the latest version
pip install -U reorc-agent

reorc login
  1. Start the agent.

# if managed by Systemd
sudo systemctl start reorc-agent

# if managed by Supervisor
supervisorctl start reorc-agent

The reorc agent start command starts the agent in the foreground, and will terminate after the terminal session closed. For production deployment, you should let the agent run in the background. For more details, refer to .

There are multiple ways to run an agent in the background with .

Docker
Run Agent in Background
systemd