June 20-22 Announcing HashiConf Europe full schedule: keynotes, sessions, labs & more Register Now
  • Infrastructure
    • terraform
    • packer
  • Networking
    • consul
  • Security
    • vault
    • boundary
  • Applications
    • nomad
    • waypoint
    • vagrant
  • HashiCorp Cloud Platform

    A fully managed platform to automate infrastructure on any cloud with HashiCorp products.

    • consul
    • terraform
    • vault
    • packerbeta
    Visit cloud.hashicorp.com
  • Overview
  • Tutorials
  • Docs
  • CLI
  • Plugins
  • Community
GitHub
Download
    • v0.8.x (latest)
    • v0.7.x
    • v0.6.x
    • v0.5.x
    • v0.4.x
    • v0.3.x
    • v0.2.x
    • v0.1.x
    • Overview
      • Overview
      • Helm
      • Heroku, Vercel, etc.
      • Kubernetes
  • Getting Started
    • Overview
    • Compatibility Promise
    • Protocol Version Table
    • Release Notifications
      • Overview
      • Upgrade to 0.2.0

    • Install
    • Externally Built Images
    • Building Container Images
    • Helm Deployment
    • YAML-Free Deployment
    • YAML Directory Deployment
    • Resource Status
    • ConfigMaps and Secrets

    • Overview
    • Git Integration
    • Remote Operations
    • Overview
    • Build
    • Deploy
    • Release
    • Hooks
    • Labels
    • Workspace and Label Scoping
    • Overview
      • Overview
      • Input Variables
      • External Data
      • artifact
      • deploy
      • entrypoint
      • labels
      • path
      • workspace
      • Overview
      • Full Reference
      • Templating
      • Overview
      • Expressions
      • JSON Syntax
    • app
    • build
    • config
    • deploy
    • hook
    • plugin
    • registry
    • release
    • runner
    • url
    • use
    • variable
  • URL Service
  • Logs
  • Exec
    • Overview
    • Dynamic Values
    • Files
    • Internal Values
    • Workspace and Label Scoping
    • Overview
      • Overview
      • OIDC
      • Overview
      • Maintenance
      • Production
      • Security
    • Express Server Install
    • Overview
    • Configuration
    • Profiles
    • On-Demand Runner
    • Additional Runners
  • Workspaces
  • Plugins
  • Triggers

    • Overview
      • Overview
      • Registering Plugin Components
      • Handling Configuration
      • Implementing the Builder Interface
      • Compiling the Plugin
      • Creating an Example Application
      • Testing the Plugin
    • Initializing the SDK
    • Passing Values Between Components
      • Overview
      • Authenticator
      • Configurable
      • ConfigurableNotify
      • Builder
      • Registry
      • Platform
      • ReleaseManager
      • Destroy
      • Status
      • Default Parameters
      • Overview
    • Overview
    • Disable
    • Overview
    • GitHub Actions
    • GitLab CI/CD
    • CircleCI
    • Jenkins
  • Troubleshooting
  • Glossary

    • Overview
    • Architecture
    • Operation Execution
  • Roadmap
Type '/' to Search

»On-Demand Runners

On-demand runners allow a single Waypoint installation to scale very high by leveraging the defined platform to spawn resources to execute Waypoint jobs remotely. Through on-demand runner profiles and runner targeting, Waypoint can even utilize resources from other platforms that Waypoint has access to. For example, a local Kubernetes server install could be configured to use Amazon ECS to execute jobs.

»Job Assignment

When a job is submitted to the server, if on-demand runners are enabled, it will spawn an on-demand runner and the job is placed in a holding set. This is the same mechanism we use for automatic CLI runners. When the on-demand runner starts, it will execute the normal Waypoint runner code and connect back to the server. On connection, the server will see the job in the holding set and send it immediately to the new runner.

The flow of information about the job uses the normal grpc runner APIs so all the input and output remain the same as if the work were performed by the CLI-runner.

Upon finishing the job, the on-demand runner will exit and the platform it runs on will reclaim its resources.

»Launching Tasks to Execute Jobs

When Waypoint goes to launch an on-demand runner task, it will generate three jobs:

  • StartJob
  • TaskJob
  • StopJob

The start job is responsible for scheduling a resource in the platform to execute the assigned job. Once that resource exists, it will be a TaskJob that is responsible for actually running the assigned job. Once it's complete, the task plugin system will launch a stop job to clean up the launched platform resource.

»On-Demand Runner Profiles

Users can configure Waypoint server to launch on-demand runners with specific configurations. These configurations are set through on-demand runner profiles. See runner profiles for more information.

»OCI/Docker Image

By default, the server will launch on-demand runners using the same OCI image it was configured with. The on-demand configuration will also accept a specific OCI image url to use instead. This allows operators the ability to prepackage any Waypoint plugins into the image and use them in their configuration.

The official on-demand runner container image is built very similar to Waypoint server, with some additional packages such as Kaniko for doing in-container builds for jobs like the Build operation.

See the runner profile docs on OCI URLs for more information.

»Official Waypoint Supported Task Plugin Platforms

When you use the waypoint server install command, we automatically set up a default On-Demand runner profile for you to use with Waypoint Server. Like our server install platform support, we also currently support these platforms for launching on-demand runner tasks:

  • Kubernetes
  • HashiCorp/Nomad
  • AWS/ECS
  • Docker

»Adding on-demand runner support for other platforms

On-demand runner support can be implemented for additional platforms using the plugin SDK.

Plugins expect to fulfill the SDK interface with the following functions:

  • StartTask
  • StopTask

These functions are what will launch and stop the spawned resource with the task launcher.

Task plugins also can accept TaskLauncherConfig configuration which users can configure through an on-demand runner profile.

github logoEdit this page

Using Waypoint

The best way to understand what Waypoint can enable for your projects is to give it a try.

Waypoint tutorials
Waypoint documentation
Tutorial

Get Started - Kubernetes

Build, deploy, and release applications to a Kubernetes cluster.

View
Tutorial

Introduction to Waypoint

Waypoint enables you to publish any application to any platform with a single file and a single command.

View

Waypoint is maintained by HashiCorp, Inc.

View Code of Conduct
DocumentationCLI ReferenceTutorialsIntegrations
All systems normal