Custom Backend Adapters

A custom backend adapter is for a task system your project already runs, but Wove does not ship with yet. The backend owns the delivery path: queueing, scheduling, retries, worker placement, batch jobs, or cluster execution. The adapter only teaches Wove how to submit one serialized Wove payload into that system.

That split keeps task code independent from the backend. The weave selects an environment, the adapter submits the payload, and the backend worker calls Wove’s worker entrypoint to send the result back to the original weave.

Adapter Interface

Custom adapters subclass BackendAdapter.

class BackendAdapter:
    required_modules = ()
    install_hint = None

    def __init__(self, *, name, config, callback_url, run_config):
        ...

    async def start(self):
        ...

    async def submit(self, payload: dict, frame: dict):
        ...

    async def cancel(self, run_id: str, submission, frame: dict):
        ...

    async def close(self):
        ...

required_modules lists import names the adapter needs at runtime. Wove checks those modules before the adapter starts.

install_hint is the package name Wove shows when a required module is missing.

start() creates backend clients, opens queues, or validates configuration.

submit(payload, frame) sends one Wove payload to the backend and returns the backend’s submission handle. Wove stores that handle and passes it back to cancel(...).

cancel(...) asks the backend to cancel the submitted work when Wove delivery policy requires cancellation.

close() releases adapter-owned resources.

Payload Flow

Backend adapters do not execute the task directly. They submit the payload Wove built, and the worker side of the backend executes that payload later.

  1. A task routed to the adapter becomes ready after its upstream Wove dependencies resolve.

  2. Wove serializes the selected task callable, resolved task arguments, delivery settings, task identity, run identity, adapter name, and callback URL into payload.

  3. The adapter submits payload to the backend without changing its Wove fields.

  4. A backend worker receives payload.

  5. The worker calls wove.integrations.worker.run(payload) or await wove.integrations.worker.arun(payload).

  6. The worker entrypoint executes the task and posts task_started, task_result, task_error, or task_cancelled back to Wove’s callback URL.

  7. Wove’s callback server receives the event and the original weave resolves the pending task.

The callback URL is created by BackendAdapterEnvironmentExecutor. For workers on another host, configure callback_url so the worker can reach the Wove process that is running the original weave.

Minimal Adapter

This example shows the adapter shape for a fictional queue client. The important part is that submit(...) enqueues the Wove payload unchanged and returns the backend submission handle.

from wove import BackendAdapter


class AcmeQueueAdapter(BackendAdapter):
    required_modules = ("acme_queue",)
    install_hint = "acme-queue"

    async def start(self):
        import acme_queue

        self.client = self.config.get("client")
        if self.client is None:
            self.client = acme_queue.Client(self.config["url"])

        self.queue = self.config.get("queue", "default")

    async def submit(self, payload, frame):
        delivery = frame.get("delivery") or {}
        return await self.client.enqueue(
            self.queue,
            payload,
            job_id=delivery.get("delivery_idempotency_key") or frame["run_id"],
        )

    async def cancel(self, run_id, submission, frame):
        if submission is not None:
            await self.client.cancel(submission)

    async def close(self):
        await self.client.close()

The backend worker receives the same payload and hands it to Wove.

from wove.integrations.worker import run


def acme_queue_worker(payload):
    return run(payload)

Use arun(payload) when the backend worker is already async.

from wove.integrations.worker import arun


async def async_acme_queue_worker(payload):
    return await arun(payload)

Callback Contract

The callback event is how data returns to the weave. A backend result store, queue acknowledgement, Kubernetes job status, or scheduler exit code is not enough by itself because Wove needs the task value or task error in the original dependency graph.

A custom backend integration fulfills the callback contract in one of two ways:

  • The backend worker calls run(payload) or await arun(payload), which executes the callable and posts the required events.

  • The backend worker implements the same event protocol itself and posts serialized task_started, task_result, task_error, or task_cancelled frames to payload["callback_url"].

Calling Wove’s worker entrypoint is the normal path. Implementing the protocol directly is only useful when the backend must run the task through a non-Python execution layer but can still produce Wove-compatible event frames.

Configure The Adapter

Custom adapters are selected by wrapping them in BackendAdapterEnvironmentExecutor and using that executor instance in an environment definition.

import wove
from myapp.wove_adapters import AcmeQueueAdapter
from wove import BackendAdapterEnvironmentExecutor, weave

executor = BackendAdapterEnvironmentExecutor(
    "acme_queue",
    adapter_class=AcmeQueueAdapter,
)

wove.config(
    default_environment="local",
    environments={
        "local": {"executor": "local"},
        "reports": {
            "executor": executor,
            "executor_config": {
                "url": "acme://queue.internal",
                "queue": "reports",
                "callback_url": "https://api.internal/wove/events/shared-token",
                "callback_token": "shared-token",
            },
            "delivery_timeout": 60.0,
            "delivery_orphan_policy": "requeue",
        },
    },
)

with weave() as w:
    @w.do
    def account_id():
        return "acct_123"

    @w.do(environment="reports")
    def build_report(account_id):
        return {"account_id": account_id, "status": "ready"}

Use an executor instance for project-local adapters. The built-in string names are for adapters registered by Wove itself.

Configuration Keys

These keys are handled by BackendAdapterEnvironmentExecutor before the adapter receives its config.

Key

Effect

callback_host

Local bind host for the callback receiver. Defaults to 127.0.0.1.

callback_port

Local bind port for the callback receiver. Defaults to 0, which asks the OS for an open port.

callback_url

Public callback URL embedded in submitted payloads for workers that run on another host.

callback_token

Token used in the callback URL path. Wove generates one when omitted.

The adapter also receives those keys in self.config, along with any backend-specific settings such as queue names, client objects, credentials, or scheduling options.

callback_host and callback_port control the server Wove starts. callback_url controls what the worker sees inside the payload. In container and cluster deployments, bind Wove to an address reachable from the host process and set callback_url to the route workers can call.

Contract Checklist

  • Submit the payload unchanged so the worker can call Wove’s worker entrypoint.

  • Make sure the worker either calls run(payload) or await arun(payload) or posts the same callback event frames itself.

  • Return a backend submission handle from submit(...) when cancellation or observability needs it.

  • Use frame["run_id"] or delivery_idempotency_key as the backend job id when the backend supports idempotency.

  • Keep backend package imports inside start(...) or methods so importing Wove does not require backend libraries.

  • Set required_modules and install_hint so missing dependencies fail with a useful install message.

  • Configure a reachable callback_url whenever workers are outside the submitting process’s host or network namespace.

  • Install wove[dispatch] in both the process that submits work and the worker environment that executes payloads.