Pairing
In a standard Peppy setup, a node subscribes to topics from its declared dependencies using link_id.
This creates a directed graph: if arm_controller depends on robot_arm, it can expect topics from robot_arm, but not the other way around, since a return dependency would make the graph circular.
Pairing is Peppy’s first-class mechanism for the cases where two nodes genuinely talk to each other. A pairing is a named, versioned contract with exactly two roles; two node instances (one per role) are paired 1:1 over it, and each side’s topics flow only to its paired peer:
- Explicit: a pair is established by an operator action (
--linkonpeppy node run, orlinks:in a launcher), never by discovery. - Exclusive: a pairing slot holds at most one peer at a time. A second instance trying to pair at a taken slot is rejected.
- Pair-before-traffic: pairs are established when an instance starts. While unpaired, a slot is silent: publishes go nowhere and subscriptions receive nothing.
- Unpair on death: when either instance dies, the pair dissolves automatically. The survivor keeps running with its slot unpaired until a new peer instance is started against it.
A pairing dependency is deliberately invisible to the node dependency graph: two nodes paired with each other never form a cycle, and neither requires the other to be present at build time. Pairings are topics only; see Why topics only?. For where pairing sits among the other mechanisms, see Choosing a communication pattern.
Example: robot arm control loop
Section titled “Example: robot arm control loop”Consider a robot arm that needs bidirectional communication between two nodes:
arm_controller: plans trajectories and sends joint commands.robot_arm: drives the physical joints and reports their state.
arm_controller (role: controller) robot_arm (role: arm) │ │ slot "arm" │─ emits joint_commands ───────────────▶│ slot "controller" │ │ │◀─────────────────── emits joint_states│ │ │ └─────────────── ⇌ ───────────────────┘ one pair over arm_link/v1One pairing document models both directions. Each topic declares which role emits it; the other role consumes it.
Define the pairing
Section titled “Define the pairing”A pairing is a standalone pairing/v1 document in a repository Peppy scans (see Repositories). Like a contract, the message_format lives here once and both sides inherit it by reference, but unlike a contract, a pairing names both directions and the two roles playing them.
{ peppy_schema: "pairing/v1", manifest: { name: "arm_link", tag: "v1", }, // Exactly two roles. Each paired instance plays one of them. roles: ["controller", "arm"], // One flat topic list; `emitted_by` names the role that publishes it, // and the other role consumes it. Pairings are topics-only. topics: [ { emitted_by: "controller", name: "joint_commands", qos_profile: "reliable", message_format: { target_positions: { $type: "array", $items: "f64", $length: 3 }, max_velocity: "f64", }, }, { emitted_by: "arm", name: "joint_states", qos_profile: "sensor_data", message_format: { positions: { $type: "array", $items: "f64", $length: 3 }, velocities: { $type: "array", $items: "f64", $length: 3 }, timestamp: "time", }, }, ],}The rules:
rolesdeclares exactly two distinct roles.topicsis one flat list; every topic’semitted_bymust name one of the two roles, and topic names must be unique across the whole list.- Pairings declare topics only: no services, no actions.
After peppy repo refresh, the pairing is cached and addressable by (name, tag).
Configure the nodes
Section titled “Configure the nodes”Each node declares a pairing slot under depends_on.pairings: the contract, the role this node plays, and a link_id naming the slot. The link_id is how the node’s own code and the pairing commands refer to the other end; while paired, exactly one peer instance sits behind it.
The slot says which conversation the node joins. interfaces.topics says what the node uses through it, exactly like a contract implementer and a contract consumer. A pairing-backed entry is just { link_id, name }: the pairing document remains the single source of shape and QoS, so an inline message_format or qos_profile on one of these entries is rejected.
{ peppy_schema: "node/v1", manifest: { name: "robot_arm", tag: "v1", depends_on: { pairings: [ // One pairing slot: this node plays the `arm` role of arm_link/v1. // The slot's link_id names the OTHER end from this node's point of // view; while paired, exactly one controller instance sits behind it. // `optional: true` says this node runs fine with nobody commanding it, // so a deployment may write the slot vacant; without it, every // deployment must pair the slot. { name: "arm_link", tag: "v1", role: "arm", link_id: "controller", optional: true }, ], }, }, interfaces: { topics: { // The `arm` role's side of the conversation, declared the same way a // contract implementer declares its contract members. The pairing // document stays the source of shape and QoS; these entries only say // which of its topics this node uses. emits: [ { link_id: "controller", name: "joint_states" }, ], consumes: [ { link_id: "controller", name: "joint_commands" }, ], }, }, execution: { language: "python", build_cmd: ["uv", "sync"], run_cmd: ["uv", "run", "robot_arm"], },}{ peppy_schema: "node/v1", manifest: { name: "robot_arm", tag: "v1", depends_on: { pairings: [ // One pairing slot: this node plays the `arm` role of arm_link/v1. // The slot's link_id names the OTHER end from this node's point of // view; while paired, exactly one controller instance sits behind it. // `optional: true` says this node runs fine with nobody commanding it, // so a deployment may write the slot vacant; without it, every // deployment must pair the slot. { name: "arm_link", tag: "v1", role: "arm", link_id: "controller", optional: true }, ], }, }, interfaces: { topics: { // The `arm` role's side of the conversation, declared the same way a // contract implementer declares its contract members. The pairing // document stays the source of shape and QoS; these entries only say // which of its topics this node uses. emits: [ { link_id: "controller", name: "joint_states" }, ], consumes: [ { link_id: "controller", name: "joint_commands" }, ], }, }, execution: { language: "rust", build_cmd: ["cargo", "build", "--release"], run_cmd: ["./target/release/robot_arm"], },}{ peppy_schema: "node/v1", manifest: { name: "arm_controller", tag: "v1", depends_on: { pairings: [ // The complementary slot: this node plays the `controller` role of // the same contract, and its slot points at one arm. `optional: true` // says it also runs with no arm behind the slot, so a deployment may // write the slot vacant. { name: "arm_link", tag: "v1", role: "controller", link_id: "arm", optional: true }, ], }, }, interfaces: { topics: { // The mirror image of the arm's block: this role emits the commands and // consumes the states. emits: [ { link_id: "arm", name: "joint_commands" }, ], consumes: [ { link_id: "arm", name: "joint_states" }, ], }, }, execution: { language: "python", build_cmd: ["uv", "sync"], run_cmd: ["uv", "run", "arm_controller"], },}{ peppy_schema: "node/v1", manifest: { name: "arm_controller", tag: "v1", depends_on: { pairings: [ // The complementary slot: this node plays the `controller` role of // the same contract, and its slot points at one arm. `optional: true` // says it also runs with no arm behind the slot, so a deployment may // write the slot vacant. { name: "arm_link", tag: "v1", role: "controller", link_id: "arm", optional: true }, ], }, }, interfaces: { topics: { // The mirror image of the arm's block: this role emits the commands and // consumes the states. emits: [ { link_id: "arm", name: "joint_commands" }, ], consumes: [ { link_id: "arm", name: "joint_states" }, ], }, }, execution: { language: "rust", build_cmd: ["cargo", "build", "--release"], run_cmd: ["./target/release/arm_controller"], },}The two manifests are mirror images, which is what makes the conversation legible from either side alone:
| Node | Role in arm_link | Slot (link_id) | Emits to the peer | Consumes from the peer |
|---|---|---|---|---|
robot_arm | arm | controller | joint_states | joint_commands |
arm_controller | controller | arm | joint_commands | joint_states |
Coverage: emit side complete, consume side free
Section titled “Coverage: emit side complete, consume side free”The two directions are checked differently, and for the same reasons contracts are:
emitsmust be exact. The entries for a slot are precisely the document’s topics whoseemitted_byis the slot’s role: no missing entry, no unknown name, no duplicate, and never a topic the counterpart role emits. A role that silently stopped emitting one of its topics would break its peer, and the pairing document is the promise that it will not.consumesmay be partial. List any subset of the counterpart role’s topics. Omitting one is a legitimate way to ignore part of a pairing: no module is generated for that topic, so no subscription can be created for it. This is a local decision with no effect on the peer. Declaring zero consumes is fine, including for a slot whose counterpart role emits nothing.
Naming a topic under consumes that this node’s own role emits is an error, as is naming a topic that is absent from the document. Both are reported as one aggregated diff per slot, so a manifest with several wrong entries produces one readable report rather than a cascade.
Every participant slot must be covered: starting the instance without pairing the slot, or without declaring it vacant, fails loudly. A deployment that runs the node with a slot unpaired says so with --vacant-link 'SLOT=<why>' / links: { <slot>: { vacant: "<why>" } }, and the reason is required, so an unpaired slot is always something the deployment wrote down, with its rationale, rather than something it forgot.
Whether a slot may be vacated at all is the node’s own call, made once in its manifest with optional: true on the depends_on.pairings entry. That is the deployment-independent half of the statement: a reader of the node can tell which of its slots run without a peer without sweeping every launcher, and a slot that must never boot unpaired cannot be waived by any deployment. The manifest says “may be empty”, the deployment says “is empty here, because”, and a slot nobody mentioned is uncovered either way. A vacancy on a required slot is refused with the manifest key that would lift the bar.
Coverage is a launch-time question and has no bearing on what the node declares through the slot: a slot that boots vacant is interface-checked exactly like a paired one.
Establishing pairs
Section titled “Establishing pairs”Pairs are established when an instance starts; there is no runtime “pair” command. The instance being started names the peer it pairs with; the peer must already be running (or starting) with a complementary unpaired slot.
peppy repo refreshpeppy node add -s ./robot_arm # sync interfaces, then add to the stackpeppy node add -s ./arm_controller
# The first instance has no peer yet. Its manifest declares the slot# `optional: true`, so this deployment may declare it vacant; it boots with# the slot silent.peppy node run --instance-id arm_1 robot_arm:v1 \ --vacant-link 'controller=the commander starts after this arm'
# The second instance pairs its `arm` slot with arm_1 at start.peppy node run --instance-id ctrl_1 arm_controller:v1 --link arm@arm_1The --link value is LINK_ID@PEER_INSTANCE[/PEER_LINK]: your slot, the running peer instance, and (only when the peer declares several complementary slots of the same pairing) which of the peer’s slots to claim. The --vacant-link value is SLOT=<why>, split on the first =, so the reason may itself contain = and needs no escaping. Running a node whose slot is neither --linked nor --vacant-linked is a hard error naming the missing slot, and the remedies it offers follow the manifest. An optional slot’s error names both:
$ peppy node run arm_controller:v1Error: instance `brave-lovelace-4821` declares optional pairing slot `arm` (pairing `arm_link:v1`,role `controller`) with no pair. Pair it (launcher `links: { arm: "<peer_instance>" }` /`--link arm@<peer_instance>`), or leave it empty on purpose(`links: { arm: { vacant: "<why>" } }` / `--vacant-link 'arm=<why>'`)A required slot’s error names only the pairing, and points at the manifest for the rest, so it never advertises an escape hatch the node forbids:
Error: instance `brave-lovelace-4821` declares required pairing slot `arm` (pairing `arm_link:v1`,role `controller`) with no pair. Pair it (launcher `links: { arm: "<peer_instance>" }` /`--link arm@<peer_instance>`), or declare the slot `optional: true` in the node manifest if it ismeant to run without a peerIn a launcher, the pair is declared once, on either instance, with links:; a slot left intentionally unpaired takes a { vacant: "<why>" } value in that same map:
{ peppy_schema: "launcher/v1", deployments: [ { source: { name: "robot_arm:v1" }, instances: [{ instance_id: "arm_1" }], }, { source: { name: "arm_controller:v1" }, instances: [{ instance_id: "ctrl_1", // My `arm` slot pairs with the arm_1 instance. Declaring the same // pair from arm_1's side instead (or as well) is equivalent. links: { arm: "arm_1" }, }], }, ],}peppy stack launch validates the whole pair plan up front (coverage, complementary roles, exclusivity, ambiguity) and establishes each pair as its second endpoint comes up.
peppy stack list shows every pairing participant slot, paired or not, with a bidirectional ⇌ arrow (observer slots are not listed there):
Instance pairings
NODE INSTANCE PAIRINGSrobot_arm:v1 arm_1 controller ⇌ ctrl_1:arm@cn-adoring-wiles (arm_link:v1)arm_controller:v1 ctrl_1 arm ⇌ arm_1:controller@cn-adoring-wiles (arm_link:v1)Using the generated API
Section titled “Using the generated API”peppy node sync generates a module per declared slot topic under peppygen.paired_topics.<link_id>.<topic> (Python) / peppygen::paired_topics::<link_id>::<topic> (Rust); both directions of a slot live under the same link_id. The modules follow the manifest’s entries, not the document’s full topic list, so a counterpart topic you left out of consumes simply has no module. Pairing topics never appear under consumed_topics or emitted_topics, whichever direction they travel. Emitting and consuming look exactly like ordinary topics, plus two pin-state helpers on every module: paired() returns the current peer’s identity (or None/None while unpaired), and wait_paired() awaits one.
import asyncioimport sysimport time
from peppygen import NodeBuilder, NodeRunnerfrom peppygen.parameters import Parametersfrom peppygen.paired_topics.controller import joint_commands, joint_states
# `robot_arm` plays the `arm` role of the `arm_link` pairing. Both# directions of its `controller` slot live under# `peppygen.paired_topics.controller`: it consumes `joint_commands` from and emits# `joint_states` to whichever single controller instance is currently# paired on the slot. Unpaired, the subscription stays silent and# publishes go nowhere; the code does not change either way.
async def handle_commands(node_runner: NodeRunner): # Declare the publisher once, then publish each state on it. try: publisher = await joint_states.declare_publisher(node_runner) except Exception as e: print(f"Failed to declare joint_states publisher: {e}", file=sys.stderr) return
# Subscribing while unpaired is legal: the held subscription yields # nothing until a controller pairs, then only that controller's messages. try: subscription = await joint_commands.subscribe(node_runner) except Exception as e: print(f"Failed to subscribe to joint_commands: {e}", file=sys.stderr) return
# Optional: block until a controller is paired and log who it is. try: peer = await joint_commands.wait_paired(node_runner) print(f"paired with controller {peer.producer.core_node}/{peer.producer.instance_id}") except Exception as e: print(f"Failed to wait for a paired controller: {e}", file=sys.stderr) return
while True: try: received = await subscription.next() except Exception as e: # Log the failure, then pause before retrying so a persistent # receive error does not spin the loop at full speed. print(f"Error receiving joint command: {e}", file=sys.stderr) await asyncio.sleep(1.0) continue
if received is None: break # subscription closed peer, command = received
# `peer` is always the paired controller's identity. print( f"command from {peer.producer.core_node}/{peer.producer.instance_id}: " f"target={command.target_positions} max_vel={command.max_velocity}" )
# Drive the joints, then report the resulting state back to the # paired controller. try: await publisher.publish( joint_states.build_message( command.target_positions, [0.0, 0.0, 0.0], time.time(), ) ) except Exception as e: print(f"Failed to publish joint state: {e}", file=sys.stderr)
async def setup(_params: Parameters, node_runner: NodeRunner) -> list[asyncio.Task]: return [asyncio.create_task(handle_commands(node_runner))]
def main(): NodeBuilder().run(setup)
if __name__ == "__main__": main()use peppygen::paired_topics::controller::{joint_commands, joint_states};use peppygen::{NodeBuilder, Parameters, Result};
// `robot_arm` plays the `arm` role of the `arm_link` pairing. Both// directions of its `controller` slot live under// `peppygen::paired_topics::controller`: it consumes `joint_commands` from and// emits `joint_states` to whichever single controller instance is// currently paired on the slot. Unpaired, the subscription stays silent// and publishes go nowhere; the code does not change either way.fn main() -> Result<()> { NodeBuilder::new().run(|_args: Parameters, node_runner| async move { tokio::spawn(async move { // Declare the publisher once; every publish below is then lock-free. let publisher = match joint_states::declare_publisher(&node_runner).await { Ok(publisher) => publisher, Err(e) => { eprintln!("Failed to declare joint_states publisher: {e}"); return; } }; // Subscribing while unpaired is legal: the held subscription // yields nothing until a controller pairs, then only that // controller's messages. let mut subscription = match joint_commands::subscribe(&node_runner).await { Ok(subscription) => subscription, Err(e) => { eprintln!("Failed to subscribe to joint_commands: {e}"); return; } };
// Optional: block until a controller is paired and log who it is. match joint_commands::wait_paired(&node_runner).await { Ok(peer) => println!( "paired with controller {}/{}", peer.producer.core_node, peer.producer.instance_id ), Err(e) => { eprintln!("Failed to wait for a paired controller: {e}"); return; } }
loop { let (peer, command) = match subscription.next().await { Ok(Some(received)) => received, Ok(None) => break, Err(e) => { eprintln!("Error receiving joint command: {e}"); continue; } };
// `peer` is always the paired controller's identity. println!( "command from {}/{}: target={:?} max_vel={}", peer.producer.core_node, peer.producer.instance_id, command.target_positions, command.max_velocity );
// Drive the joints, then report the resulting state back to // the paired controller. match joint_states::build_message( command.target_positions, [0.0, 0.0, 0.0], std::time::SystemTime::now(), ) { Ok(payload) => { if let Err(e) = publisher.publish(payload).await { eprintln!("Failed to publish joint state: {e}"); } } Err(e) => eprintln!("Failed to build joint_states message: {e}"), } } });
Ok(()) })}On the other side, arm_controller emits joint_commands and consumes joint_states through its arm slot:
import asyncioimport sys
from peppygen import NodeBuilder, NodeRunnerfrom peppygen.parameters import Parametersfrom peppygen.paired_topics.arm import joint_commands, joint_states
# `arm_controller` plays the `controller` role of the `arm_link` pairing.# Both directions of its `arm` slot live under `peppygen.paired_topics.arm`: it# emits `joint_commands` to and consumes `joint_states` from the single# arm instance currently paired on the slot. If that arm dies, the slot# unpairs and the loop simply stops receiving until a new arm is paired.
def compute_next_target(current: list[float]) -> list[float]: # Trajectory planning logic return [current[0] + 0.1, current[1], current[2]]
async def control_loop(node_runner: NodeRunner): # Declare the publisher once, then publish each command on it. try: publisher = await joint_commands.declare_publisher(node_runner) except Exception as e: print(f"Failed to declare joint_commands publisher: {e}", file=sys.stderr) return
# Subscribing while unpaired is legal: the subscription follows the # slot's live pin, silent until an arm is paired. try: subscription = await joint_states.subscribe(node_runner) except Exception as e: print(f"Failed to subscribe to joint_states: {e}", file=sys.stderr) return
# Optional: block until an arm is paired and log who it is. try: peer = await joint_states.wait_paired(node_runner) print(f"paired with arm {peer.producer.core_node}/{peer.producer.instance_id}") except Exception as e: print(f"Failed to wait for a paired arm: {e}", file=sys.stderr) return
while True: try: received = await subscription.next() except Exception as e: # Log the failure, then pause before retrying so a persistent # receive error does not spin the loop at full speed. print(f"Error receiving joint state: {e}", file=sys.stderr) await asyncio.sleep(1.0) continue
if received is None: break # subscription closed peer, state = received
# `peer` is always the paired arm's identity. print( f"state from {peer.producer.core_node}/{peer.producer.instance_id}: " f"positions={state.positions}" )
# Compute the next target from the reported state, then command it. target = compute_next_target(state.positions) try: await publisher.publish( joint_commands.build_message( target, 1.0, # max_velocity ) ) except Exception as e: print(f"Failed to publish joint command: {e}", file=sys.stderr)
async def setup(_params: Parameters, node_runner: NodeRunner) -> list[asyncio.Task]: return [asyncio.create_task(control_loop(node_runner))]
def main(): NodeBuilder().run(setup)
if __name__ == "__main__": main()use peppygen::paired_topics::arm::{joint_commands, joint_states};use peppygen::{NodeBuilder, Parameters, Result};
// `arm_controller` plays the `controller` role of the `arm_link` pairing.// Both directions of its `arm` slot live under `peppygen::paired_topics::arm`: it// emits `joint_commands` to and consumes `joint_states` from the single// arm instance currently paired on the slot. If that arm dies, the slot// unpairs and the loop simply stops receiving until a new arm is paired.fn main() -> Result<()> { NodeBuilder::new().run(|_args: Parameters, node_runner| async move { tokio::spawn(async move { // Declare the publisher once; every publish below is then lock-free. let publisher = match joint_commands::declare_publisher(&node_runner).await { Ok(publisher) => publisher, Err(e) => { eprintln!("Failed to declare joint_commands publisher: {e}"); return; } }; // Subscribing while unpaired is legal: the subscription follows // the slot's live pin, silent until an arm is paired. let mut subscription = match joint_states::subscribe(&node_runner).await { Ok(subscription) => subscription, Err(e) => { eprintln!("Failed to subscribe to joint_states: {e}"); return; } };
// Optional: block until an arm is paired and log who it is. match joint_states::wait_paired(&node_runner).await { Ok(peer) => println!( "paired with arm {}/{}", peer.producer.core_node, peer.producer.instance_id ), Err(e) => { eprintln!("Failed to wait for a paired arm: {e}"); return; } }
loop { let (peer, state) = match subscription.next().await { Ok(Some(received)) => received, Ok(None) => break, Err(e) => { eprintln!("Error receiving joint state: {e}"); continue; } };
// `peer` is always the paired arm's identity. println!( "state from {}/{}: positions={:?}", peer.producer.core_node, peer.producer.instance_id, state.positions );
// Compute the next target from the reported state, then command it. let target = compute_next_target(&state.positions); match joint_commands::build_message(target, 1.0) { Ok(payload) => { if let Err(e) = publisher.publish(payload).await { eprintln!("Failed to publish joint command: {e}"); } } Err(e) => eprintln!("Failed to build joint_commands message: {e}"), } } });
Ok(()) })}
fn compute_next_target(current: &[f64; 3]) -> [f64; 3] { // Trajectory planning logic [current[0] + 0.1, current[1], current[2]]}The key behavioral guarantees, all handled by the runtime with no application code:
- Silent while unpaired. Publishing on an unpaired slot is a legal no-op, and a subscription on an unpaired slot just waits. Nodes never need pairing-state conditionals around their control loops.
- Only the paired peer. A subscription delivers messages from the paired peer instance and nothing else: not from other instances of the same node, not from implementing third parties. The slot is the identity; there is no payload-level demultiplexing (
arm_idfields and the like) to write. - A pairing is a live stream, not a mailbox. Messages published before the pair was established are never delivered.
- Re-pinning is seamless. When the daemon re-pairs a surviving slot to a new peer, the held subscription switches over internally; no stale messages from the old peer leak through.
Lifecycle: death, failover, exclusivity
Section titled “Lifecycle: death, failover, exclusivity”A pair lives exactly as long as both endpoints. When an instance dies (crashes, is stopped with peppy node stop, or is torn down by a stack operation), the daemon dissolves its pairs and live-notifies each survivor that its slot is now unpaired. The survivor keeps running; its slot goes silent.
Failover is then just another --link at start:
# ctrl_1 died (or was stopped). arm_1 keeps running, slot unpaired.peppy node run --instance-id ctrl_2 arm_controller:v1 --link arm@arm_1ctrl_2 pairs with the surviving arm_1, whose subscription and publisher switch to the new peer live; the arm’s code never restarts and never notices beyond paired() reporting a new identity.
Exclusivity is enforced at establishment: while arm_1’s slot is paired, a third instance running with --link arm@arm_1 is rejected naming the existing pair. To hand an arm to a different controller, stop the old controller first (or run the new one against a different arm).
Multiple slots: the two-arm commander
Section titled “Multiple slots: the two-arm commander”Because the slot, not the node, is the unit of pairing, a node can declare several slots of the same pairing and hold one peer per slot, each a fully isolated stream.
Each slot is declared and covered on its own, so the two arms stay independent all the way down to the generated modules:
// two_arm_commander/peppy.json5 (manifest excerpt)depends_on: { pairings: [ { name: "arm_link", tag: "v1", role: "controller", link_id: "left_arm" }, { name: "arm_link", tag: "v1", role: "controller", link_id: "right_arm" }, ],},interfaces: { topics: { // Emit coverage is per slot, so the controller role's `joint_commands` // is listed once for each arm. emits: [ { link_id: "left_arm", name: "joint_commands" }, { link_id: "right_arm", name: "joint_commands" }, ], consumes: [ { link_id: "left_arm", name: "joint_states" }, { link_id: "right_arm", name: "joint_states" }, ], },},peppy node add -s ./two_arm_commander
peppy node run --instance-id arm_l robot_arm:v1 \ --vacant-link 'controller=the commander starts after both arms'peppy node run --instance-id arm_r robot_arm:v1 \ --vacant-link 'controller=the commander starts after both arms'peppy node run --instance-id cmd_1 two_arm_commander:v1 \ --link left_arm@arm_l --link right_arm@arm_rThe generated code addresses each arm through its slot module (paired_topics.left_arm.joint_commands vs paired_topics.right_arm.joint_commands in Python, paired_topics::left_arm::joint_commands vs paired_topics::right_arm::joint_commands in Rust), and each subscription receives only its own arm’s states. When a peer instance declares several complementary slots (pairing two commanders’ arms to each other, say), disambiguate the target with --link left_arm@cmd_2/right_arm.
Observers: watching a pairing without joining it
Section titled “Observers: watching a pairing without joining it”A pair is exclusive between two instances, but sometimes a third node just wants to watch one side’s stream: a recorder logging every joint_states an arm emits, a dashboard tailing a controller’s setpoints. Joining the pair is wrong (it is 1:1, and the watcher has nothing to send back), and a separate contract would duplicate the topic shape. An observer slot solves this: it passively taps the topics a chosen role emits, following that source instance’s own lifecycle rather than any pairing.
Declare an observer slot in depends_on.pairing_observers, where role names the role being observed rather than one this node plays, and list the topics it taps in interfaces.topics.consumes under its link_id. An observer plays no role, claims no endpoint, and emits nothing, so it never needs coverage on an emit side:
// recorder/peppy.json5 (manifest excerpt)depends_on: { pairing_observers: [ // Watch whatever the `arm` role of arm_link/v1 publishes. { name: "arm_link", tag: "v1", role: "arm", link_id: "watch" } ]},interfaces: { topics: { // The `arm` role emits joint_states; tap it through the `watch` slot. consumes: [{ link_id: "watch", name: "joint_states" }] }}Link the observer to a source instance the same way you establish a pairing, with --link (or a launcher links: entry); the target is the instance to observe, not a peer to pair with:
peppy node add -s ./recorder
# Observe arm_1, whichever controller it happens to be paired with (or none).peppy node run --instance-id rec_1 recorder:v1 --link watch@arm_1The runtime guarantees mirror pairing’s, with the lifecycle twist that defines an observer:
- Follows the source, not a pair. The observer receives
arm_1’s emissions whether or notarm_1is paired to anyone, and it is unaffected whenarm_1re-pairs to a different controller. It is pinned to the source instance itself. - Many observers, one source. Observation is not exclusive: any number of recorders can watch the same arm at once, and none of them affects the pair.
- Incarnation-isolated. If the source instance restarts, the observer transparently drops the old stream and picks up the new incarnation: the drop discards everything its subscription buffered before the switch, so no buffered message from a previous run of the source leaks into the new stream. The switch happens when the daemon’s lifecycle notify records the new incarnation and delivers it, which follows the restart by the notify latency; isolation is enforced at that boundary, not by wall-clock simultaneity with the restart.
- Read-only. The generated module for an observer topic exposes the same subscription surface as a consumed topic, plus a helper reporting the pairings the slot observes. There is no publisher and no
paired(); an observer never talks back. - Not shown in
peppy stack list. TheInstance pairingstable renders participant slots only; inspect an observer’s links through its generated source helper instead.
The generated modules live alongside participant slots under peppygen.paired_topics.<link_id>.<topic> (Python) / peppygen::paired_topics::<link_id>::<topic> (Rust).
Observing several pairings through one slot
Section titled “Observing several pairings through one slot”An observer slot carries a cardinality the way a producer-binding slot does, so one slot can watch a whole fleet:
cardinality | Launcher links: value | Omitting the slot |
|---|---|---|
one (the default) | a single target, watch: "arm_1" | rejected: it observes exactly one pairing, so it takes a source |
zero_or_one | a single target, or { vacant: "<why>" } | rejected: link it or declare it vacant |
one_or_more | an array, watch: ["arm_1", "arm_2"] | rejected: it has no empty state, so it takes at least one source |
zero_or_more | an array, possibly empty | valid, and means the empty set |
Among observer slots vacancy is only for the zero_or_one row, which is the observer counterpart of a participant slot’s optional: true: the node declaring it is the node saying it runs fine watching nothing. A one slot observes exactly one pairing, a one_or_more slot has no empty state to write, and a zero_or_more slot already writes its empty set as []. A producer-binding slot reads the same key the same way (see dependency cardinality). Only zero_or_more may be omitted, because it is the only one whose absence has no failure mode; every other slot needs an explicit line, which is what keeps a slot you forgot distinguishable from one you emptied on purpose.
On the CLI the array becomes repetition: --link watch@arm_1 --link watch@arm_2. Either way the order you write is the order the node reads back, so a deployment can associate the Nth observed pairing with its own Nth command slot.
The declared cardinality types the generated accessor, name and return type together, so flipping it surfaces at every call site rather than silently reading one member of many, or an empty set the plan said could not happen. A one slot exposes source() returning the observed pairing directly, with no absent case to handle. A zero_or_one slot exposes the same singular source() returning Optional[ObservedSource] in Python and Option<ObservedSource> in Rust, empty wherever the deployment wrote the slot vacant. A one_or_more slot exposes sources() returning a set that is never empty, so its head needs no unwrap: in Rust that is a NonEmptyObservedSources whose first() is infallible, and in Python a List documented never-empty, since Python has no non-empty list type and the name flip is what surfaces the change. A zero_or_more slot exposes sources() returning a plain, possibly empty list in plan order, so the empty branch exists exactly where a deployment can produce one.
subscribe keeps its shape at every cardinality and fans in across the whole set, tagging each message with the ObservedSource that published it: the producer’s wire address plus the producer-side link_id, the same identity the accessors enumerate, so members stay distinct even when several observe the same instance.
from peppygen.paired_topics.watch import joint_states
# `watch` is one_or_more: read the whole observed set, in plan order. The# set is never empty at that cardinality, so `[0]` is always valid.sources = joint_states.sources(node_runner)print("lead arm:", sources[0].producer.instance_id)for source in sources: print(source.producer.instance_id, source.source_link_id)
# One subscription covers every member; the source tag says which pairing# sent what, even when members share one instance.subscription = await joint_states.subscribe(node_runner)async for source, states in subscription: print(source.producer.instance_id, source.source_link_id, states.positions)use peppygen::paired_topics::watch::joint_states;
// `watch` is one_or_more: read the whole observed set, in plan order. The// set is never empty at that cardinality, so `first()` needs no unwrap.let sources = joint_states::sources(&node_runner)?;println!("lead arm: {}", sources.first().producer.instance_id);for source in &sources { println!("{} {}", source.producer.instance_id, source.source_link_id);}
// One subscription covers every member; the source tag says which pairing// sent what, even when members share one instance.let mut subscription = joint_states::subscribe(&node_runner).await?;while let Some((source, states)) = subscription.next().await? { println!( "{} {} {:?}", source.producer.instance_id, source.source_link_id, states.positions );}An observed set is seeded at spawn and live after. The boot config the daemon hands the process carries the planned member set of every slot the deployment wrote, stamped with each source’s incarnation and liveness at that moment, so sources() answers the deployment’s membership from the node’s first instruction, setup included: a node can discover the robot’s shape before it starts serving. After that the set is live, which is the one way it differs from a bound producer set: the daemon replaces it whole whenever the plan’s observed pairings change, so sources() can return a different set between reads. A member whose source is down (or has not started yet) stays in the set, at its position, so the slot’s shape does not flicker with the fleet’s health. A zero_or_more slot the deployment omitted is the one slot neither boundary touches: it carries no seed and receives no delivery, and reads the empty set throughout.
What is live is what the set says about each member, not how many members it has. The launcher sizes a slot at plan time and the node re-checks that size against its seed before setup runs, so each cardinality’s floor holds on every read: only zero_or_one and zero_or_more ever read empty, and an empty read there means the plan observes nothing through that slot. That is what lets the accessor be typed instead of handing every caller a set to re-check.
Why topics only?
Section titled “Why topics only?”Peppy has three communication patterns (topics, services, and actions), but a pairing may declare topics only. The reason is the distinction that shapes the rest of the dependency model: a topic dependency is passive, while service and action dependencies are caller-driven.
| Topics | Services | Actions | |
|---|---|---|---|
| Pattern | Publish-subscribe | Request-response | Goal-feedback-result |
| Runtime data flow | One-way: producer → consumer | Two-way: consumer calls provider, provider replies | Multi-step: consumer drives the provider’s lifecycle |
| Consuming means | “I passively receive these messages” | “I actively call this provider” | “I actively orchestrate this provider” |
| Allowed in a pairing? | Yes | No | No |
A pairing’s two directions are two independent one-way streams: each side publishes whenever it has data, and neither ever blocks on the other at the protocol level. That is what makes the mutual relationship safe to hide from the static dependency graph. Services and actions are caller-driven: each side would actively invoke the other, forming a real request-response cycle at runtime, which is exactly the deadlock hazard the acyclic dependency model exists to prevent. The same rule applies to contract implementation: mutual service or action relationships are rejected, however they are wired.
If two paired nodes also need a bounded request-response exchange, keep the continuous streams in the pairing and model the bounded job as an action in one direction (the client depends on the server, no cycle).
Pairing vs. contracts
Section titled “Pairing vs. contracts”Pairing and contract implementation both decouple nodes from each other through a shared contract. They answer different questions:
| Pairing | Contract | |
|---|---|---|
| Declaration model | The node enumerates what it uses in interfaces.topics; the document owns shape and QoS | The node enumerates what it uses in interfaces; the document owns shape and QoS |
| Coverage | Produced side (emits) exact per slot, consumed side free | Produced side (implements) exact per slot, consumed side free |
| Cardinality | Exactly 1:1 per slot, exclusive (no cardinality key); optional: true says the slot may also run with no peer | Declared per slot: exactly one by default, at most one for zero_or_one, an application-selected set for one_or_more / zero_or_more |
| Directionality | Both directions in one contract (two roles) | One direction per contract |
| Establishment | Declared once on either side (--link arm@peer), covering both endpoints, at instance start | Each consumer slot bound to its producer(s) (--link slot@producer), at instance start |
| Peer identity | The slot is the identity; the runtime guarantees whose messages you get, and each message rides with the peer’s PeerInfo | Per-message ProducerRef; the consumer tells producers apart itself |
| Lifecycle | Pair dissolves on death; survivor’s slot goes silent until re-paired | Producers come and go freely |
| Natural fit | Control loops, teleoperation, any “these two specific instances belong together” relationship | Telemetry, monitoring, swappable sources (dedicated slots or one multi-cardinality slot) |
Use a pairing when the relationship is exclusive and both directions belong to one conversation, like a controller and its arm. Use contracts when several loosely-coupled producers should feed a consumer, like every arm on the floor publishing diagnostics to one observer (one one_or_more slot bound to all arms, or one slot per arm when each has a distinct role). The two compose: robot_arm can be paired with its controller and implement a one-way joint_state_source contract so dashboards can watch it.
Pairing vs. actions
Section titled “Pairing vs. actions”Pairing and actions can both look like “two nodes talking back and forth”, but they model different shapes of interaction. The distinction is lifecycle: does the exchange have a defined start and end, or does it run continuously for as long as the nodes are up?
| Pairing | Actions | |
|---|---|---|
| Lifecycle | Continuous; runs for as long as the pair is live | Bounded; each goal has an explicit start (goal accepted) and end (result delivered, cancelled, or errored) |
| Initiator | Either side publishes whenever it has data | Client issues a goal; server responds |
| Concurrency | Both streams flow in parallel, indefinitely | Concurrent goals allowed; the server’s goal handler sets the acceptance policy |
| Per-exchange feedback | Implicit (each side publishes its own stream) | Explicit feedback channel during the goal |
| Cancellation | Unpair (stop an endpoint); or just stop publishing | First-class cancel request mid-flight |
| Natural fit | Inner control loops, continuous state mirroring | Discrete jobs with a clear “done” condition |
A useful test: if you find yourself wanting to say “the controller asks the arm to do X and waits until it’s done”, that is an action. If you find yourself saying “the controller streams setpoints at 100 Hz and the arm streams state back at 1 kHz”, that is a pairing.
Both shapes can coexist on the same pair of nodes. A robot_arm might expose a calibrate action (run once at startup, has a definite end) and exchange joint_commands / joint_states continuously with its paired controller.