Data Movement Through Nodes (Payload & JSON)

How data flows through a Rayven workflow using JSON payloads

Overview

In Rayven, all data passing through a workflow moves from node to node in the form of a JSON payload. This structure allows workflows to be highly modular and flexible, with each node able to inspect, modify, append to, or filter the data it receives. Understanding how data moves and is transformed throughout a workflow is essential for effectively designing logic, displaying widgets, and writing to tables.

This article explains how JSON payloads flow through nodes, how to inspect data at any point, and the key things to consider when building workflows that process dynamic or structured data.

JSON Payloads: The Core Structure

Every time data enters a workflow—whether via a connector node, a trigger, or a frontend interaction—it is encapsulated in a JSON object known as a payload. This payload can contain any number of key-value pairs, including raw values, arrays, nested objects, or structured metadata.

Each node in the workflow:

• Receives the JSON payload via its input

• Processes the payload according to its configuration

• Outputs a new JSON payload to the next node(s) downstream

If a node generates new data (e.g. performs a calculation or transformation), that data is appended to the existing payload and passed forward.

Example JSON Payload

As this payload flows through the workflow, nodes may add fields like:

• "isOverThreshold": false

• "alertMessage": "Temperature is within normal range"

How Nodes Interact with Payloads

Transformation & Logic Nodes (e.g. JavaScript, Rule Builder, Data Summary, Advanced Function):

• Add new fields to the payload or modify existing ones

• Used for applying logic, making calculations, or aggregating values across UIDs or label groups

• Examples:

  • JavaScript Node: Use custom code to manipulate payloads dynamically. Ideal for complex logic not supported by other nodes. 

  • Rule Builder: Apply conditional logic using a visual interface. Based on rule evaluation, the node can assign outputs directly into the payload.

  • Data Summary: Aggregate or summarise values across UIDs or label groups—e.g. sum, average, min/max. Outputs are appended to the payload.

  • Advanced Function Node: Offers a wide range of built-in formulas and functions (maths, time, custom table lookups) that operate on payload data. It’s ideal for applying numeric transformations or derived calculations.

Filter Nodes (e.g. Conditional Filter):

• Evaluate the payload against defined conditions

• Do not alter the payload, but determine whether it should proceed

• Often used for branching workflows or preventing further execution

Table Nodes (Query / Update):

• Query Table nodes append matching table data into the payload using filter conditions

• Update Table nodes extract data from the payload and write it into structured tables

Frontend Nodes:

• Require data to be passed into them via the payload

• Visualise specific fields or calculated results as widgets in the application interface

• Many Frontend nodes (especially chart and summary types) also perform calculations for the purpose of visualisation and append those results to the outgoing payload

Connector & Output Nodes:

• Ingest external JSON payloads or export processed payloads to third-party systems

• May initiate a workflow (Trigger/Connector) or terminate one (Output to Email/API/etc.)

AI Nodes:

• Use values from the payload to form a prompt or context

• Output the AI response as an unstructured string in the payload

Inspecting Payloads

Each node includes an Inspect Data tab where you can:

• View the incoming payload

• View the outgoing payload (after the node has processed it)

• Filter by UID, date range, or text content

This is the main way to debug workflows and understand how your data is being transformed step by step.

Best Practices

• Always inspect payloads during development to ensure your logic is behaving as expected.

• Use meaningful keys in your payload structure—this makes it easier to reference values in later nodes. 

• Only store necessary fields in tables or in time series—unnecessary fields increase storage volume.

Summary

Rayven workflows process and pass data through nodes using structured JSON payloads. Each node can inspect, modify, and extend these payloads, enabling powerful, flexible logic chains that handle dynamic or structured data. Understanding how payloads behave in a workflow is essential for both logic design and front-end visualisation.

Use the Inspect Data tabs on each node to monitor your data in real time and debug effectively.

FAQs

What format is data in when it moves through a workflow?
All data is encapsulated in a JSON payload as it moves from node to node.

Does each node overwrite or add to the payload?
Nodes append new values to the existing payload unless specifically configured to overwrite.

Can I see what data was passed through a node?
Yes. Use the "Inspect Data" tab on any node to see the latest incoming and outgoing payloads.

What happens if I add a node after the workflow has been running?
The new node will only start receiving data from the moment it is added. Historical data is not automatically populated into the node. However, you can manually stream historical data into the node if needed to backfill it.

Can I pass dynamic values between nodes?

Yes. Data flows between nodes via the JSON payload. Each node receives the payload from the previous node, and can read, modify, or append to it. This is how values are passed and updated dynamically throughout a workflow.

Is data persisted after passing through a node?

Yes. All data that flows through a workflow is automatically stored in Rayven’s Cassandra database, including the full JSON payload and the output from each node. This provides a complete time series history of data flowing through the workflow.

You can also explicitly store selected data in structured tables using nodes like Update Table, but this is optional and depends on your application’s needs.

Do I need to format payloads a certain way for widgets?
No. Frontend nodes let you specify which keys in the incoming JSON payload to visualise, so the structure does not need to follow a strict format. However, maintaining a flat and consistent payload structure with clear key names will make configuration and debugging much easier.