12. Data Storage & Access
      Back to home
      1. Home
      2. 12. Data Storage & Access

      Understanding How Storage Works in Rayven

      How Rayven stores structured and unstructured data across Tables and Workflows.

      Overview

      Rayven uses a hybrid storage approach that combines structured database tables with unstructured time series storage. This design allows you to build flexible, high-performance applications that support both traditional business records and real-time or semi-structured data. Understanding how storage works in Rayven will help you design efficient applications, optimise data retrieval, and manage data access correctly.

      Structured vs Unstructured Storage

      Rayven Applications use two distinct storage layers:

      • Structured Storage (MySQL)

        • Managed explicitly by Workspace users.

        • Used for Tables (Primary and Secondary).

        • Ideal for core business records, reference data, or data requiring relational integrity.

      • Unstructured Storage (Cassandra)

        • Managed automatically by the Rayven platform.

        • Used for all workflow data.

        • Ideal for dynamic, high-volume, or time series data like events, sensor updates, form submissions, or transaction logs.

      Primary Tables and UID Anchoring

      Every application must have one Primary Table. This table:

      • Contains a UID (Unique Identifier) per row.

      • Acts as an anchor for storing unstructured data in the workflow layer.

      • Supports Label columns to enable grouping, filtering, and user access control.

      Workflow data is always stored against one or more UIDs from the Primary Table.

      Secondary Tables for Structured Metadata

      Secondary Tables:

      • Are optional, but often used for metadata or supplementary records.

      • Can be linked to Primary Tables via UID or Label.

      • Do not store time series data but can be read from or updated using workflow nodes.

      Examples:

      • Storing a list of contacts per customer

      • Holding configuration settings per job

      Workflow Data and Time Series Storage

      Every time data flows through a workflow, Rayven automatically:

      • Stores the data in Cassandra as a JSON payload.

      • Links each payload to:

        • A UID from the Primary Table

        • A Node ID (where the data was stored)

        • A Timestamp (when the data was stored)

      This results in a flexible, three-dimensional storage model that supports:

      • Real-time data ingestion

      • Historical analysis

      • Event tracking

      • Dynamic payload structures

      Storage and Interface Widgets

      Frontend nodes (widgets) only display data if data has been passed through them in a workflow.

      • Adding a Frontend node to a workflow does not automatically populate it with historical data.

      • You must send data through the node during workflow execution for it to be stored and displayed.

      • UI Code widgets can optionally retrieve data directly from tables using built-in JS functions (getDataFromTable).

      Practical Implications

      • When designing a Rayven application:

        • Use Primary Tables to define core entities (Customers, Assets, Jobs, etc.).

        • Use Secondary Tables for linked data (Contacts, Notes, Tags, etc.).

        • Use workflows to process and store dynamic or frequent updates.

        • Use frontend nodes to visualise time series data from workflows.

      • While there is no traditional database interface for managing Cassandra-stored data, Rayven provides multiple tools for accessing it:

        • The Inspect Data tab is available within every workflow node and lets you view recent JSON payloads, filterable by UID, date, or content.

        • You can include an Inspect Data node in a workflow for more flexible debugging or real-time inspection.

        • The Node Export function in the Workspace allows you to extract historical data for a specific node and UID/time range.

      Summary

      Rayven's hybrid storage model gives you the power of both relational and time-series databases in one platform. Structured tables are ideal for persistent, queryable records, while the unstructured workflow layer enables high-frequency, schema-flexible data storage. Together, they provide a robust foundation for building data-driven applications—without forcing trade-offs between performance and flexibility.

      FAQs

      Where is data stored when it flows through a workflow?
      Data is stored in Rayven’s unstructured time series storage (Cassandra), linked to a UID, Node ID, and Timestamp.

      Do I need to explicitly save data in workflows?
      No. Data is stored automatically when a workflow node processes it—provided the node has received data during execution.

      Can I view the data stored in Cassandra?
      Not directly. There is no traditional database UI or management interface. However, you can:

      • Use the Inspect Data tab on each workflow node to view recent payloads, with filters for UID and date range.

      • Add an Inspect Data node to a workflow for debugging purposes.

      • Use Node Export in the Workspace to download raw data from a specific node over a given time range.

      Do UI Code widgets need data to be flowed in via workflow?
      Not necessarily. They can retrieve data directly from tables using JavaScript (getDataFromTable()), or they can receive payloads from the workflow like other frontend nodes.

      Is it better to store everything in tables?
      No. Use tables for structured, slow-changing data. Use workflow-based storage for dynamic, high-frequency, or time series data.

      How does Rayven ensure performance with this hybrid model?
      MySQL handles structured queries efficiently, while Cassandra is optimised for high-speed inserts and reads of large-scale, distributed data.