Key concepts
      Back to home
      1. Home
      2. 1: Getting Started
      3. Key concepts

      Understanding Primary Data Tables and Workflow Data

      In Rayven.io, the Primary Data Table in the MySQL database serves as the organized view of your data, connecting to the raw, unstructured data stored in the Cassandra database through a Unique ID

      What is a Primary Data Table?

      In Rayven.io, a Primary Data Table is a structured table stored in a MySQL database. This table acts as the core structure that organizes and manages key data within a solution. It ensures that data being processed in your workflows is structured, easily accessible, and connected to other relevant data points through unique identifiers.

      Data Flow and Integration in Rayven.io

      Rayven.io manages data using two types of databases:

      1. Structured Data: Stored in the MySQL database, where the primary data table resides.
      2. Unstructured Data: Stored in the Cassandra database, where raw integration data from various systems and devices is collected.

      When data enters Rayven.io through a workflow, it is first ingested into the unstructured Cassandra database. This data can come from various sources such as IoT devices, external APIs, or other systems. The critical part of the process is how this incoming data connects to the primary data table.

      Data Connection via Unique ID

      To ensure the structured and unstructured data stay in sync:

      • Each entry in the workflow is associated with a Unique ID.
      • This Unique ID links the unstructured data in Cassandra to the corresponding record in the primary data table within the MySQL database.

      The primary data table in MySQL provides the organized view of the data, while the Cassandra database collects raw, unstructured inputs. The workflow connects these two layers by appending data through the unique identifier, ensuring all ingested data has a reference point in the structured primary table.

      Data Outputs from the Workflow

      The data outputs in Rayven.io workflows come from the unstructured data stored in Cassandra. This allows the platform to provide real-time processing and visualization of incoming data. However, attributes in the MySQL primary table can be used as labels for filtering or grouping this data, offering a high degree of flexibility.

      Leveraging Labels from the Primary Table for Grouping & Filtering

      Rayven.io allows you to use attributes in the MySQL primary table as labels to filter and group the data flowing through your workflows. These labels provide granular control over how data is sliced, diced, and visualized.

      Filtering and Grouping in Workflows

      In each workflow node, you have the option to filter the incoming data by any of the labels defined in the MySQL primary database. This gives you immense flexibility in processing and presenting data. For example:

      • You can group data by a label such as building name, allowing you to preprocess and aggregate data by building.
      • This approach lets you summarize data by building, providing insights at different levels, such as building, floor, or room.

      This level of customization ensures that data is both actionable and presented in a way that meets your business needs.

      Example Use Case: Smart Building Energy Monitoring

      Let’s use an example of a smart building energy monitoring system to demonstrate the power of the primary data table and workflows.

      Scenario: An organization monitors energy consumption from various smart meters in different buildings across multiple cities. The solution captures real-time energy data, processes it, and presents insights through dashboards.

      Steps:

      1. Data Ingestion:

        • Raw energy data (kWh) is ingested from smart meters and saved in the unstructured Cassandra database.
        • Each smart meter is assigned a Device ID and associated with building names and locations stored in the primary data table.

      2. Primary Data Table:

        • The primary table in MySQL holds structured data fields like:
          • Device ID
          • Building Name
          • Energy Consumption (kWh)
          • City
          • Timestamp

      3. Workflow Processing:

        • The workflow ingests data from the Cassandra database, using the Device ID to append it to corresponding records in the primary data table.
        • By using labels like Building Name, the workflow can aggregate and summarize data by building, offering a clear view of energy consumption across different locations.

      4. Filtering in the Workflow:

        • In each node of the workflow, data is filtered by building using the Building Name label from the primary data table. This allows energy usage data to be grouped by building, providing summarized energy reports.

      5. Data Outputs:

        • The aggregated and filtered data from the Cassandra database is visualized in a dashboard, showing real-time energy consumption for each building.
        • Alerts can be triggered if energy consumption exceeds set limits, sending notifications to building managers.

      Conclusion

      In Rayven.io, the Primary Data Table in the MySQL database serves as the organized view of your data, connecting to the raw, unstructured data stored in the Cassandra database through a Unique ID. This system allows workflows to process, filter, and group data based on attributes from the primary table, offering immense flexibility in how data is sliced, aggregated, and visualized.

      By utilizing attributes from the primary table as labels, you can create powerful data visualizations and business logic that provide actionable insights, tailored to specific needs such as location, device type, or time period.