In Rayven, we handle two types of databases—Cassandra and MySQL—each serving a distinct purpose to optimize the performance and flexibility of our platform.
Databases in Rayven: A Technical Overview
In Rayven, we handle two types of databases—Cassandra and MySQL—each serving a distinct purpose to optimize the performance and flexibility of our platform.
1. Cassandra for Unstructured Data
We utilize Cassandra for storing unstructured data. Cassandra is a highly scalable NoSQL database designed to handle large amounts of raw data. This is particularly useful for managing the payloads and raw data that flow through Rayven’s workflows. These can include data from IoT devices, logs, events, and other large datasets that require efficient and distributed storage.
-
Why Cassandra? Cassandra excels in handling high write and read throughput across distributed systems, making it ideal for real-time data ingestion. It allows Rayven to store and manage raw data at scale, ensuring that workflows process this data efficiently without bottlenecks.
-
Usage in Rayven: Raw data coming from various sources like devices, machines, or APIs is directly inserted into workflows and stored in Cassandra. This is where all the payloads reside, ensuring data is always accessible for processing in real-time or batch workflows.
2. MySQL for Metadata Information
For storing metadata and structured information, Rayven uses MySQL, a relational database. Metadata refers to data that describes the organization, structure, and relationships of the data flowing through the platform. This includes information about projects, users, workflows, and connectors.
-
Why MySQL? MySQL is highly optimized for structured, relational data. It is known for its reliability and consistency in managing metadata that requires transactional accuracy, such as user details, workflow configurations, and other structured relationships within the system.
-
Usage in Rayven: MySQL is responsible for handling the metadata that keeps the system operational. This includes:
- User and project details: Information related to users and projects, such as permissions and configurations.
- Workflow metadata: How workflows are set up, including connectors, transformations, and logic.
- Dashboard metadata: Layout and configurations for dashboards that visualize data.
Why Two Databases?
Using both Cassandra and MySQL allows Rayven to achieve optimal performance and scalability. Unstructured, high-velocity data like sensor readings or logs fits best in a NoSQL database like Cassandra, while MySQL handles the structured, transactional data that is critical for managing metadata.
- Cassandra is fast and scalable for high-volume data processing.
- MySQL ensures structured data integrity and transactional consistency.
By leveraging both databases, Rayven balances the needs of real-time data processing with reliable metadata management, making our platform highly adaptable and scalable for a variety of use cases.
This separation between raw data storage and metadata management provides flexibility, performance, and scalability to Rayven’s platform, enabling it to support both IoT and AI-driven applications effectively.