Input Connector Nodes
      Back to home
      1. Home
      2. 5: Workflow Toolkit
      3. Input Connector Nodes

      Databricks Connector

      The Databricks connector node in Rayven.io provides a powerful way to automate SQL queries and integrate Databricks data into your workflows

      The Databricks connector node in Rayven.io allows seamless integration with your Databricks environment, enabling you to run SQL queries, execute statements, and create queries directly from your Rayven.io workflows. With this integration, you can automate data analysis and reporting, ensuring real-time insights by bridging Databricks with other systems in your workflows.

      Setting up the Databricks Connector in Rayven.io

      1. Create a Databricks Account and API Token:

        • Sign up or log in to Databricks at Databricks.
        • Navigate to Settings > User Settings > Access Tokens and generate a new API token.
        • Copy and store the token securely for use in Rayven.io.

      2. Add Databricks Credentials in Rayven.io:

        • In Rayven.io, go to Settings > Credentials.
        • Add your Databricks API token, workspace URL, and other relevant credentials to establish the connection.

      Available Databricks Operations in Rayven.io

      Below are the pre-configured endpoints available for the Databricks connector within Rayven.io.

      Input Operations:

      1. Statement Execution API
        Endpoint: Databricks SQL Execution Tutorial
        Description: Execute SQL statements against Databricks and retrieve query results for further processing.
        Usage: Use this node to run SQL queries on your Databricks workspace, analyze data, and retrieve results in real time.

      2. Create a Query
        Endpoint: Create a Query
        Description: Automate the creation of new SQL queries in Databricks for reuse in future workflows.
        Usage: Use this node to create reusable queries directly from your Rayven.io workflow.

      Using the Databricks Connector in Rayven.io Workflows

      1. Create a New Workflow:

        • Navigate to Workflows in Rayven.io and click Create Workflow.

      2. Add Databricks Input Nodes:

        • Drag and drop the Statement Execution API Node or the Create Query Node into your workflow.

      3. Configure the Nodes:

        • For the Statement Execution API, enter the SQL statement you want to execute.
        • For the Create Query Node, define the query parameters and save the query for reuse.

      4. Incorporate Triggers and Business Logic:

        • Add triggers to initiate workflows based on query results or events (e.g., trigger an alert if sales figures exceed a threshold).
        • Use logic nodes to filter, transform, or group data retrieved from Databricks.

      5. Push Data to Dashboards or External Systems:

        • Use visualization nodes to display Databricks data on Rayven dashboards.
        • Output nodes can send processed data to other systems for further analysis or reporting.

      Example Use Case: Automated Sales Reporting

      Scenario: A company wants to automate its sales report by querying sales data from Databricks and displaying it in real-time dashboards within Rayven.io.

      1. Workflow Setup:

        • Use the Statement Execution API Node to run a SQL query retrieving the latest sales data.
        • Configure a trigger node to run the query daily.
        • Use a dashboard node to display sales trends and summary statistics.

      2. Optimization:

        • Create a reusable query using the Create Query Node for consistent reporting.
        • Use trigger nodes to notify stakeholders via email or Slack if sales figures exceed targets.

      HTTP Node for Custom API Calls

      If additional Databricks operations are required, use the HTTP Node in Rayven.io to call any Databricks API endpoint directly.

      Steps:

      1. Add an HTTP Node to your workflow.
      2. Select Authentication > Predefined Credential Type and choose Databricks.
      3. Provide the API endpoint, headers, and request body as required.

      Refer to the Databricks API Documentation for more details on available operations.

      Conclusion

      The Databricks connector node in Rayven.io provides a powerful way to automate SQL queries and integrate Databricks data into your workflows. With support for statement execution and query creation, this connector ensures seamless access to your data warehouse and analytics platform.

      For further details, refer to:

      If you need further assistance with setup or have additional requirements, contact our support team for help.