This is a service provided by Amazon Web Services that allows for real-time processing of streaming data at massive scale. In IoT, AWS Kinesis can be used to continuously capture and store large streams of data from multiple devices.
What is AWS Kinesis?
The AWS Kinesis Input Node serves as a crucial component in leveraging Amazon Web Services' Kinesis, a fully managed service tailored for real-time processing of extensive data streams. With its capability to manage high-volume, high-velocity data, Kinesis finds application in diverse scenarios, including log and event data collection, real-time analytics, and monitoring. It operates on the basis of streams, each identified by a unique name, enabling producers to send data to designated streams and consumers to extract data from the correct ones. These streams are subdivided into shards to enhance scalability, with each shard hosting an independent sequence of data records. Shard iterators facilitate tracking the position within a shard for data retrieval, ensuring seamless continuity in data processing. Furthermore, the input node periodically polls data from an AWS Kinesis Bucket, ensuring a steady flow of information for processing.Outputs
The AWS Kinesis node will output each device’s payload in the order that they are received.
How to Use
Implementation
In the Rayven Workflow Builder:
- Select Inputs.
- Drag the AWS Kinesis node to the canvas.
- Provide an input to the AWS Kinesis node by connecting it to another node.
- Double click on the AWS Kinesis node to open the configuration window.
Configuration
|
Section: General |
||
|
This section contains basic configuration elements required for any AWS Kinesis implementation |
||
|
Field |
Requirement |
Comments |
|
Node Name |
Mandatory |
Enter a name for this node. This provides a handle to which you and others can refer, so it should be simple but meaningful and explain the node’s purpose. |
|
AWS Key ID |
Mandatory |
Enter the Access Key ID that was configured on the AWS account |
|
AWS Key Secret |
Mandatory |
Enter the key secret that was configured on the AWS account |
|
Stream Name |
Mandatory |
Enter the name of the AWS Kinesis data stream |
|
AWS Region |
Mandatory |
Enter the region of the AWS account from the dropdown menu. |
|
Activation Time Request (minutes) |
Optional |
Enter a numeric value that is greater than 0. |
|
Interval (minutes) |
Optional |
Enter a numeric value that is greater than 0. |
|
Device ID Field |
Mandatory |
Enter the field name that holds the Device ID. If this is not configured, the payload will be tied to the device __none__ by default which can be viewed using the debug node. |
|
Reset Shard Iterator |
Optional |
The iterator may be reset in scenarios such as reprocessing historical data or recovering from a failure where you need to reprocess data from a certain point. |
|
Section: Take a Date Field from JSON |
||
|
In the case that you would like the timestamp of the data to be based on the data coming from the device, and if the device is coordinated with a time zone, you will need to define the following three fields: |
||
|
Field |
Requirement |
Comments |
|
Date Field Name |
Optional |
Define the name of the date field coming in from your device. |
|
Date Format |
Optional |
Define the format of how the data will be coming through, for example: dd/MM/yyyy HH:mm:ss. If the date is coming through in 24 hour format, the hours need to be in capital letters, (ie. HH/mm/ss) |
|
Timezone |
Optional |
Define from a drop down list, the time zone that the device is recording the data. |
Activation
Once the node has been configured, click the Activate button and then click Save. The node is ready to receive data into the workflow.