The FTP Node enables a Rayven workflow to retrieve and ingest files from an FTP or SFTP server. It periodically scans a specified location, downloads new files, parses their contents, and emits each parsed entry as a payload.
What It Does
This node connects to a remote file server (FTP/SFTP), downloads files on a recurring schedule, and parses them based on the selected format (e.g., JSON). It supports UID mapping, timestamp extraction, and output filtering, allowing workflows to respond to externally uploaded datasets or logs.
Step-by-Step: How to Configure the FTP Node
-
Add the node
-
Drag the FTP Node from the Inputs section to your workflow canvas.
-
-
Open the configuration panel
-
Double-click the node to configure connection, timing, and extraction logic.
-
-
Connect downstream nodes
-
Attach to logic, transformation, or storage nodes that consume the parsed data.
-
Configuration Fields
FTP Connection
| Field | Requirement | Description |
|---|---|---|
| Node Name* | Required | Logical identifier used in the workflow. |
| FTP Address* | Required | FTP/SFTP server address (e.g., ftp.example.com). |
| Username* | Required | FTP/SFTP login username. |
| Password* | Required | FTP/SFTP login password. |
| Enable SFTP | Optional | When enabled, uses SFTP (SSH File Transfer Protocol). |
| Allow File from Email Address | Optional | Accept files from emails if supported by the FTP implementation. |
File Polling & Download Settings
| Field | Requirement | Description |
|---|---|---|
| File Format* | Required | Format of the file (currently: JSON). |
| File Encoding* | Required | Character encoding (e.g., UTF-8, ASCII). |
| Interval Between Output Payloads (ms) | Optional | Throttle output by delaying emission of each payload. Default: 200 ms. |
| File Download Interval (minutes)* | Required | Frequency at which the server is polled for new files. |
| Download Start Date & Time* | Required | Scheduled time to begin file download each day. |
| Download Start Time Zone | Optional | Time zone to interpret the download start time. |
| Skip If Filename Same As Previous | Optional | Prevents duplicate ingestion when filenames match the last processed file. |
| Ingest All Files in Folder | Optional | When enabled, all available files are downloaded at each interval. |
Data Extraction & Filtering
| Field | Description |
|---|---|
| Ignore First N Lines | Skips initial lines (e.g., headers) before parsing begins. |
| Filename Filter | Only files matching the pattern will be processed (e.g., *.json). |
| Output JSON Key for Filename | If set, includes the filename in each payload under this key. |
| Remove File Extension | If enabled, strips file extension from filename. |
| UID JSON Key / Column / Tag | Field used to extract device UID from the file. |
| Use UID IP Address | Optionally use the device IP as the UID. |
| Call FTP Address for Each UID | When enabled, makes individual FTP requests per UID. |
Payload Date Settings (Optional)
| Field | Description |
|---|---|
| Timestamp JSON Key | Field in the file containing the timestamp. |
| Timestamp Format | Format of the timestamp string (e.g., yyyy-MM-dd HH:mm:ss). |
| Time Zone | Interprets the timestamp using the given zone. |
Activation Filters
| Field | Description |
|---|---|
| Logical Operand | Defines how multiple filters are combined (AND or OR). |
| Select Data Source Filter | Select UID(s) or label(s) to restrict ingestion scope. |
Output Example
If a JSON file contains:
{
"device_id": "pump-001",
"pressure": 2.4,
"timestamp": "2025-07-15 08:00:00"
}
With configuration:
-
UID JSON Key =
device_id -
Timestamp JSON Key =
timestamp -
Output JSON Key for Filename =
source_file
The emitted payload:
{
"uid": "pump-001",
"pressure": 2.4,
"timestamp": "2025-07-15T08:00:00Z",
"source_file": "log_20250715"
}
Best Practices
-
Always enable file name filtering if the FTP folder contains mixed formats.
-
Use skip filename check to avoid reprocessing unchanged files.
-
Adjust output interval for downstream nodes that cannot handle burst input.
-
Align timestamp format and time zone with the source system for correct time alignment.
Use Cases
-
Ingest JSON logs uploaded to FTP by remote gateways
-
Download daily batch reports and parse each row as a payload
-
Sync legacy data from secure SFTP storage
-
Integrate with external systems exporting structured files on a schedule
FAQ
Q: What happens if a file cannot be parsed?
A: The workflow will skip the file. You can inspect the raw file manually or enable logging to debug.
Q: Can this node write to FTP?
A: No. This node is for input only. Use the Export Node for outbound file transfers.
Q: Does the node delete files after processing?
A: No. Files remain on the server unless removed externally or by a retention policy.