Remote Data Load

The Remote Data Load activity retrieves a file from a remote storage location and imports its contents into a Una staging table.

Purpose

Use the Remote Data Load activity to:

• Pull a CSV or Excel file from an FTP server, Azure File Storage, Azure Blob Storage, or AWS S3 into a staging table
• Automate the ingestion of files delivered by external systems on a schedule
• Feed data from remote storage into the Una ETL pipeline without manual downloads

Supported Storage Types

Type	Description
FTP	Standard FTP server
Azure File Storage	Azure Files share
Azure Blob Storage	Azure Blob container
AWS S3	Amazon S3 bucket

The storage type is determined by the selected data source.

Configuration

Data Source

Select the remote storage data source. The data source type determines which storage backend is used.

Folder

The folder path within the storage location where the file resides. Supports interpolation:

/uploads/{{var:EntityName}}/{{var:Period}}

Leave blank for the root of the storage location.

File Name

The name of the file to retrieve. Supports interpolation:

Actuals_{{var:Period}}.csv
Budget_{{ctx:year}}.xlsx

Staging Table

The name of the Una staging table to import the file data into.

Table Has Headers

When enabled (default), the first row of the file is treated as a column header row and is not imported as data.

Disable only if the file genuinely has no header row and the first row is data.

Clear Existing Data

When enabled, all existing rows in the staging table are deleted before the new data is imported.

When disabled, the imported rows are appended to any existing data.

Behavior

1. The folder, file name, and staging table name are interpolated.
2. The file is retrieved from the remote storage using the credentials in the data source.
3. The file contents are imported into the staging table using the Una file import service.
4. If the import fails, the error is logged and the exception is re-thrown (activity returns Error).
5. On success the activity returns Success.

Usage Patterns

Retrieve Then ETL

Remote Data Load ──> Data Quality Validation ──> ETL
  └── Source: FTP / uploads / Actuals_{{var:Period}}.csv
      Staging: StagingActuals
      Clear:   true

Per-Entity Loop

Foreach: entity in entities
  └──> Remote Data Load
         Folder:  /uploads/{{foreach:item}}
         File:    Budget_{{var:Period}}.csv
         Staging: StagingBudget_{{foreach:item}}
         Clear:   true

Usage Notes

• All four storage types (FTP, Azure File Storage, Azure Blob, AWS S3) are supported. Any other data source type will throw an exception.
• For AWS S3, the activity performs an STS role assumption login before retrieving the file. Ensure the role ARN is configured in the data source if cross-account access is needed.
• The file must be parseable by the Una staging table import service. Supported formats depend on the import service configuration (typically CSV and Excel).
• The staging table must already exist, or be auto-created by the import service on first load.

Best Practices

• Enable Clear Existing Data unless you explicitly need to accumulate data across multiple loads.
• Use interpolation in the file name and folder to make the activity work across periods and entities without reconfiguration.
• Wire an error arrow — file-not-found or connection failures return Error and should route to a notification.

JSON Reference

{
  "discriminator": "RemoteDataLoadWorkflowActivity",
  "activityId": "<uuid>",
  "name": "Remote Data Load",
  "positionX": 0,
  "positionY": 0,
  "advanceRule": 2,
  "tableName": "StagingActuals",
  "tableHasHeaders": true,
  "clearExistingData": true,
  "dataSource": "MyFtpConnection",
  "folder": "/uploads/{EntityName}/{Period}",
  "fileName": "Actuals_{Period}.csv"
}

Property	Type	Description
tableName	string	Corresponds to the Staging Table field. The name of the staging table to import data into. Supports {variable} interpolation.
tableHasHeaders	boolean | null	Corresponds to the Table Has Headers toggle. When true (default), the first row of the file is treated as a header.
clearExistingData	boolean	Corresponds to the Clear Existing Data toggle. When true, existing table rows are deleted before the import.
dataSource	string	Corresponds to the Data Source field. The remote storage data source (FTP, Azure File Storage, Azure Blob Storage, or AWS S3).
folder	string	Corresponds to the Folder field. The folder path within the storage location. Supports {variable} interpolation.
fileName	string	Corresponds to the File Name field. The name of the file to retrieve. Supports {variable} interpolation.