Payments Data Platform | Modernbanc

Working with Files

With Modernbanc, handling files is straightforward and flexible. This guide covers how files can be sourced and utilized within your workflows.

File Sources

Files can be introduced into workflows through two primary methods:

  • Webhook Triggers: Attach files directly to webhook requests. These files become accessible within your workflow at _trigger_version._input.files. Each file reference includes attributes such as name, mime_type, and size (in bytes).
  • Container Steps: Files generated within container steps are also available to the workflow. Any file saved to the /data directory by a container can be used in subsequent steps.

Utilizing Files

Files can be referenced and used in various steps within a workflow. Here’s how:

  • Direct Reference: Use the file or files field in step parameters to directly reference a file. For example, file: {{_trigger_version._input.files[0]}} for the first file from a webhook trigger, or file: my-passed-in-file.csv for a file named explicitly.
  • Supported Step Types: Currently, parse_file and container steps can accept file inputs.

Container Step

The container step allows the execution of a Docker container within your workflow. This step can both access existing files and generate new ones. You can read more about it here.

Data Directory Access: The container step has access to a /data directory, where all files for all containers across the execution are stored. Any files placed in this directory will be available for the duration of the execution, allowing for seamless file sharing and manipulation across multiple container steps.

Passing Files to Container: To introduce files into the /data directory when running a container step, you can specify them in the parameters.files field. This approach is useful for utilizing files that are either attached to a run workflow request or are generated by previous steps in the workflow.

For example: to use a file attached to a run workflow request, you would specify it as follows:

  "files": ["{{_trigger_version._input.files[0]}}", "my-passed-in-file.txt"]

Parse File Step

The parse_file step is designed to convert files, such as CSVs, into JSON format, making the data easier to work with in subsequent steps.

  • Passing a File: To parse a file, set the file parameter to the file you wish to convert, e.g., "file": "{{_trigger_version._input.files[0]}}".
  • Accessing Container-Generated Files: Files created in a container step and saved to /data can be referenced directly. For instance, a file saved as /data/out/out.csv is accessible via /out/out.csv.