09. Datasets

A table your workflows can read and write. You manage them on the Datasets screen in the sidebar.

A workflow remembers nothing once its execution ends. When you need to remember something between executions — like "have I already processed this order?" — you use a dataset.

---

When to use one

Situation	How you use it
Preventing duplicate processing	Store the order numbers you processed and compare on the next run
Keeping a running record	Append one row of daily revenue and watch the trend
Managing a list of settings	Keep the list of notification recipients as a table
Keeping an execution log	Record what was processed and when

For a single-value setting, a variable is a better fit. Datasets are for things where rows pile up.

---

Creating one

Press the create button on the Datasets screen and fill it in.

Field	Description
Title	The dataset title. It must be unique within the workspace, and workflows reference it by this name
Description	A description (optional)
Columns	Column definitions. At least one is required

Each column gets a name and a type.

Type	What it holds
STRING	Text
NUMBER	Numbers
BOOLEAN	True/false
DATETIME	Dates and times

You can reorder columns with the arrow buttons.

Example — "processing-log"

Column	Type
orderId	STRING
amount	NUMBER
processedAt	DATETIME
notified	BOOLEAN

Once created, click it in the list to view and edit rows directly.

---

Using it in a workflow

Use the dataset integration. → 04. Integration Catalog

Reading (QUERY)

- id: load-history
  name: Read the processing log
  type: CALL
  integration: dataset
  input:
    datasetTitle: "processing-log"
    operation: QUERY
    limit: 100        # default 100, maximum 1000
    offset: 0

The output has the shape { rows: [...] }.

${nodes.load-history.response.body.rows | raw}      # the whole array (| raw required)
${nodes.load-history.response.body.rows.0.orderId}  # the orderId of the first row

Every row contains the id the system assigns plus the columns you defined.

Writing (INSERT)

- id: save-history
  name: Save to the processing log
  type: CALL
  integration: dataset
  input:
    datasetTitle: "processing-log"
    operation: INSERT
    data:
      orderId: "${nodes.trigger.response.body.orderId}"
      amount: "${nodes.trigger.response.body.amount}"
      processedAt: "${nodes.trigger.response.body.windowEnd}"
      notified: "true"

The output is the row you just saved: ${nodes.save-history.response.body.id}

Keys that are not columns are dropped silently. If a value never made it in, suspect a typo in the column name first.

Updating (UPDATE) / deleting (DELETE)

- id: mark-done
  type: CALL
  integration: dataset
  input:
    datasetTitle: "processing-log"
    operation: UPDATE
    rowId: "${nodes.find.response.body.rows.0.id}"
    data:
      notified: "true"

- id: cleanup
  type: CALL
  integration: dataset
  input:
    datasetTitle: "processing-log"
    operation: DELETE
    rowId: "${nodes.find.response.body.rows.0.id}"

The DELETE output is { deleted: true, id: ... }.

---

Reading directly from an expression

For simple lookups you do not need a dataset node at all — an expression is enough.

Expression	Meaning
${datasets.processing-log.count}	The row count
${datasets.processing-log.latest}	The most recent row, whole (an object — needs \| raw)
${datasets.processing-log.latest.orderId}	A specific field of the most recent row

userPrompt: |
  Processed so far: ${datasets.processing-log.count}
  Last processed at: ${datasets.processing-log.latest.processedAt}

A handy way to save yourself a node. If you need to look at several rows, use QUERY on a dataset node.

---

A worked example — preventing duplicate processing

# 1) read the recent processing log
- id: load
  type: CALL
  integration: dataset
  input:
    datasetTitle: "processing-log"
    operation: QUERY
    limit: 200

# 2) check whether this order number is already there
- id: check
  type: CALL
  integration: transform_jmespath
  input:
    expression: "{alreadyProcessed: length(rows[?orderId == `${nodes.trigger.response.body.orderId}`]) > `0`}"
    data: "${nodes.load.response.body | raw}"

# 3) branch on the result
- id: route
  type: CONDITIONAL
  execution-info:
    conditions:
      - label: skip
        expression: "alreadyProcessed == 'true'"
      - label: process
        otherwise: true

Do not forget to hand alreadyProcessed over to the conditional node on the edge. → 06. Flow Control

---

Things to watch out for

• They are per workspace.
• datasetTitle looks the dataset up by its title string. Rename the title and every workflow using the old name fails.
• The maximum number of rows you can read at once is 1000. Beyond that, page through with offset.
• This is not a bulk data store. Use it for the things your automation needs to remember.

---

Next → 10. Executions and Monitoring