> AI agents: this is one page from Mammoth Analytics documentation. The index of all pages as Markdown is https://docs.mammoth.io/llms.txt. Append `.md` to any docs URL, or send `Accept: text/markdown`, to get Markdown.

# Data Consolidation

> Create a Data Consolidation automation that merges files uploaded to a source folder into a destination dataset using Combine or Replace.

Data Consolidation automatically processes files uploaded to a source folder and merges them into a destination dataset. Every time a new file lands in the folder, Mammoth triggers the consolidation—no schedule needed.

## How it works

```plaintext
New file uploaded to source folder
        │
        ▼
Mammoth detects the upload
        │
        ▼
Schema validation (columns must be compatible)
        │
        ▼
Consolidate into destination dataset (Combine or Replace)
```

The consolidation fires on each upload event.

## Before you start

- **Source folder**: A folder in your Data Library that your team uploads files to. Create one if it doesn't exist.
- **Destination dataset**: The dataset that will receive the consolidated data. It must already exist and have a schema compatible with the files you plan to upload.
- **Schema compatibility**: All files uploaded to the source folder must have columns that match the destination dataset's schema. How Mammoth handles mismatches depends on the destination dataset's **If column type doesn't match** and column mapping settings. See [Add or replace data](https://docs.mammoth.io/learn/data-library/add-or-replace-data/).

## Creating a Data Consolidation automation

### Step 1: Open the automation wizard

Open the **New Automation** entry point (available from the Data Library header, a folder's empty-state, or a resource's context menu), then select **Data Consolidation** from the automation type picker.

The wizard opens as a modal with a step indicator at the top. For Data Consolidation it walks through **Select source folder**, **Select destination**, and **Review & save**.

![20261007_190207_draft_01_learn_automation-creation-editing-moved-from-full-page-routes-to-in-place-modals-with-live-status-and-quick-actions-2_0 (1).png](https://docs.mammoth.io/api/v1/images/20261008_124813_20261007_190207_draft_01_learn_automation-creation-editing-moved-from-full-page-routes-to-in-place-modals-with-live-status-and-quick-actions-2_01.png)

### Step 2: Select the source folder

In the **Select source folder** step, choose the folder that contains (or will contain) the incoming files.

Once selected, the folder name appears. Any file uploaded to this folder after you create the automation will trigger consolidation.

You can also refine which files are processed:

- Turn on **Include subfolders** to also process files added to subfolders of the selected folder.
- Turn on **Only process files matching a name pattern** to pick up only files whose names match a pattern. Choose a match type — **Starts with**, **Ends with**, **Contains**, or **Matches exactly** — and enter the text to match. Optionally turn on **Case sensitive** to require an exact-case match. If you leave name-pattern matching off, all file names are allowed and every file in the folder is processed.

The "Select source folder" wizard step, showing the selected source folder with the "Switch" action, the "Include subfolders" checkbox, and the expanded name-pattern controls (match-type dropdown, text input, and Case sensitive checkbox)

**Match type reference:**

| Match type | Matches a file when its name... |
| --- | --- |
| **Starts with** | begins with the text you enter |
| **Ends with** | ends with the text you enter |
| **Contains** | includes the text you enter anywhere |
| **Matches exactly** | is identical to the text you enter (aside from case, unless Case sensitive is on) |

By default, matching is not case sensitive (`Invoice` and `invoice` are treated the same). Turn on **Case sensitive** if you need an exact-case match.

> If you're editing an automation that was created before these options existed, its original "contains text" filter is preserved automatically and shown as a **Contains** match when you open it for editing.

### Step 3: Select the destination dataset

In the **Select destination** step, choose the dataset that receives the data.

> *"Data from the source folder will be added to this dataset."*

The destination dataset must have a schema that's compatible with the files in your source folder. From this step you can also switch the destination dataset, and adjust its update/mapping settings or new-data settings inline.

![20261007_190207_draft_01_learn_automation-creation-editing-moved-from-full-page-routes-to-in-place-modals-with-live-status-and-quick-actions-2_2 (1).png](https://docs.mammoth.io/api/v1/images/20261008_124829_20261007_190207_draft_01_learn_automation-creation-editing-moved-from-full-page-routes-to-in-place-modals-with-live-status-and-quick-actions-2_21.png)

### Step 4: Review and save

In the **Review & save** step, name the automation, optionally add a description, review the summary of the source folder, destination dataset, and filter settings, then click **Save automation**.

The "Review & save" wizard step, showing the Name and Description fields along with the automation summary listing Type, source folder, and destination dataset

## Consolidation modes

Mammoth offers two modes for how incoming data is written to the destination dataset:

| Mode | Behavior | Best for |
| --- | --- | --- |
| **Combine** | Appends rows from the new file to the existing dataset. Data accumulates over time. | Incremental data (daily transactions, event logs) |
| **Replace** | Overwrites the destination dataset with the contents of the new file. Previous data is removed. | Full snapshots (weekly inventory exports) |

> Configure the update method on the destination dataset before creating the automation.

## Schema requirements

Data Consolidation validates column schemas before merging. The columns in uploaded files must be compatible with the destination dataset's schema. This is a separate check from the file name filtering described in [Step 2: Select the source folder](#step-2-select-the-source-folder) — a file can match your name pattern and still fail here if its columns don't line up with the destination dataset.

**What Mammoth checks:**

- Column names must match (by display name)
- Data types must be compatible

**If a schema mismatch occurs:**

- Depending on the dataset's settings, Mammoth converts the column to text, converts incompatible values to null, or does not update the dataset
- If the consolidation fails, the automation status shows **Error**

Keep your source files consistent. If different team members upload files with slightly different column names, the columns may not map to the destination dataset. Define a standard template for uploads.

## Monitoring consolidations

Click the automation row to open the detail panel. The **Automation Summary** shows:

- The source folder name
- The destination dataset(s)
- The last run result (Success or failure reason)

The automation status in the list reflects the outcome of the most recent consolidation:

| Status | Meaning |
| --- | --- |
| **Active** | Watching the folder; last consolidation succeeded |
| **Error** | The automation has an error (for example, a deleted source or destination) |
| **Suspended** | Paused—uploads to the folder will not trigger consolidation |

## Troubleshooting

### Status shows Error after a file upload

**Cause:** The uploaded file's columns or data types do not match the destination dataset schema.

**Fix:** Open the detail panel and check the **Last run result**. Compare the uploaded file's columns with the destination dataset schema, then correct the file and upload it again.

### Source folder or destination dataset was deleted

**Cause:** The automation references a folder or dataset that no longer exists. It shows an error indicator.

**Fix:** Edit the automation to point to new resources, or delete it.

---
Source: https://docs.mammoth.io/learn/automations/data-consolidation.md · Updated: 2026-10-13