# S3 Bucket Uploader (`mstephen190/s3-bucket-uploader`) Actor

Upload the items from the default dataset of an actor's run to an S3 bucket in JSON format.

- **URL**: https://apify.com/mstephen190/s3-bucket-uploader.md
- **Developed by:** [Matthias Stephens](https://apify.com/mstephen190) (community)
- **Categories:** Automation, Open source
- **Stats:** 46 total users, 7 monthly users, 99.2% runs succeeded, 2 bookmarks
- **User rating**: No ratings yet

## Pricing

Pay per usage

This Actor is paid per platform usage. The Actor is free to use, and you only pay for the Apify platform usage, which gets cheaper the higher subscription plan you have.

Learn more: https://docs.apify.com/actors/running/actors-in-store.md#pay-per-usage

## What's an Apify Actor?

Actors are web data automations that power AI and operations. They run on the Apify platform to scrape websites, process data, connect APIs, and automate workflows.
In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours,
and optionally produces a well-defined JSON output, datasets with results, or files in key-value store.
In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server.
Actors are written with capital "A".

## How to integrate an Actor?

If asked about integration, you help developers integrate Actors into their projects.
You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready.
The best way to integrate Actors is as follows.

- **AI agents and MCP clients** — the [Apify MCP server](https://docs.apify.com/integrations/mcp.md) at `https://mcp.apify.com` (remote, streamable HTTP, OAuth on first use).
- **Agentic workflows and local Actor development** — [Agent Skills](https://apify.com/.well-known/agent-skills/index.json) with the [Apify CLI](https://docs.apify.com/cli/docs.md): `npm install -g apify-cli`, then `apify login`.
- **JavaScript/TypeScript projects** — the official [JS/TS client](https://docs.apify.com/api/client/js/docs.md): `npm install apify-client`.
- **Python projects** — the official [Python client](https://docs.apify.com/api/client/python/docs.md): `pip install apify-client`.
- **Any other language** — the [REST API](https://docs.apify.com/api/v2.md).

For usage examples, see the [API](#api) section below.

For more details, see Apify documentation as [Markdown index](https://docs.apify.com/llms.txt) and [Markdown full-text](https://docs.apify.com/llms-full.txt).

# README

## Amazon S3 Bucket upload

This actor allows you to upload the default dataset of an actor's run to an AWS S3 bucket. It provides easy-to-use tools for injecting information into file names/path names. Additionally, it provides the option of either uploading the entire dataset to one file, or each dataset item as its own file.

The Amazon S3 Bucket upload actor is best used within the webhooks of another actor, as to allow for the automatic uploading of its default dataset's items.

### Input

Divided into two short sections, this actor's input is intuitive.

#### Bucket configuration

This section contains details for configuring your bucket. Ensure that you've filled in everything exactly right, especially your credentials.

> To learn more about how to get your credentials, check out [this](https://docs.aws.amazon.com/powershell/latest/userguide/pstools-appendix-sign-up.html) page.

#### Data configuration

In this section, you provide the run ID of the actor, as well as the path for the item and the file name. There are some restrictions for the `fileName` and `pathName` inputs as to prevent unnecessary errors:

- `pathName` **cannot** start with or end with a `/` symbol.
- Neither field can include a period (`.`) character. (the file extension will be automatically added for you)
- `fileName` can't include any `/` characters.

Within `pathName` and `fileName`, you have access to 6 variables:

| Variable      | Example                                | Description                                         | Unique for each item |
| ------------- | -------------------------------------- | --------------------------------------------------- | -------------------- |
| `actorName`   | `my-actor`                             | The name of the actor matching the provided run ID. | No                   |
| `runId`       | `BC6hdJvyNQStvYLL8`                    | The run ID of the actor which was provided          | No                   |
| `date`        | `2022-05-29`                           | The date at which the actor finished its run.       | No                   |
| `now`         | `1653851198127`                        | The current time in milliseconds.                   | Yes                  |
| `uuid`        | `b2638dac-00b5-4e29-b698-fe70b6ee6e0b` | A totally unique ID.                                | Yes                  |
| `incrementor` | `3`                                    | An integer that increments up for every item.       | Yes                  |

Variables allow you to easily generate unique file names when writing multiple files (preventing files from being overwritten). `now` and `uuid` are great options when you need unique values. Here is an example of some variables being used in the actor's input:

```JSON
{
  "pathName": "{actorName}/datasets/{date}",
  "fileName": "{uuid}-item{incrementor}",
  "separateItems": true
}
```

> Notice that you must wrap a variable name in `{curlyBraces}` for it to work.

Here is what the final path for one file might look like with this configuration:

```text
my-actor/datasets/2022-05-29/b2638dac-00b5-4e29-b698-fe70b6ee6e0b-item7.json
```

By default, the actor will write the entire dataset as one file in the S3 bucket. In order to write each dataset item as a separate file in the S3 bucket, set `separateItems` to `true`. When you have this option set to `true`, ensure that you are using at least one unique variable in the `fileName`, otherwise it will keep writing and overwriting the same file (unless you use unique variables in the `pathName`, however, that is not recommended).

# Actor input Schema

## `bucketName` (type: `string`):

The name of the bucket.

## `region` (type: `string`):

Select your bucket's region

## `accessKeyId` (type: `string`):

The access key found on your AWS account.

## `secretAccessKey` (type: `string`):

The secret access key found on your AWS account.

## `actorRunId` (type: `string`):

The ID of the actor's run.

## `pathName` (type: `string`):

The path you'd like each record to live in. Check the README to learn about which variables are available. It is not recommended to use unique variables in the path name.

## `fileName` (type: `string`):

The name each file should have. You do not need to include the file's extension. Check the README to learn about which variables are available.

## `separateItems` (type: `boolean`):

When set to "false", the entire dataset will be uploaded as a single JSON file. When set to "true", each dataset item will be its own file.

## Actor input object example

```json
{
  "bucketName": "my-bucket-name",
  "region": "us-east-2",
  "accessKeyId": "OYIAZ6R3TZ9LK2UXZGV5",
  "secretAccessKey": "8vTy4zsSmjLiiRS9hSOeRPHRC6cNRC2y57r4KGSY",
  "actorRunId": "BC6hdJvyNQStvYLL8",
  "pathName": "{actorName}/runs/{date}",
  "fileName": "{runId}-{incrementor}",
  "separateItems": false
}
```

# API

You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup.

## JavaScript example

```javascript
import { ApifyClient } from 'apify-client';

// Initialize the ApifyClient with your Apify API token
// Replace the '<YOUR_API_TOKEN>' with your token
const client = new ApifyClient({
    token: '<YOUR_API_TOKEN>',
});

// Prepare Actor input
const input = {
    "pathName": "{actorName}/runs/{date}",
    "fileName": "{runId}-{incrementor}"
};

// Run the Actor and wait for it to finish
const run = await client.actor("mstephen190/s3-bucket-uploader").call(input);

// Fetch and print Actor results from the run's dataset (if any)
console.log('Results from dataset');
console.log(`💾 Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`);
const { items } = await client.dataset(run.defaultDatasetId).listItems();
items.forEach((item) => {
    console.dir(item);
});

// 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/js/docs

```

## Python example

```python
from apify_client import ApifyClient

# Initialize the ApifyClient with your Apify API token
# Replace '<YOUR_API_TOKEN>' with your token.
client = ApifyClient("<YOUR_API_TOKEN>")

# Prepare the Actor input
run_input = {
    "pathName": "{actorName}/runs/{date}",
    "fileName": "{runId}-{incrementor}",
}

# Run the Actor and wait for it to finish
run = client.actor("mstephen190/s3-bucket-uploader").call(run_input=run_input)

# Fetch and print Actor results from the run's dataset (if there are any)
print(f"💾 Check your data here: https://console.apify.com/storage/datasets/{run.default_dataset_id}")
for item in client.dataset(run.default_dataset_id).iterate_items():
    print(item)

# 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/python/docs/quick-start

```

## CLI example

```bash
echo '{
  "pathName": "{actorName}/runs/{date}",
  "fileName": "{runId}-{incrementor}"
}' |
apify call mstephen190/s3-bucket-uploader --silent --output-dataset

```

## MCP server setup

```json
{
    "mcpServers": {
        "apify": {
            "type": "http",
            "url": "https://mcp.apify.com/?tools=fetch-actor-details,mstephen190/s3-bucket-uploader"
        }
    }
}

```

The hosted server signs you in with OAuth on first connect, so no API token belongs in this config. Clients without OAuth support can send an `Authorization: Bearer <APIFY_API_TOKEN>` header instead, using a token from API & Integrations in Apify Console (https://console.apify.com/settings/integrations).

## OpenAPI specification

Download the OpenAPI definition: https://api.apify.com/v2/actors/RuXbgQWDu48VpXON6/builds/BQy4dKndtK6kqW6js/openapi.json