Defining External S3 Buckets

You can define an external S3 bucket (with its Connection parameters), and reference it when you create new sources in your ingestion pipelines. It helps mutualizing the Connection parameters, and avoid defining them again each time you create a new source.

This task shows you how to:

See Also
Main Workflows
Configuring the Ingestion Pipeline

Add a New External S3 Bucket

  1. From the home page, click Create Storage.
    The Add new storage dialog box opens to let you create a new storage or import an existing storage configuration (see Exporting and Importing a Storage Configuration.)
  2. Click Create Storage.
  3. Enter a Storage name.

    It is possible to edit this name later.

  4. For Storage type, select External S3 Bucket.
  5. Click Create.
    The S3 Bucket configuration menu opens.

Configure the Connection

You can configure the source, that is the connection to the storage on which Data Factory Studio must fetch source files.

  1. Select the Connection tab.
  2. Configure the Connection parameters.

    Parameter Description
    Provider

    Prefills the URL with the provider's site.
    Url

    Defines the URL of the storage.
    Region

    Defines the region of the storage.
    Access key id

    Defines the access key to connect to the storage.
    Secret access key

    Defines the private access key to connect to the storage.

  3. Enter the Bucket name, that is in which S3 bucket Data Factory Studio fetches source files.
    Data Factory Studio test the connectivity to the external S3 source automatically. If it fails, an error message describes the reason of this failure in the Validation section.
  4. Click Save.

Now, if you add a new Ingestion pipeline, the Source tab displays a Storage field suggesting the External S3 Bucket you have just defined.

Delete an S3 Storage Bucket

  1. In the General tab, go to the bottom of the S3 Bucket page.
  2. Click Delete.