R2022x FD01 (FP.2205)
- You can define an external S3 bucket (with its Connection parameters), and reference it when you create new sources in your ingestion pipelines.
- This new type of storage allows you to store files as objects (with their associated metadata and unique identifiers), in a DFS Object Storage.
- You can index data from tabular files (CSV, Parquet, JSON) stored in an Object Storage Bucket, as one or several classes in a Semantic Graph Index Unit.
- You can push indexing event orders, serialized in JSON files in an external S3, into an index unit.
- You can copy objects from an external S3 storage to an Object Storage Bucket, with a source scheduling.
Defining an External S3 Bucket for Quick Reference
Benefits: It helps mutualizing the
Connection parameters, and avoid defining
them again each time you create a new source.
For more information, see
Defining External S3 Buckets
DFS Object Storage
Benefits: You can store binary data, such as images or videos,
that do not fit into Semantic Graph Index Units, in the DFS Object Storage service of the 3DEXPERIENCE platform.
For more information, see
Configuring Object Storage Buckets
Indexing Tabular Data
Benefits: You can store data from tabular files into Semantic Graph Index Units.
For more information, see
Indexing Tabular Data
Indexing JSON Event Data
Benefits: Serializing the indexing of event data into Semantic Graph Index Units can save a lot of indexing time. As you can define
precisely the lines to update (with Add and Delete orders) from
source files, you get a faster partial update by lines of files. The
default partial update by entire files is longer since Data Factory Studio has to delete and push them again.
For more information, see
Indexing Event Data
Copying an External S3 Storage
Benefits: You can easily copy objects from one source to another.
For more information, see
Copying Object Data