FCS Bucket Replication

Bucket replication improves File Collaboration Server (FCS) replication/synchronization performance for small files (<5 MB) across networks showing a non-negligible latency (>5 ms). It does this by reducing and condensing the HTTP traffic that occurs between FCS servers during the replication process. Bucket replication replaces the former algorithm as it improves replication process performance. For example, with the default system fcssettings, bucket replication is 2.5 times faster than the former sync algorithm for small files (<5 MB) over a network with 100 ms latency. When network latency is null, bucket replication shows a 15 ms improvement with the same data set.

See Also
Synchronizing Captured Stores
FCS Compressed Synchronization
MQL Commands for Compressed Synchronization

Bucket replication performance can be further improved by using the new kernel settings, fcssettings.syncmaxfilenum and fcssettings.syncmaxsize. These settings provide a way to adjust bucket replication bandwidth consumption to your network capacity, as they set the size of sync buckets.

Bucket replication applies only to captured, hashed stores and locations using file protocol and having a valid FCS URL (i.e., stores and locations that are managed by an FCS Server).

To be fully operational, bucket replication requires FCS servers to have at least V6R2013 level software to be installed. If this requirement is not met, replication switches to the former (non-bucket) replication protocol.

FCS Bucket Replication Use Cases

Bucket replication affects the following use cases:



MQL Sync Business Object command
With this command, all of the files of the business object are copied between FCS servers by sets or packs of files instead of one-by-one. The scope of the command has not changed, all of the files are just synchronized in one transaction. Synchronized files are still grouped in buckets.
MQL Sync Store command
With this command, all of the files of a store or location are part of the replication process. The new replication protocol required a complete rewrite of this command. The important changes brought by bucket synchronization include:
  • Transaction scope—Formerly, sync store provided a commit N argument, which allowed the replication to be committed after successful replication of N business object files. Since the bucket replication granularity is now a pack of files, a replication transaction is committed after successful replication of each pack of files.
  • Reliability and error management—The continue argument of the Sync Store command implies that the synchronization process goes on even in the case of errors. So, if the replication of a pack of files fails, the new synchronization algorithm tries to replicate each file of the failing pack individually. This allows for a finer granularity of error diagnostics.
  • Performance—The FCS replication service relies on the Multiple File Copy Protocol, which can divide by 10 the number of FCS requests required to perform store/location synchronization on small files (<5 MB).
Alternative Sync Store use cases
The Sync Store command offers the following options, each of which can be identified by a distinct use case:
  • Overwrite Sync—In this case, each up-to-date file at the source location/store is projected to the destination location/store, even if there is already a valid copy of this file at the destination.
  • Update Sync—In this case, an up-to-date file at the source location/store is projected to the destination location/store only if there is an obsolete copy of this file at the destination.
Implicit Sync use case
This use case is often referred to as Sync On Checkout. It covers any synchronization that is triggered by an end-user checkout operation. The impact of bucket replication on this operation is mainly a protocol change, thus allowing the operation to benefit from a performance improvement with only minor changes to its implementation.
Special use cases
The following special use cases apply to both explicit and implicit synchronization:
  • Copy On Write Clone Files—This use case is related to the Copy On Write Store feature. The Copy Business Object command creates clone files, or a family of clone files. Clone files inside a store created with Copy On Write activated share the same physical file. Bucket synchronization of the clone files ensures that the synchronized-clone-files family still shares the same replicated physical file.
  • Directory Files—This use case is related to files created by a directory check in. Even if the actual synchronization method does not support this type of file (i.e., sync on checkout ignores them), it is still possible to check in a directory in a captured, hashed store. Bucket replication improves directory file support. Replication of directory files is supported for business object synchronization and store synchronization with the following limitations:
    • Replication of directory files is not supported for stores having a replication trigger.
    • Replication of directory files is not supported for the Sync On Checkout (Implicit Sync) use case.
  • Replication Trigger—This use case is related to the replication trigger store feature. Replication is an extra step that the customer can implement. Only the Replicate Through Store trigger is provided. This forces the store to be updated for each synchronization use case. The Replicate Through Store trigger uses the bucket replication feature.
FCSSyncServer use case
The FCSSyncServer also benefits from the bucket replication performance improvement.