FCS File Validation

Files may be reported as missing as a result of data migration errors, bugs in the File Collaboration Server (FCS), or other unknown reasons. FCS file validation allows you to perform a health check on files that are referenced by the database to ensure that they are not missing.

Use the validate fcsfileexist command to perform a file data integrity check to ensure that the FCS files in the file system are, in fact, referenced by the 3DEXPERIENCE platform database. Running this command on a regular basis can prevent File Not Found errors from being reported by the FCS.

This topic covers the following items:

This page discusses:

User Level
Syntax
Output
File consistency validation
Periodic file store administration for orphan files
Restoring a File
Performance
Backward Compatibility

User Level

Business Administrator

Syntax

To optimize file data integrity check, generate the index file with the following MQL command:

mod store <name_store> addfileindex [thread <number> <parallel_mode>];

If the thread keyword is not set, there is no parallelization and only one thread is processed.

If thread keyword is set to x, x threads will be created. The maximum authorized value is 200.

By default, parallel mode is set to prefix, it also can be set to any.

The number of threads created can’t exceed the number of directory parallelizable.

Example: If parallel mode = prefix, thread number = 100 and store contains 4 prefix, 5 threads will be created: 1 per prefix + 1 for store files without prefix.

Note:

If the index has already been generated less than 7 days before, the previous command will update the index in order to do not regenerate all the data which are still valid.

To perform a file data integrity check to ensure that files that are expected to exist are, in fact, present in the database, use the validate fcsfileexist command with the following syntax:

validate | bus TNR fcsfileexist          									 |
         | buslist QUERY fcsfileexist              |
         | store STORE fcsfileexist [prefix PREFIXNAME] [file FILEFILE] [indexmode]|
         | location LOCATION fcsfileexist [prefix PREFIXNAME] [file FILEFILE] [indexmode]|
         ;

To find and display orphan files without deleting them, use the validate fcsorphanfileexist command with the following syntax:

validate | store STORE_NAME fcsorphanfileexist [prefix PREFIXNAME]								|
         | location LOCATION_NAME fcsorphanfileexist [prefix PREFIXNAME]		|;

To find, display, and delete orphan files use the validate fcsorphanfiledelete command with the following syntax:

validate store STORE_NAME 
        |file FILENAME                                                     |;
        |fcsfileexist [prefix PREFIXNAME] [file FILEFILE] [indexmode]  |
        |fcsorphanfileexist [prefix PREFIXNAME]                      |
        |fcsorphanfiledelete [prefix PREFIXNAME]                           |

validate location LOCATION_NAME 
       |fcsfileexist [prefix PREFIXNAME] [file FILEFILE] [indexmode]    |;
       |fcsorphanfileexist  [prefix PREFIXNAME]                      |
       |fcsorphanfiledelete [prefix PREFIXNAME]                        |

Note: To improve the performance of data consistency checking, the optional prefix for a command narrows the check to the prefix given. The optional prefix is the same as the following command:

print store s select prefix;

Output

The output of the validate fcsfilexist command can be one of the following:

File Size Check

This command also performs a file size check and reports files where the current size is different from the database record.

MQL<4>validate store s fcsfileexist;
FAILED: validating existence of file  alias tenant_global/bf/35/bf35nlx09pabaui0ueb4euh75vbn8zdqu61ug2mhie6.rsw at store s location ADMINISTRATION
FAILED: validating size of file  alias tenant_global/09/f1/09f161s1txvfxqhrjxbc0wmjvt4gixt56msominzchx.ag4 at store s location ADMINISTRATION
2 missing file instance(s) detected.

No missing files

If there are no missing files, there is no output (the command is silent).

MQL<n>validate store s fcsfileexist;

Missing files

If there are missing files, the output is as follows:

MQL<n>validate store s fcsfileexist;
FAILED: validating existence of file alias 86/31/8631f0ffq1xt9mjax9mwpdezujlp7pb_8lz2csyyu-9.cy2 at store s location ADMINISTRATION
1 missing file instance(s) detected.

Note: The validate fcsfileexist command simply sends a remote message to the FCS to check the existence of a file. It does not perform any data correction.

File consistency validation

To check consistency related to files size, checksum and fileID in the LXFILE table run the following commands:

validate bus … fcsmetadata
validate buslist … fcsmetadata
validate store … fcsmetadata

Periodic file store administration for orphan files

We recommend that you perform a periodic cleanup of FCS orphan files. Orphan files are files uploaded via FCS that do not have database references. In order to save disk space, orphan file should be removed periodically (for instance once a month).

Use the validate fcsorphanfileexist command to find, and display orphan files, files created by FCS that do not have database references. Use the validate fcsorphanfiledelete command to find, display, and remove orphan files. These commands do not process files created less than 24 hours ago that do not have a database record.

Restoring a File

FCS can maintain multiple copies of files on FCS remote servers. If there is a single point of corruption, for example, a file instance is missing or corrupted, and there is a healthy copy of the file instance on a different location, you can restore the health of a file by removing the corrupted file instance from the system.

Note: Restoring files is an advanced administration task and you should be an Administrator to perform this task.

To restore a file, at least one healthy and non-obsolete copy of a file must exist. You can collect information about missing or corrupted files by creating a file restore list using the validate command. Then you can use the restore command to remove the corrupted file instance from the system.

The file restore list contains multiple records of FCS files that require restoration. The following six values, each on a separate line, represent one file in the file restore list:

Business Object ID
Format ID
Store ID
Location ID
FileName
HashName

To collect information about missing or corrupted files, use the following syntax to create a file restore list:

validate | store fcsfileexist      
 | location fcsfileexist   
 | bus fcsfileexist        
 | buslist fcsfileexist    
 | bus fcsdbchecksum    	
 | buslist fcsdbchecksum [file FILENAME];

where:

FILENAME is the output directory path of the file restore list.

To remove the corrupted data when a valid copy is available on a remote site, use the following syntax:
```
restore fcsfile  file FILENAME [simul] ;
```
For example:
```
MQL<3>restore fcsfile file c:/bo1.txt ;

Business Object: t bo1 0
Format: f
File Name: test_file.txt
Store: s1
Location: s1l1
Hashname: 26/36/26360shxyjalxwdfkapqlf2tvrc4pvy27ojgaqdbsyd.jni
restored
```
The restore fcsfile command removes only the corrupted data from the system (database record and the physical file) if there is another file instance in a different location. File synchronization is not part of the restore command.
Optionally, synchronize the data using the MQL commands sync store, sync location, sync bus, or sync businessobjectlist.
If you do not do an explicit sync, then implicit synchronization or synchronization-on-demand takes place when someone on the related site checks out the file.

Performance

These commands send remote messages to the FCS. Therefore, network speed and latency affect its performance. Checking data consistency between a database and the file system can take a lot of time (hours), depending on the number of files in the system and on the IO performance of the disk sub-system. However, the existence check that takes place at the FCS does not have much impact on performance.

For the validate bus|buslist fcsfileexist command, the remote messages are grouped by each business object and location.

Backward Compatibility

If a validate fcsfileexist message is sent to an FCS version older than V6R2014 and the health check mechanism is not completed, the command assumes that there is no missing file and reports the result accordingly.