Configuring Folders to Monitor to Automatically Import Files

You can configure the importer to monitor a folder and trigger imports when files are placed in the folder.

This feature can be useful in cases when a company has:

An external enterprise BOM system from which BOM data is periodically exported and then imported into Materials Compliance Management
A procedure for importing supplier material declarations without using the Materials Compliance Supplier Portal or IMDS

Configuring this feature requires operating system administrator rights to create the folders, set operating system-level access restrictions on them, edit web server configuration files, and to start/stop the web server. Placing import files into the folders requires operating system-level write access to the monitored folders.

This task shows you how to:

Define the Folder Hierarchy
Enable the Timer Servlet
Configure the Monitor Folder Property

Define the Folder Hierarchy

Each monitored folder must have a folder structure as defined here, and each is completely independent of the others. Each could be owned by different OS users with different ACLs, and different configuration files and options suitable for import files from different sources.

The recommended configuration is for the monitored folder to be on a file server, independent of the web server, and also independent of any user's workstation. The user (or external BOM system) pushes files from their workstation into the monitored folder in the file server, and the Materials Compliance Management server pulls from the monitored folder into the database.

Any user who has write access to a monitored folder can import without logging in to Materials Compliance Management, and the person listed in the MonitorFolders property (Configure the Monitor Folder Property) shows as the person who imported the file. Carefully consider who has write access to the monitored folder. Each monitored folder should have its OS level ACLs set so that only a specific user or user group has write access to each folder. The Materials Compliance Management server itself also needs to be able to have write access. Although an end-user and the Materials Compliance Management server both access the same folder on disk, the user does not have access to any server files.

Define the folder to monitor. You can use any name for this folder. Create these subfolders, using the exact names shown, within that folder with the listed accesses. If any of these conditions are violated, the folder monitoring function will fail, and error messages will be sent to the server log.


Folder Name	Folder Access	Description
`Config`	OS-level read access to the folder and the files in it.	Can contain these files (both are optional, but if included, they must have the exact names shown): `pn_filter.dat` used to filter which parts will be imported. See Step 2. `params.dat` used to define parameters that control how the data is imported. See Step 3.
`Submitted`	OS-level read and write access to the folder.	The folder where BOM files are dropped so that Materials Compliance Management will pick them up. The app periodically looks for any files present in the submitted folder. When any such files are found, they are immediately moved to the `Processing` folder.
`Processing`	OS-level read and write access to the folder.	Immediately upon discovering a new file in the submitted folder, the monitor folder process moves it to the `Processing` folder, where it remains until processing of that file is complete. This move avoids the same file being re-discovered in the submitted folder at the next monitor poll period if the import of this file has not yet finished. If the import fails, the import and log files are moved to the `Failed` folder. If the import succeeds, the action taken depends on the `Archive` parameter in `params.dat`. (See Step 3.) Never manually put files in or remove files from this folder.
`Failed`	OS-level write access to the folder.	If an import fails, the import file and the resultant import_log.txt file are zipped together and placed in the `Failed` folder. The name of the zip archive is the same as the submitted import file's name, prepended with a date stamp, and appended with ".zip". You should periodically clean up this folder; Materials Compliance Management never removes anything from the `Failed` folder.
`Archived`	OS level write access to the folder.	If an import succeeds and archiving is enabled in the `params.dat` file (see Step 3), the import file and the resultant import_log.txt file are zipped together and placed in the `Archived` folder. The name of the zip archive is the same as the submitted import file's name, prepended with a date stamp, and appended with ".zip". You should periodically clean up this folder; Materials Compliance Management never removes anything from the `Archived` folder.

If you want to filter the parts to be imported, create a pn_filter.dat in the Config folder. The file must have this exact name.

This file uses the same format as described in Filtering Part Numbers When Importing and performs the same function. If not present, no part number filtering is done. If it is present, the option PNFilterMode in params.dat (Step 3) must specify the PNFilterMode parameter to determine how the filter is to be applied.

To configure the import, create a params.dat in the Config folder. The file must have this exact name.

The file defines parameter names and values that control various import actions. You can include any of the emxMaterialComplianceCentral.importer.* properties described in Configuring the XML Importer. When entering a parameter, only include the last part of the property name. For example, the property emxMaterialComplianceCentral.importer.fuzzySupplierMatching=false would be entered as parameter fuzzySupplierMatching=false. You only need to enter parameters if you want to use different values than those defined by the properties in emxMaterialsComplianceCentral.properties.

In addition, you can include these parameters in the params.dat file:

Parameter Description

PNFilterMode

If you filter part numbers using the pn_filter.dat file, you must also include this parameter in the params.dat file. If a pn_filter.dat file is found, and no PNFilterMode option is defined, the import fails.

This parameter can take one of these values:


Value	Description
`IncludeL1PNs`	Only the top level parts whose part number exists in the filter file are filtered.
`ExcludePNs`	All parts whose part number exists in the filter file are excluded from the import.
`IncludeL1CPNs`	Only the top level parts whose customer part number exists in the filter file are imported.
`ExcludeCPNs`	All parts whose customer part number exists in the filter file are excluded from the import.

conflictMineralRegulationIds

Defines a comma-separated list of the IDs of the conflict mineral regulations to be imported. This property must be manually entered in the params.dat file.

To define the US Conflict Minerals (2014) regulation, you would enter this property:

conflictMineralRegulationIds=US-2014

For reference, the regulation IDs can be found in the ConflictMineralConfiguration.xml file.

Archive

Determines the action to take after successfully importing a file:


Value	Description
`true`	Files are archived in the Archived folder.
`false`	The import file and the log file are deleted. If an import job completes successfully, but with warnings, the log file containing the warnings is also discarded.

For either value, a copy of the import file and log file are attached to the import job object. If not specified, the default value true is used. This parameter has no effect on files that fail.

Define the user in whose name the imports will done. This user will become the owner of all imported files. To do so, follow these steps:
1. Create the user in whose name the imports will be done. (This user will also be assigned as the value for the ematrix.timer.agent parameter in the web.xml file.
2. Assign this user the Materials Compliance Management (MCM) license.
3. Add this user with full access to all the folders created in Step 1.
4. If you use the baseline access roles, assign this user to all sets of credentials that will be added to the emxMaterialsComplianceCentral.importer.MonitorFolders property (described below).

Enable the Timer Servlet

The timer servlet is installed with the Collaboration Server but is commented out. Follow these steps to uncomment the servlet and its parameters.

If you are also updating JSPs, open this web.xml file for editing:
```
C:\enovia<VERSION>\server\STAGING\ematrix\WEB-INF\web.xml
```
If you are only configuring the timer servlet, open this web.xml file for editing:
```
C:\enovia<VERSION>\server\distrib\enovia\WEB-INF\web.xml
```
To enable the servlet, add the following lines.
```
<servlet id="Servlet_19">
<servlet-name>TimerServlet</servlet-name>
<servlet-class>com.matrixone.servlet.TimerServlet</servlet-class>
<load-on-startup>2</load-on-startup>
</servlet>
```
```
<servlet-mapping id="ServletMapping_16">
<servlet-name>TimerServlet</servlet-name>
<url-pattern>/servlet/timer/*</url-pattern>
</servlet-mapping>
```
Note: Make sure there isn't already a servlet id="Servlet_19", or "ServletMapping_16" in the web.xml file. If there are, use the next number available for each id. These numbers are simply identifiers, so it does not matter what they are but they cannot be duplicated in the file.

Add these lines to enable the servlet to monitor folders:

<context-param id="ContextParam_14">
<param-name>ematrix.timer.agent</param-name>
<param-value>creator</param-value>
</context-param>

<context-param id="ContextParam_15">
<param-name>ematrix.timer.agent.key</param-name>
<param-value></param-value>
</context-param>

<context-param id="ContextParam_16">
<param-name>ematrix.timer.interval</param-name>
<param-value>10</param-value>
</context-param>

<context-param id="ContextParam_17">
<param-name>ematrix.timer.command</param-name>
<param-value>execute program jpo.materialscompliance.importer.ImporterMonitor -method pollConfiguredRootFolders</param-value>
</context-param>

Note: If other apps have defined parameters, make sure there isn't any "ContextParam_n" in the web.xml file. If there are, use the next number available for each id. These numbers are simply identifiers, so it does not matter what they are but they cannot be duplicated in the file.

In the above entries, edit the username, password, and timer interval values between the corresponding <param-value> and </param-value> tags. The interval value is in seconds.

For ematrix.timer.agent, enter the username the servlet should use to set context in MQL. This should be a creator who has privileges to run the program.
Note: This user must have a license assigned for Materials Compliance Management.

For ematrix.timer.agent.key, enter the password for the username, if there is any. For the creator user, for example, there is no password.
Save the web.xml file.
If you updated the web.xml file in the STAGING directory, run the warutil to create the archive files that include the updated web.xml file, then copy the enovia folder from distrib to the Application folder.

Configure the Monitor Folder Property

You need to define the folders to be monitored by the servlet.

Create or open the text file to contain customized properties. For more information, see Editing Properties Using MQL.
To configure folders to monitor, follow these steps:
Folder monitoring is enabled at server startup if this property value is non-empty.
1. Add or edit this line in the file used to import properties into a page object:
```
emxMaterialsComplianceCentral.importer.MonitorFolders
```
  The format of the value of this property depends on whether you use the baseline access roles (Author, Leader and so on) or the app-specific roles (Compliance Engineer, Senior Compliance Engineer, and so on).
2. For baseline access roles, set the value for this property using this format:
```
<folderpath1>#<user1>::role.organization.collaborativespace
#<folderpath2>#<user2>::role.organization.collaborativespace
```
  You can specify more than one monitor folder. For each monitor folder, you must identify the name of a user assigned as the owner of the import jobs initiated for the monitor folder including the full credentials (role.organization.collaborativespace). The path names must be absolute path names, as the Materials Compliance Management server would see them (the mount point or drive letters from the server’s point-of-view).
  
  For example:
```
emxMaterialsComplianceCentral.importer.MonitorFolders=\
        W:/BOM_IMPORT#R. Smith,\
        K:/ENG/Compliance/MDS_DROP#Compliance Engineer 1
```
  In the above example, two folders will be monitored. Imports for files found in the first folder will be performed as user R. Smith, and that user will own any objects created by that import; similarly, the user Compliance Engineer 1 will own any objects created by an import through the second monitored folder.
3. For app-specific access-roles, set the value for this property using this format:
```
<folderpath1>#<user1>#<folderpath2>#<user2>
```
  You can specify more than one monitor folder. For each monitor folder, you must identify the name of a user assigned as the owner of the import jobs initiated for the monitor folder. The path names must be absolute path names, as the Materials Compliance Management server would see them (the mount point or drive letters from the server’s point-of-view).
  
  For example:
```
emxMaterialsComplianceCentral.importer.MonitorFolders=\
        W:/BOM_IMPORT#R. Smith,\
        K:/ENG/Compliance/MDS_DROP#Compliance Engineer 1
```
  In the above example, two folders will be monitored. Imports for files found in the first folder will be performed as user R. Smith, and that user will own any objects created by that import; similarly, the user Compliance Engineer 1 will own any objects created by an import through the second monitored folder.
When finished updating the text file with all properties you want to modify, including the ones shown here, use the text file to modify the emxMaterialsComplianceCentral.properties page object. For more information, see Editing Properties Using MQL.