Extending the Files Connector Through Plugins

The filesystem connector embeds natively a number of schemes and protocols: native filesystem, Windows share filesystem (\\path or smb:// URLs), basic ftp support (ftp:// URLs), basic http (http://), HDFS (hdfs://), etc.

It is possible to extend the features of the filesystem connector and use all the embedded features of the connector (multithreaded scan, containers handling, etc.) without having to create a new connector. You can do so by implementing additional protocol schemes through plugins.

Before going for a deep dive into customization, you must check whether:

  • The source APIs are compatible with the FileInterface interface.

  • The connector relies on hierarchical sources.

Therefore audit the source APIs to see whether you can crawl the source with a class implementing FileInterface. If it is OK, you can customize the connector. See the complete implementation procedure Extending the Files Connector through Plugins in the Exalead CloudView Connector Programmer's Guide.