CSSCurrent en:Pre Processing

Aus Cryptshare Documentation
Version vom 26. November 2021, 08:49 Uhr von imported>Birkenmeierm (→‎Filter Output)
(Unterschied) ← Nächstältere Version | Aktuelle Version (Unterschied) | Nächstjüngere Version → (Unterschied)
Wechseln zu:Navigation, Suche




About Pre-Processing

Every file that is being transferred to the Cryptshare server can be checked by a pre-process command. This provides the possibility to check files for viruses before encrypting them. If a check finishes with a 'positive' exit code (i.e. a virus has been found) the file will be removed from the transfer. The Pre-Processing settings are part of the policy. That means for every policy it is possible to have a different Pre-Processing configuration.  The settings available in this menu are the defaults for creating new policy rules.

Virus check

Using ClamAV

Our Virtual Appliance and Hardware Appliance systems are delivered with a preinstalled ClamAV virus scanner.

Please note that the ClamAV virus scanner cannot scan files larger than 2 GB. This technical limitation applies only to individual files, not to the total size of the transfer. For example, a transfer with 10 files of 300 MB each can be checked, but not a transfer containing a single file of 3 GB. For performance reasons, the default configuration of ClamAV is set to only scan files up to 25M files.

74383443.png If you also want to check larger files (up to 2GB), change the `MaxScanSize` and `MaxFileSize` parameters in the `/etc/clamd.conf` file and restart the ClamAV service with the `rcclamd restart` command. 74383444.png Please refer to the Pre-Processing Limit section to define how Cryptshare should process files that are larger than the processing limit of your virus scanner.

Using other virus scanners

For self-installed systems it is necessary to integrate a separate virus scanner into the pre-processing. Please use the required command line for the pre-processing, which you can get from the manufacturer of your virus scanner. Please check the manufacturer's documentation to determine the maximum file size the scan can process. Most virus scanners have a technical limit with a file size of 2 GB. This limitation applies only to individual files, not to the overall size of the transfer.  For example, a transfer with 10 files of 300 MB each can be checked, but not a transfer containing a single file of 3 GB. Please refer to the Pre-Processing Limit section to define how Cryptshare should process files that are larger than the processing limit of your virus scanner.

Data Leakage Protection Integration

Data Leakage Protection (DLP) Tools can be integrated into Cryptshare using the pre-processing functionality. For example a tool which allows for data leak detection can be executed as a pre-processing command and be used to scan the transfer files that are being uploaded in order to remove files which are not allowed to be provided to others based on the exit code of the data leak detection tool.

Pre-Processing Command

Command and Additional Arguments

This is the command that will be used to check the file. It is necessary to only type in the command as you would do in the Windows command line or Linux console. Additional arguments for this command can be added by using the list-element right below the input field for the command. In addition to static arguments, it is possible to specify dynamic arguments that will be replaced by the respective value when files are analyzed. Currently the following dynamic arguments are available.

For technical reasons, individual special characters may be removed from parameters: " \* / : < > ? \\ |

Placeholder Example Description / Remarks
${file.name} Report.docx For technical reasons, files are passed to pre-processing with a secure file name that does not contain any information about the original filename or file extension.

This placeholder is used to pass the original file name of the file to be checked to the external program for pre-processing.

${file.encryptedName} TibavAT0qp The encrypted name of a transfer file.
${file.size} 1184 The file size of a transfer file in bytes.
${sender.email} john.doe@domain.com The email address of the sender.
${sender.name} John Doe The name of the sender.
${sender.phone} 01234 56789 The phone number of the sender.
${trackingId} 20211119-162926-AQgB58nh A Cryptshare Tracking ID is a unique ID for a single Cryptshare Transfer.

Exit Code on Successful Execution

By default, common applications use exit code '0' to indicate a negative match (i.e. no virus was found – the file can be encrypted). Nevertheless some applications use a different exit code so the pre-processing result would indicate a virus where there is none. In this case you can change the exit code setting to correct false matches.

Output of the Pre-Processing Result

Display Output

Example of pre-processing output
Example of pre-processing output

Activating the error output shows the output of the pre-processing command in the summary screen at the end of a transfer if one or more files from the transfer have been removed during pre-processing.

The following modes can be selected:

  • No output: No output is displayed. The list of files on the summary screen does not contain any further details.
  • Show output from STDOUT: The output of the pre-processing command from the standard output channel is displayed.
  • Show output from STDERR: The output of the pre-processing command from the error output channel is displayed.
  • Show output from both STDOUT and STDERR: The output of the pre-processing command from both output channels are displayed combined.

Please note that the control over which outputs are shown in which output stream depends on the pre-processing command itself. If you need more control over the output, instead of calling the preprocessing command directly, you can consider creating a shell script that calls the command and manipulates the output streams if necessary.

Filter Output

The output, which can be displayed if pre-processing is not successful, can additionally be filtered and replaced in order to customise the output. To do this, a search pattern can be specified which is evaluated as regular expressions (regex). The output is then ran against this pattern and checked for matches. The matches are then replaced based on the replacement pattern. If a regular expression with a group was used, it can be reused when replacing it:

Description Original output Pattern for filtering Replacement pattern Result
Some simple 1:1 text replacement.
Some other text
Some other text
Yet Another Text
Yet Another Text
A part of a text is replaced by another, but a certain part is taken over.
Some other text. Virus found in file:test.exe 
.*Virus found in file:(?<fileName>.*)
File was removed due to a virus: ${fileName}
File was removed due to a virus: test.exe
A part of a text is removed.
Some other text
text
Some other
Filter a multi-line output by a specific line.
Some other text.
Virus found in file:test.exe
Addional text.
(?m-)^(?!Virus found in file).*?\R
Virus found in file:test.exe

For more information about the syntax of regular expressions in Java please look at the Java Runtime documentation.

To simplify the construction of a proper regular expression, we recommend the use of a test application such as regex101.

File Size Limit

As some virus scanners are not capable to handle files beyond a specific size, a size limit for pre-processing files can be set. With this setting, size beyond this size will either be removed from the transfer or excluded from pre-processing depend on selected action. Please refer to the according context help in the Administration Interface for further details.

Performance impact when scanning larger files.
Please note that increasing the limit for the file size to be scanned in the configuration of your virus scanner can have a considerable effect on the performance of the system, depending on your usage behavior. The size limitation of 25 MB has been set by the manufacturer of the anti-virus system, since viruses are usually spread over small files to spread them further, but seldom over very large files.

Email Notification

The administrator can be informed via email if pre-processing has excluded one or multiple files from the transfer.

13 Vorverarbeitung EN.png

Conflict Solution

For the conflict solution of Pre-Processing settings the following rules are active:

  • Exit Code, Pre-Processing command or command Arguments

→ If at least one of the three parameters causes a conflict, all three global settings available in this menu are used.

  • Pre-Processing Limit

→ If at lease one policy has the setting for removing the file from the transfer, if preprocessing fails, this setting is used. → When the setting 'Use the maximum value for the Pre-Processing file size limit.' is selected in conflict solution menu, the maxium value will be used, otherwise the minimum value.

  • Notification

→ If the notification is activated for at least one policy, the notification will be send.