In a different thread Drew posted a blog I wrote concerning the false positive known as the EICAR test file.

 

Some people were of the opinion that the EICAR test file was useless. Mostly because they were giving examples of use cases that the EICAR test file is not designed for. That really speaks to the problem of people drawing unsubstantiated conclusions from the results of a test that uses the EICAR test file.

 

So, here is a valid use case with an analysis of correct, incorrect, and partial conclusions that can be drawn from this specific use case. We will assume I am using a scanner that is designed to detect the EICAR test file. This actually is a real world use case.

 

The scanner(s) I am using allow me to set the number of levels of nested compression scanned from 0 to 7. For balance of performance vs security preferences I want the level to be set to 3 levels of nested compression. I adjust the setting and then I want to run a test to see if the settings are functioning as intended. I could use a piece of malware in the test, but why? The EICAR test file is harmless and sufficient to perform the test. Step one is to create or download the EICAR test file and scan it. If it isn’t detected I have to get that fixed before I proceed. Perhaps the file itself was created incorrectly. Perhaps it was corrupted on download. Perhaps the scanner is malfunctioning. OK this test has passed. Now I create my test sample – a compressed file containing a compressed file containing a compressed file that contains the EICAR test file.

 

Now I scan and the test file is detected. What can I conclude from this test? I can conclude that the scanner successfully scanned through three layers of compression. What would be some of the incorrect assumptions?

 

• The scanner can scan through all types of compression
  
• The scanner stops at three levels of nested compression
  
• The scanner would detect a virus I put in the compressed files. It won’t if it can’t detect the file in the first place.
  
• The scanner will detect the test file in one or two layers of compression
  
• If I use multiple types of compression formats that the scanner will work the same through all iterations of the compression sequences.

For the purposes of my test, I must test a file with four layers of compression or I haven’t really performed the test I needed to do to get the answer I intended to find out. Incorrect conclusions #1 and #3 would mean I have been lulled into a false sense of security. Conclusion #2 is invalid for the previously stated reason. While not empirically validated, conclusion four is an unlikely scenario, but since I already have created the single and double level compressed files I may as well test them too. #5 is really quite pedantic. It is an unlikely but not impossible scenario. Whether or not I test that scenario depends on how paranoid I am.

 

Now for security preferences I am setting the levels of nested compression to 7. I scan and the file is detected. Correct conclusion – the scanner successfully scanned through 7 levels of nested compression. All of the above incorrect conclusions remain.

 

In a job I once held, I actually needed to test to see if the maximum level of nested compression were actually being scanned through. I also needed to test against multiple types of compression types.

 

There are a number of valid and useful real-world scenarios for the EICAR test file. There are even more ways to draw incorrect conclusions depending upon what you intended to test for.

 

The EICAR test file could not be more useless for testing the efficacy of a security product, but that does not mean the file is useless.