Virus scanning
Moderator: General Moderators
-
aschlosberg
- Forum Newbie
- Posts: 24
- Joined: Fri Jan 23, 2009 10:17 pm
Virus scanning
I am upgrading the security of my upload class to include, among other things, virus scanning before the files are moved to a publicly accessible directory.
First question: any recommendations for a UNIX virus scanner that I can call via a command line? So far I am leaning towards ClamAV.
Second issue (may be solved by the first one though): virus scanning takes some time (Clam on my system has about 8 seconds of latency while it checks that it is up to date etc. before even scanning) which isn't really acceptable for end users who are simply uploading a profile picture. I could queue the scan and do them in batches every 5 minutes or so but this doesn't provide the instant user feedback that I need.
First question: any recommendations for a UNIX virus scanner that I can call via a command line? So far I am leaning towards ClamAV.
Second issue (may be solved by the first one though): virus scanning takes some time (Clam on my system has about 8 seconds of latency while it checks that it is up to date etc. before even scanning) which isn't really acceptable for end users who are simply uploading a profile picture. I could queue the scan and do them in batches every 5 minutes or so but this doesn't provide the instant user feedback that I need.
-
aschlosberg
- Forum Newbie
- Posts: 24
- Joined: Fri Jan 23, 2009 10:17 pm
Re: Virus scanning
Second issue resolved for ClamAV. There is a daemon available (clamd) which once running provides the ability to scan with clamdscan (as against the standard clamscan) and eliminates the latency that I mentioned before.
Any recommendations on anti virus software? Any experience with Clam?
Any recommendations on anti virus software? Any experience with Clam?
Re: Virus scanning
I usually use clamav in mail servers to scan incoming e-mails. It's good.
There are 10 types of people in this world, those who understand binary and those who don't
- kaisellgren
- DevNet Resident
- Posts: 1675
- Joined: Sat Jan 07, 2006 5:52 am
- Location: Lahti, Finland.
Re: Virus scanning
I have a better idea.
After the upload is complete, upload the file to http://www.virustotal.com/fi/ and see if they find anything.
This means less CPU processing for you, less worries and probably better virus scanning, too.
After the upload is complete, upload the file to http://www.virustotal.com/fi/ and see if they find anything.
This means less CPU processing for you, less worries and probably better virus scanning, too.
-
aschlosberg
- Forum Newbie
- Posts: 24
- Joined: Fri Jan 23, 2009 10:17 pm
Re: Virus scanning
Using a 3rd party site will only increase the latency and double bandwidth used. With 50k+ members this is probably going to cause more problems than it solves.
- kaisellgren
- DevNet Resident
- Posts: 1675
- Joined: Sat Jan 07, 2006 5:52 am
- Location: Lahti, Finland.
Re: Virus scanning
The scanner will be better than anything you can construct. You are paying bandwidth for increased security of your files and you get less CPU processing. Latency? Why do you not handle that on background. The user uploads a file, it gets uploaded. Since it happens on background, the user can close the browser and come back in 15 seconds and see that the file is now marked as clean.aschlosberg wrote:Using a 3rd party site will only increase the latency and double bandwidth used. With 50k+ members this is probably going to cause more problems than it solves.
With 50k+ members, you will have increased bandwidth and a lot safer files and no extra CPU processing.
So, make your call.
Re: Virus scanning
Closely related to the question: is it even possible to check the safety of uploaded files (or even images) by users without using a virus scanner?
I was reading
http://blog.insicdesigns.com/2009/01/se ... lications/
and it seems there just no way to check (with php only) if a file or image is safe or not.
How do sites do this normally? There are so many sites (including this one) allowing users to upload files.
(if I need to start a new thread for this, I'll do so)
I was reading
http://blog.insicdesigns.com/2009/01/se ... lications/
and it seems there just no way to check (with php only) if a file or image is safe or not.
How do sites do this normally? There are so many sites (including this one) allowing users to upload files.
(if I need to start a new thread for this, I'll do so)
- kaisellgren
- DevNet Resident
- Posts: 1675
- Joined: Sat Jan 07, 2006 5:52 am
- Location: Lahti, Finland.
Re: Virus scanning
Define "safety".matthijs wrote:is it even possible to check the safety of uploaded files (or even images) by users without using a virus scanner?
What would you like to achieve?
Re: Virus scanning
Well, prevent users from uploading an image which can be misused? And with misused I mean anything other then showing a nice picture of a cute kitten or something. I understand that with files it's a bit more complicated, as you can't possibly check every kind of file in any way. But if you'd start with basic image formats gif, jpg and png. Do you really need a virus scanner or are there other ways to check the format/safety of the image?kaisellgren wrote:Define "safety".matthijs wrote:is it even possible to check the safety of uploaded files (or even images) by users without using a virus scanner?
What would you like to achieve?
- kaisellgren
- DevNet Resident
- Posts: 1675
- Joined: Sat Jan 07, 2006 5:52 am
- Location: Lahti, Finland.
Re: Virus scanning
I doubt virus scanners are what you want. The point of them is to make sure that if the user downloads the file and runs it, they will be "safe". For instance, if it is a malicious .doc file, a virus scanner is supposed to detect a malicious use of a .doc file. For sure some file types such as .exe and .doc are not directly supported in some OS'es like Unix operating systems, but the point remains. A virus scanner is meant to protect the user, not you, not your site and not your user's account on the site. If the user wants to be safe, he should install an antivirus on his PC. It is not your responsibility really, although some web software like Gmail do scan attachments.matthijs wrote:Well, prevent users from uploading an image which can be misused? And with misused I mean anything other then showing a nice picture of a cute kitten or something. I understand that with files it's a bit more complicated, as you can't possibly check every kind of file in any way. But if you'd start with basic image formats gif, jpg and png. Do you really need a virus scanner or are there other ways to check the format/safety of the image?
You can make sure that a file is a valid JPEG file, but you cannot make sure it is only a valid JPEG file. It may be a so-called hybrid file having more than one purpose. Ultimately, you cannot make sure that a file is just and only a JPEG file, for instance. That is the reason why you can never reveal the actual filename of the uploaded file. You must upload the file in a place outside of the document root, you must rename it to something random (unpredictable and unknown). If you need to have that original filename, store it in a file or in a database, but do not use it as the actual filename.
There are plenty of problems in the web. A JPEG can steal your cookies, for example. A Java virtual machine running in the background will execute if you happen to look at a JPEG file that contains appropriate code. And using FireFox, Opera, Chrome, wtvr makes no difference. I am not sure what the latest news are. I think Adobe has patched Flash so that you can't execute Flash with a JPEG file. However, Java... well... they do not seem to care a damn thing. There are countless of other file types that may be an issue and the worst is - since you can combine them with a JPEG file, you have to protect from all of them, which is impossible (blacklisting). Ultimately, the right approach would be to upload the files on a different domain, therefore the hybrid files have no access to your website. They are provided "as is".
Re: Virus scanning
That's interesting to know. So while loading this page and looking at your avator, that could be enough to do harm?
And why is the renaming so important? If I upload my pictures to, say Flickr or any other site, I can immediately view (and therefore) execute them anyway?
And why is the renaming so important? If I upload my pictures to, say Flickr or any other site, I can immediately view (and therefore) execute them anyway?
- kaisellgren
- DevNet Resident
- Posts: 1675
- Joined: Sat Jan 07, 2006 5:52 am
- Location: Lahti, Finland.
Re: Virus scanning
First, you do not even need to look at itmatthijs wrote:So while loading this page and looking at your avator, that could be enough to do harm?
Well. "important". Depends really. It helps making sure that no one can e.g. use LFI to execute uploaded files.matthijs wrote:And why is the renaming so important? If I upload my pictures to, say Flickr or any other site, I can immediately view (and therefore) execute them anyway?
And what comes to Flickr, wtvr sites. They may output HTTP headers telling a filename, but it may not be the actual filename on the server. URL rewriting can also make you think that files/file.jpg is indeed called file.jpg on the server.
Re: Virus scanning
And why does the vulnerability depend on that stuff being installed? Is running some php and or javascript inside an image not bad enough (if possible?)kaisellgren wrote:First, you do not even need to look at itmatthijs wrote:So while loading this page and looking at your avator, that could be enough to do harm?. Second, yes it could. Provided that you have some stuff installed. May it be Java virtual machine, Flash, Flex, Silverlight, something. I am not really sure which are vulnerable which are not, but the fact is you cannot be sure what version of the software your user is running. So, even if Java has a new patch out there, how many are going to update their software?
Please excuse me, but what does " .. can e.g. use LFI .. " mean?kaisellgren wrote:Well. "important". Depends really. It helps making sure that no one can e.g. use LFI to execute uploaded files.matthijs wrote:And why is the renaming so important? If I upload my pictures to, say Flickr or any other site, I can immediately view (and therefore) execute them anyway?
I understand the mechanism you describe, but what is it that you want to say with this?kaisellgren wrote:And what comes to Flickr, wtvr sites. They may output HTTP headers telling a filename, but it may not be the actual filename on the server. URL rewriting can also make you think that files/file.jpg is indeed called file.jpg on the server.
Re: Virus scanning
LFI = local file include
This, combined with a liberal file upload script can lead to code execution - upload backdoor.php renamed as backdoor.jpg, navigate to it with the LFI:
vulnerable.php?page=../uploads/backdoor.jpg
To mitigate, we:
1. Rename with a good random name in a secret location
2. Serve it through proxy script
So now the attacker with a LFI on his hands can't find his backdoor to be included and must find another route (log files, database dumps)
Code: Select all
//vulnerable.php
include("include/" . $_GET['page']);vulnerable.php?page=../uploads/backdoor.jpg
To mitigate, we:
1. Rename with a good random name in a secret location
2. Serve it through proxy script
So now the attacker with a LFI on his hands can't find his backdoor to be included and must find another route (log files, database dumps)
- kaisellgren
- DevNet Resident
- Posts: 1675
- Joined: Sat Jan 07, 2006 5:52 am
- Location: Lahti, Finland.
Re: Virus scanning
I was talking about hybrid files. As what comes to PHP files, they are only dangerous if you run them through the parser. So, LFI vulnerabilities are one way to exploit that. No JavaScript can be ran inside a JPEG. Browsers do not do that.matthijs wrote:And why does the vulnerability depend on that stuff being installed? Is running some php and or javascript inside an image not bad enough (if possible?)
Local File Inclusion. For instance,matthijs wrote:Please excuse me, but what does " .. can e.g. use LFI .. " mean?
Code: Select all
include("$_GET[page]");That the files on the server are renamed and placed into a secret place that no one knows except for the owners. At least, this should be the case.matthijs wrote:I understand the mechanism you describe, but what is it that you want to say with this?