I am the local Cable Operator chillpill_rohit is talking about.
GIven below is a simple procedure that our network with the help of Rohit and some invaluable input from Anjanesh (who devoted his valuable time coming down to Andheri - a suburb in Mumbai - last sunday) and together brainstormed into finding a solution. Below is how we proceeded thereafter.
Ours is a multi ISP network, being serviced by Sify, Pacenet, IOL Broadband, IN Cablenet, Tata and Syscon Technologies. Interestingly. we were getting network disconnection problems only in the IOL Broadband network, and so our primary suspicion was on users who had taken packages from IOL, and to a greater extent the ISP and the servers deployed therein. Also, we do not use managable switches.
Our first test was disconnecting the netire network and isolating one PC and connecting it directly with the feed from the ISP. When we did that, we realised that the injected javascript just disappeared. We loaded the network and connected my machine again from the network switch, and voila, the javascript and the disconnections appeared.
This atleast gave us a direction so as to where to look into.
Next, with the help of Rohit, and our tech team, we started seeking out machines showing unusual behaviour. What we discovered is that some machines were showing extremely high ARP requests, Since our network has a log of which ips are given to which users, and the sniffers gave us indications of which ip addresses are throwing ARP requests, we simply identified the users, and our tech team visited their place.
When we visited the first place, what we found is that that machine was NOT using any anti-virus protection. We installed a Trial Version of NOD 32 (they have a fully working 30 day trial version, can be downloaded from
http://www.eset.com)
When we installed NOD 32 on his system and scanned his system , we discovered around 1700 milacious infections (including variants of trojans). We got a log file from the machine in order to study whether ADS.EXE or ADS.JS was found.
Though there was no trace of any such file, we discovered, that the moment the machine got clean, the ARP request stopped comming from that machine declogging the network. For almost an hour, the network was absolutely clear, when we noticed the js appearing again.
We followed the same procedure again, and we discovered 2 more ip addresses - following the same pattern. We isolated the machines from the network, and installed and sacnned their machines using NOD 32, and the ARP requests stopped comming from those machines.
From Sunday - January 20, 2008 through today, January 23, 2008, we managed to identify around 11 machines.
What we found was -
of the 11 machines, 6 of them DID NOT HAVE ANY ANTI VIRUS
3 of them had a NON UPDATED Antivirus - as good as having no Anti virus
and 2 of them having some free antivirus which could not detect any viruses, which were detected when e installed NOD 32.
Our procedure was simple. Identify rogue ips, isolate machines of those ips, educate people using those machines about what is happening (usually clients when informed, co-operate), installed the anti-virus, and scanned the machines, and EACH OF THE MACHINE WE SCANNED, we found viruses.
At of the time of writing this, I am relieved to say that around 90% of my network is now back to full function, barring scant instances of the js floating. However, the frequency of disconnections has reduced drastically, and we strongly believe that as the week goes by, we would be able to put a stop to this js once and for all.
Two important things here.
1. Our investigations showed, that even though it was just IOL broadband subscribers whose internet faced frequent disruptions, the infected machines were not limited to users using IOLs service. We got infected machines of subscribers using Sify and Syscon too. What we are now evaluating is, why is it that only one ISP was suffering breakdowns, inspite of machines using alternative services too were infected. I shall get a resolution of this too soon.
2. The machine NEED NOT be connected to the internet for sending these ARP requests and clogging the network. Even if an infected computer is merely switched on, the process of clogging and JS injections start. More on this too soon.
I hope this post of mine may help those infected networks by way of the affected users being able to convince their service providers, and all of us able to use the internet peacefully.
Lastly, without the help of our ISP - IOL, Mr. Rohit, and Mr Anjanesh, and our tech team, I would not have been able to share what i just did.
All the best.
Aashish
Admin - CIES (Andheri West)