Tracking a user by IP - concerns

Discussions of secure PHP coding. Security in software is important, so don't be afraid to ask. And when answering: be anal. Nitpick. No security vulnerability is too small.

Moderator: General Moderators

User avatar
flying_circus
Forum Regular
Posts: 732
Joined: Wed Mar 05, 2008 10:23 pm
Location: Sunriver, OR

Tracking a user by IP - concerns

Post by flying_circus »

One of the things we can try to do to mitigate brute force attacks is to log users IP Addresses and run a query to determine if they are trying to access our website too frequently. As in, tipping us off to an automated script and not a normal user activity pattern (example: too many login attempts within a short time frame).

It is entirely possible (and likely), that our users will go through a proxy server at some point between their browser and our site. Typically you would grab a users IP address from the $_SERVER['REMOTE_ADDR'] predefined variable. If the user goes through a proxy server, $_SERVER['REMOTE_ADDR'] will be the proxy server's address, potentially setting many "user's" IP addresses to the same value, making IP address restriction quite a bit more... effective, than we desire. *optionally, the proxy may add the X_FORWARDED_FOR header, in a comma-delimited format, with the final value being that of the end user's. Cool, but it should be noted that it is not required for the proxy server to set this header.

Assuming the proxy server provides the extra header, the curve-ball comes with how we validate an IP. Typically, we could use long2ip((ip2long($ip_address))). Now, I'm not sure if this is OS dependant, but an invalid IP address results to 0.0.0.0 on my dev box (Ubuntu). Would you simply not allow an ip address of 0.0.0.0, assuming that all proxy servers should provide the additional header AND a valid IP? This leaves the door open to deny access to valid users because of the way the proxy server(s) handle requests. However, if you did allow the user in, based on an ip address of 0.0.0.0, we can manually set the X_FORWARDED_FOR header to something like 'a' and bypass our attempt to filter IPs.

Even if we restrict an invalid IP address, we have no method of determining if $_SERVER['REMOTE_ADDR'] is a proxy or end user. We would have to assume that, if the X_FORWARDED_FOR header is set, that $_SERVER['REMOTE_ADDR'] is a proxy. Once again, the X_FORWARDED_FOR header can be set by the user, which is trivial to modify but difficult to account for in our code.

In other threads, Kai and Vlad had mentioned tracking a user's IP address on each request. Vlad, I believe, recommended mod_evasive. Kai, I belive, recommended filtering IPs assuming that the trailing 2 octets would not change significantly.

It seems like such a rickety method of tracking users and with more pitfalls than benefits. Perhaps I am missing something? I realize that there is no concrete answer, and it's mostly a game of probability. What are your thoughts?
Last edited by flying_circus on Tue Jan 26, 2010 10:22 am, edited 1 time in total.
User avatar
Luke
The Ninja Space Mod
Posts: 6424
Joined: Fri Aug 05, 2005 1:53 pm
Location: Paradise, CA

Re: I've got a dumb question

Post by Luke »

I read your question and unfortunately I don't have much help for you, but I do need to ask you to change your post title. Our rules state that you must provide a detailed summary of your post in the title, and "I've got a dumb question" doesn't quite fit the bill. Thanks :)
User avatar
VladSun
DevNet Master
Posts: 4313
Joined: Wed Jun 27, 2007 9:44 am
Location: Sofia, Bulgaria

Re: I've got a dumb question

Post by VladSun »

We had another discussion recently on preventing brute force attacks: viewtopic.php?f=19&t=109569
Josh's approach looks good to me - tracking the login attempts per user and insisting a CAPTCHA enabled login after a predefined "login_attempts/time_period" threshold is reached.

Lately, I don't use mod_evasive because my applications use a lot of AJAX calls and it's always the front controller addressed - i.e. it's always the http://mysite.com/index.php requested. That's why mod_evasive is hard to be tuned properly.
There are 10 types of people in this world, those who understand binary and those who don't
User avatar
flying_circus
Forum Regular
Posts: 732
Joined: Wed Mar 05, 2008 10:23 pm
Location: Sunriver, OR

Re: I've got a dumb question

Post by flying_circus »

@ Luke: You're right, I will modify it. All the cool kids seemed to be doing IP checks, so I thought I was truly missing something, as it doesnt add up, to me.

@ Vlad: Thanks, I will read that thread at some point today and give it some more thought. I have an idea of what will be in that thread, which leads to more concerns, but let me read it before I post.

I was also going to ask you about mod_evasive. Being that I pay for hosting on a shared host who doesnt have it installed, is there a way to load it dynamically or is it simply not available to me? I tried to find an answer on google a few weeks ago, and it seemed like I couldnt do it.

Thanks
User avatar
kaisellgren
DevNet Resident
Posts: 1675
Joined: Sat Jan 07, 2006 5:52 am
Location: Lahti, Finland.

Re: I've got a dumb question

Post by kaisellgren »

You can't trust X_FORWARDED_FOR at all. It can be sent by anybody. The only good source to use is REMOTE_ADDR.

Logging IPs directly is effective, but you will soon notice that your database is filled with rows...

As Vladsun mentioned, you can try placing a counter in the "users" table that you increase per each invalid log-in attempt and clear every time the user successfully logs in. After there have been too many invalid log-in attempts, show a CAPTCHA. The stupid thing about this situation is that the most important/popular accounts will always have a CAPTCHA, thus, it would make more sense to show the CAPTCHA on the page regardless without thresholds.

Preventing brute forcing comes down to either A) blocking specific clients or B) blocking specific accounts. The IP address is the only reliable source when you block certain clients and the ladder relies on a control that makes brute forcing a lot harder, but not impossible (e.g. a CAPTCHA).
User avatar
timWebUK
Forum Contributor
Posts: 239
Joined: Thu Oct 29, 2009 6:48 am
Location: UK

Re: Tracking a user by IP - concerns

Post by timWebUK »

@Kaiselgren

Surely following VladSun's method would not be too much of an issue (with the popular/important accounts) if you do not display any usernames on the site, only use specific display names, and the usernames are solely used for logging in? Thus remaining completely unknown giving the Bruteforcer nothing to work with - unless they coax the username out of the user themselves or find it out by other means?
User avatar
flying_circus
Forum Regular
Posts: 732
Joined: Wed Mar 05, 2008 10:23 pm
Location: Sunriver, OR

Re: I've got a dumb question

Post by flying_circus »

kaisellgren wrote:Logging IPs directly is effective, but you will soon notice that your database is filled with rows...
I do want to touch on this subject, but maybe a bit later. I am at work, and since I dont work for a development company, the boss is less understanding about surfing development forums :)

kaisellgren wrote: As Vladsun mentioned, you can try placing a counter in the "users" table that you increase per each invalid log-in attempt and clear every time the user successfully logs in. After there have been too many invalid log-in attempts, show a CAPTCHA. The stupid thing about this situation is that the most important/popular accounts will always have a CAPTCHA, thus, it would make more sense to show the CAPTCHA on the page regardless without thresholds.
What is the benefit to CAPTCHA? As far as I am aware, CAPTCHA is a method to verify the request came from our site, and that the user is human (with reasonable success). If the aim is to throttle login requests on a per-account basis by forcing the user to take a minute to read and then type a code, wouldn't it seem less intrusive to use form tokens, increment a sleep timer, and/or finally lock the account? It would seem a sleep timer would be more painful to someone actually trying to be malicious.

kaisellgren wrote: Preventing brute forcing comes down to either A) blocking specific clients or B) blocking specific accounts. The IP address is the only reliable source when you block certain clients and the latter relies on a control that makes brute forcing a lot harder, but not impossible (e.g. a CAPTCHA).
A) I understand that blocking an IP is a valid method of restricting access, but it would seem it's easy to restrict access to too many users, which was my original concern. For example, if I were to be an internet menace, I surely wouldn't do it from my home pc. One might head to their local coffee shop and bounce around to other coffee shops if they got restricted and were motivated, all the while denying access to others trying to connect from the coffee shop who's IP was just blocked. That's an extreme example compared to something a bit more common, like say, connecting through the proxy at work or your university. There must be some merit to this method though, as most message forums implement an IP ban.

B) I believe this is the same as my proposed idea above. Something you know (username & password), something you have (token, session cookie), and maybe even, something you are (user_agent or a hash of a collection of headers, such as user_agent, Accept-Encoding, etc). (I've been reading Ilia's book after you recommended it) ;)


Regardless, thanks to everyone for taking a little time to consider my ramblings and for providing feedback. I have many other dumb questions on this subject that I will probably bounce off of you in the near future.
User avatar
kaisellgren
DevNet Resident
Posts: 1675
Joined: Sat Jan 07, 2006 5:52 am
Location: Lahti, Finland.

Re: Tracking a user by IP - concerns

Post by kaisellgren »

timWebUK wrote:Surely following VladSun's method would not be too much of an issue (with the popular/important accounts) if you do not display any usernames on the site, only use specific display names, and the usernames are solely used for logging in? Thus remaining completely unknown giving the Bruteforcer nothing to work with - unless they coax the username out of the user themselves or find it out by other means?
It's worth noting that someone brute forcing log-in forms can first harvest usernames based on whether you show a CAPTCHA on the log-in form. Remember, the CAPTCHA was displayed if the specified account has too many log-in attempts. If you do it the other way around (i.e. limit attempts per username request), then you introduce an irritating problem that you face with IP address limitations -- a database filled with records. Whether it's a problem or not is up to the developer to decide.
flying_circus wrote:What is the benefit to CAPTCHA? As far as I am aware, CAPTCHA is a method to verify the request came from our site, and that the user is human (with reasonable success).
The CAPTCHA is there to prevent automatic brute forcing attempts.
flying_circus wrote:wouldn't it seem less intrusive to use form tokens, increment a sleep timer, and/or finally lock the account?
Form tokens do not prevent bots and locking accounts can lead to account lockout attacks.
flying_circus wrote:A) I understand that blocking an IP is a valid method of restricting access, but it would seem it's easy to restrict access to too many users, which was my original concern.
Yes, it potentially restricts users from logging in when they use a proxy or a cafeteria computer.
flying_circus wrote:B) I believe this is the same as my proposed idea above. Something you know (username & password), something you have (token, session cookie), and maybe even, something you are (user_agent or a hash of a collection of headers, such as user_agent, Accept-Encoding, etc). (I've been reading Ilia's book after you recommended it) ;)
Hmm, hopefully my last sentence wasn't misleading. The B approach did not mean that one should block the user account, I meant that one should block brute forcing attempts per user account whereas in case A he blocks per clients (IP). Anyway, with User Agents and such info, you are relying on obscurity (that the brute forcer does not know those things).

Did I recommend some of Ilia's books? :roll:
User avatar
flying_circus
Forum Regular
Posts: 732
Joined: Wed Mar 05, 2008 10:23 pm
Location: Sunriver, OR

Re: Tracking a user by IP - concerns

Post by flying_circus »

Thanks Kai, I think you more or less answered my questions, or atleast prompted me to study the correct thing, to answer my own questions.

I'm discovering that there isnt necessarily "the solution" that I was looking for, as any security component is a compromise. It's something that we realistically cannot defend against, but merely weed out grossly obvious attacks.
  • IP based filtering presents a potential denial of service to real users, and there is no guarantee that it actually stops the threat, it may or may not be effective. It could be desireable in a situation like internet banking, or it could cost you dearly, in a situation like e-commerce where you are losing customers.
  • Captcha is an inconvenience and presents its own usability problems for internet users. Not to mention that captcha seems to be that hurdle set halfway between useless and deny all access. I suppose it stops dumb bots, but it doesnt stop all bots or spammers, this site uses captcha, and we still get the occasional spammer. Abuse prompted captcha could reveal usernames as you suggested as well. I'm undecided on the use of captcha on registration or login forms. I think a different method, such as a simple math or story problem would be more in line. Sure, it is an annoyance, but atleast it's accessible.
  • I think the biggest point I learned from your response, is that I wrongly assumed brute force on a login script would come from a remote site. While form tokens will effectively mitigate that problem, it doesnt stop a bot from using the actual form. I've been reading about creative solutions, such as honey pots to mitigate this risk, but again, nothing is a guarantee.
  • I think sleep timers might actually have some merit to them. Inconvenient sure, but will ultimately slow an attack. Not without their pitfalls though. Tied to an IP, rendered ineffective if the IP can change and may also throttle real users as a side effect. Tied to a session, ineffective if the attacker were to begin a new session. Tied to an account, ineffective if the attacker tries another user name.
  • About the only solution I can come up with is a combination of the above strategies, and eventual account lock out after it is obvious the user has forgotten their password or the account is being brute forced. Something rediculous, like 24 inccorrect attempts should be enough to trigger a lockout and email based reset. I mean, at what point does repeated trial of a forgotten password become a brute force attack?
  • While the user-agent method may be security through obscurity, tied to a valid session, it should never change durring the sessions lifetime. Forcing another login shouldn't be out of the question in this situation.

I've found some interesting concepts though, like kittenauth (hot-or-not-auth made me laugh), reCaptcha, invisible Captcha, counting keystrokes, honey-pots, or javascript / ajax / lightbox based login forms. Some of them present usability issues, and others assume the attacker will not handle javascript. I'm beginning to see the "real challenges" of creating a secure website. Kind of like designing an airplane, everything is a compromise. :banghead:
User avatar
kaisellgren
DevNet Resident
Posts: 1675
Joined: Sat Jan 07, 2006 5:52 am
Location: Lahti, Finland.

Re: Tracking a user by IP - concerns

Post by kaisellgren »

flying_circus wrote:[*]About the only solution I can come up with is a combination of the above strategies, and eventual account lock out after it is obvious the user has forgotten their password or the account is being brute forced. Something rediculous, like 24 inccorrect attempts should be enough to trigger a lockout and email based reset. I mean, at what point does repeated trial of a forgotten password become a brute force attack?
I think restricting attempts per IP or displaying a CAPTCHA like reCAPTCHA is the way to go instead of locking the account. There was a funny problem with eBay long time ago where accounts were locked after several failed log-in attempts. Some people purposely locked out other users who bid on the same items they did, preventing their challengers from bidding. :)
User avatar
flying_circus
Forum Regular
Posts: 732
Joined: Wed Mar 05, 2008 10:23 pm
Location: Sunriver, OR

Re: Tracking a user by IP - concerns

Post by flying_circus »

haha, that is pretty funny! It's true, the only problem with building a better mouse trap is that someone builds a better mouse. :mrgreen:
User avatar
timWebUK
Forum Contributor
Posts: 239
Joined: Thu Oct 29, 2009 6:48 am
Location: UK

Re: Tracking a user by IP - concerns

Post by timWebUK »

kaisellgren wrote: There was a funny problem with eBay long time ago where accounts were locked after several failed log-in attempts. Some people purposely locked out other users who bid on the same items they did, preventing their challengers from bidding. :)
That's brilliant! Haha.

flying_circus, a lot of your points and analysis are very valid, but I do think you're not giving a CAPTCHA enough credit. Abusing a prompted CAPTCHA based on usernames would reveal a username? Why not just base it on number of attempts using the same value in the username field? Not necessarily a valid username...
User avatar
kaisellgren
DevNet Resident
Posts: 1675
Joined: Sat Jan 07, 2006 5:52 am
Location: Lahti, Finland.

Re: Tracking a user by IP - concerns

Post by kaisellgren »

timWebUK wrote:Why not just base it on number of attempts using the same value in the username field? Not necessarily a valid username...
I mentioned this earlier, and you need to be careful. This basically means that you need one row per username attempt... so if someone brute forces 5 times a second, that's 432 000 database records per day, or 13 million records per month. That's a very simple way to abuse your server. The same thing pretty much happens with IP address limitation, except that no one can spoof IPs just like they can spoof requests (so it's much more limited). The good thing about the approach Vladsun mentioned (having a counter in the members table that gets increased by one per attempt) is that it won't fill up your entire database with data and barely has any effect on the database performance. All it requires is one additional column in the "users" -table with the type of TINYINT.
User avatar
flying_circus
Forum Regular
Posts: 732
Joined: Wed Mar 05, 2008 10:23 pm
Location: Sunriver, OR

Re: Tracking a user by IP - concerns

Post by flying_circus »

timWebUK wrote:but I do think you're not giving a CAPTCHA enough credit.
My main issues with CAPTCHA are useability problems. Any image based output deny's all blind users by default. reCAPTCHA aims to combat this problem by adding audio, which deny's all blind and deaf users. Distorted words are hard to read, especially for "older" users, and can be impossible for color blind users (a surprisingly large percentage of the male population).

My other issue with CAPTCHA is that, while it may work today, it wont last long. There are websites that provide API's for cracking CAPTCHA images. The aim is to help the blind, but its not like this this type of system will be abused or anything... Simple CAPTCHA's can be cracked with basic, off the shelf, OCR software.

My point is, as attackers get more creative to crack image based CAPTCHA's (in their traditional usage) to get after the bigger sites like yahoo, msn, or google, passing the simpler CAPTCHA's used on individuals websites become easier. Unless the CAPTCHA's are as robust as those used on big name sites, are they really effective? Often, the only way to make CAPTCHA systems more robust, is to complicate them, which significantly deteriorates useability.

I read of many user's and their struggles with the CAPTCHA system on ticketmaster.com. They could not get the right captcha value because the text was so distorted, ultimately costing them an opportunity to buy tickets for a show. Can you imagine losing a "bag of crap" on a woot off, because you couldnt pass the captcha?

I believe I will try a different path of using simple math or story problems. They are every bit as annoying as captcha, but they dont come with all the useability problems. It's likely a personal decision and a compromise between getting that extra nth degree of spam protection vs losing customers.
User avatar
timWebUK
Forum Contributor
Posts: 239
Joined: Thu Oct 29, 2009 6:48 am
Location: UK

Re: Tracking a user by IP - concerns

Post by timWebUK »

Very valid points, however, I must make something clear... I can see no purpose a blind and deaf person would require a website for. They cannot interact with computers easily - if at all. They require braille. So dismissing reCAPTCHA because it doesn't provide an interface for a blind and deaf person to pass a test is slightly absurd.

Also, simple math's problems? They could be solved just as easily as a Captcha by character recognition software... I mean Windows7 has formula program built in allowing you to draw formulas and it converts it's to computer copyable characters.

I believe this is the sort of thing you would have to resort to, which could be modified to have audio:

http://server251.theory.cs.cmu.edu/cgi- ... ix/esp-pix

But then we're getting ridiculous...
Post Reply