Page 2 of 3

Posted: Sat Jul 14, 2007 3:26 am
by GameMusic
What I mean is, if the database stores a salt and a salt/pepper password, if you change the salt every time (and I assume you'd do this when the user logs in, since that would be the only point at which the script should know the password to reencode), anybody who reads both would just as easily crack one combination as another. Is this actually the one time pad technique being discussed or is he talking about something else?

Posted: Fri Jul 20, 2007 8:27 am
by Mordred
I realize there are unanswered issues about the article, here's a late reply:
The Phoenix wrote:"Use obscurity as another security layer where appropriate": Many people misunderstand this concept. Obscurity != Security, even for large values of Obscurity. It doesn't ever add security. Here's an example that illustrates it. If I have a million dollars, and hide it in my house, given sufficient search time & resources, you will locate it, and take it.

If however, I put it inside a safe (fireproof, axeproof, etc), it is *secured*. Not *obscured*. Thats why we trust safes and vaults, and treasure maps went out of style long ago.

In fact, having obscure/difficult names for tables and the like make it likely the programmer will make mistakes, lowering the security by accident. Security withstands attack - it doesn't avoid it.

Even if you are married to the idea, and want to plead the case for obscurity as a positive add to overall security, it belongs in a separate thread. One likely to generate flames, and a ton of misinformed posts.
My experience in pen-testing real-world applications shows that this is utterly wrong. In fact, I will move this advice from point 4 straight to being point 1. Obscuring names is the best mitigation tactic, and it costs nothing to the programmer (you mistakenly assume that the programmer will make mistakes - pray, how exactly?)
Security withstands attack - it doesn't avoid it.
The whole article deals with mitigation - what to do [s]if[/s] when ordinary security measures fail. Your stance that "security shouldn't fail" is not helpful, and - alas - not based in the real world, sorry.
The Phoenix wrote: - Hashing (Salt & Pepper): You left out the most secure, and most widely recommended approach: One-time pads. By sending a unique salt that is generated per-request, you can avoid all of the attacks listed.
This just doesn't make sense, or I misunderstand what you mean in a big way. Here's what I thing, please correct me:

1. One-time pads cannot be used in a practical/secure way for storage. They will either be non-one-time, not practical or not more secure.
2. "sending unique salt" is maybe about secure transport of login data, something this paper is not about. Maugrim has a nice tutorial about that.


@Maugrim: Aye, TLS/SSL is the currently most secure transport option, and something like your tutorial is the next best thing when https is not possible, I will mention both options.

Posted: Sat Jul 21, 2007 4:05 am
by The Phoenix
Mordred wrote: My experience in pen-testing real-world applications shows that this is utterly wrong. In fact, I will move this advice from point 4 straight to being point 1. Obscuring names is the best mitigation tactic, and it costs nothing to the programmer (you mistakenly assume that the programmer will make mistakes - pray, how exactly?)
Mitigation means to lessen in severity. Obscuring the names does not reduce the impact of the breach - once the attacker gains the names of the variables, the impact is precisely the same. As such, it is not a good mitigation tactic.

Worse, it does cost something to the programmer. By memory, I can say that username=bob while coding. On the other hand, in your scheme, I'd likely need to check to see if username is actually field 'sirguhgh' or field 'guhffed'. If you chose fields with similar schemes (field names ooo1, ooo2, ooo3, etc), it would make it likely you would mistype at some point. Its relatively easy to see when you've used the wrong fieldname on username, not so with ooo2 v. ooo3.

But as I said in my original post, obscurity v. security is a meta-topic, and it was only one paragraph in a very long article. Simply avoid that topic in this one, as it has a huge wealth of discussion material.
Mordred wrote:
Security withstands attack - it doesn't avoid it.
The whole article deals with mitigation - what to do [s]if[/s] when ordinary security measures fail. Your stance that "security shouldn't fail" is not helpful, and - alas - not based in the real world, sorry.
On the contrary, while much of your article deals with mitigation, that section did not. It was out of place. As to being based in the real world, I have sufficient experience to refute that claim.
Mordred wrote:
The Phoenix wrote: - Hashing (Salt & Pepper): You left out the most secure, and most widely recommended approach: One-time pads. By sending a unique salt that is generated per-request, you can avoid all of the attacks listed.
This just doesn't make sense, or I misunderstand what you mean in a big way. Here's what I thing, please correct me:

1. One-time pads cannot be used in a practical/secure way for storage. They will either be non-one-time, not practical or not more secure.
2. "sending unique salt" is maybe about secure transport of login data, something this paper is not about. Maugrim has a nice tutorial about that.
I think you understand my meaning. You generate a per-request salt. Your argument that they cannot be used in a practical or secure way is inaccurate. Thats the exact method that has been used for over twenty years in CHAP logins. The server sends a one-time pad, and the user receives it. The user combines the OTP and a hashed version of their password, and sends it back. The server does the same and compares it. If they match, its in.

Its stored securely - the password is hashed. Its transmitted securely - the hashed password itself isn't sent. It's one time, and it's secure.

Maugrim's article does do a good job heading down that path. I think in many ways your article covers some of the same territory without being as clear, or offering working code that clarifies the examples.

I reiterate, I think the article was good, but its not the place for a security v. obscurity argument. It reduces the value of the rest of the information presented. It goes substantially against the real-world information security community, and as such, if it is to be covered, should be its own topic.

Posted: Sat Jul 21, 2007 9:07 am
by superdezign
The Phoenix wrote:Mitigation means to lessen in severity. Obscuring the names does not reduce the impact of the breach - once the attacker gains the names of the variables, the impact is precisely the same. As such, it is not a good mitigation tactic.
But it can greatly reduce the likeliness of a successful attack through SQL injection, and, if the hacker does gain access to your database, they'll need to take the time and effort to decipher what's what.
The Phoenix wrote:Worse, it does cost something to the programmer. By memory, I can say that username=bob while coding. On the other hand, in your scheme, I'd likely need to check to see if username is actually field 'sirguhgh' or field 'guhffed'. If you chose fields with similar schemes (field names ooo1, ooo2, ooo3, etc), it would make it likely you would mistype at some point. Its relatively easy to see when you've used the wrong fieldname on username, not so with ooo2 v. ooo3.
No one said the names had to be senseless, but they shouldn't be obvious, either. Username and password are extremely common. Usrnam and Pasw are not.
The Phoenix wrote:
Mordred wrote:
The Phoenix wrote: - Hashing (Salt & Pepper): You left out the most secure, and most widely recommended approach: One-time pads. By sending a unique salt that is generated per-request, you can avoid all of the attacks listed.
This just doesn't make sense, or I misunderstand what you mean in a big way. Here's what I thing, please correct me:

1. One-time pads cannot be used in a practical/secure way for storage. They will either be non-one-time, not practical or not more secure.
2. "sending unique salt" is maybe about secure transport of login data, something this paper is not about. Maugrim has a nice tutorial about that.
I think you understand my meaning. You generate a per-request salt. Your argument that they cannot be used in a practical or secure way is inaccurate. Thats the exact method that has been used for over twenty years in CHAP logins. The server sends a one-time pad, and the user receives it. The user combines the OTP and a hashed version of their password, and sends it back. The server does the same and compares it. If they match, its in.
I think he means that it wouldn't be secure because you would need to store that salt, which would likely mean that the salt would be stored in the database with an obvious relation to the user that the salt belongs to. If your database is compromised, then so is your salt.

You can make that approach as secure as the personal salts, though. As long as you have some sort of method for processing the salt between creating it and using it to salt the password, the attacker remains clueless as to what the salt truly is. But storing the one-time salt, I would say, would give the attacker the first clue to how your salt is processed.

Posted: Sat Jul 21, 2007 10:06 am
by The Phoenix
I think he means that it wouldn't be secure because you would need to store that salt, which would likely mean that the salt would be stored in the database with an obvious relation to the user that the salt belongs to. If your database is compromised, then so is your salt.
You don't store the salt. Not at all. One-time pad. You generate a challenge, send the challenge to the user. The user combines that with the hashed password, and sends it in:

User:
Challenge (OTP) -> hash(OTP+HashedPW) -> SERVER

Server:
Create new challenge (OTP) -> User -> Check hash(OTP+HashedPW)

See? No storage of the salt.

Posted: Sat Jul 21, 2007 10:20 am
by superdezign
The Phoenix wrote:
I think he means that it wouldn't be secure because you would need to store that salt, which would likely mean that the salt would be stored in the database with an obvious relation to the user that the salt belongs to. If your database is compromised, then so is your salt.
You don't store the salt. Not at all. One-time pad. You generate a challenge, send the challenge to the user. The user combines that with the hashed password, and sends it in:

User:
Challenge (OTP) -> hash(OTP+HashedPW) -> SERVER

Server:
Create new challenge (OTP) -> User -> Check hash(OTP+HashedPW)

See? No storage of the salt.
That's sounds interesting. I'm not exactly sure as to how this is implemented, but I am curious. Is this ruled by user input or a server-side process? And do you have any examples / articles in regards to the method?

Posted: Sat Jul 21, 2007 10:37 am
by The Phoenix
superdezign wrote:That's sounds interesting. I'm not exactly sure as to how this is implemented, but I am curious. Is this ruled by user input or a server-side process? And do you have any examples / articles in regards to the method?
Here's a quick example/psuedo-code. on the server side, you make a OTP.. $otp = md5(uniqid(rand(), true));

Then you send that to the user. The user in javascript does md5($otp . md5($password)), and sends it to the server. The server then checks if the stored hashed password (md5(password)) matches.

Next time you come to the site, its a whole new OTP (thanks to rand/uniqid). Maugrim's tutorial deals with this a bit, and so does Paj's site.

(Please note, I actually suggest using sha256, not md5, but md5 is an easy example).

Posted: Tue Jul 24, 2007 9:38 am
by timgolding
Actually i even bookmarked this page and will show it to my students. I joined this site because google directed me to this page a few days ago.

Posted: Tue Jul 24, 2007 10:35 am
by Mordred
The Phoenix wrote: Mitigation means to lessen in severity. Obscuring the names does not reduce the impact of the breach - once the attacker gains the names of the variables, the impact is precisely the same. As such, it is not a good mitigation tactic.
You don't seem to follow what I'm saying. Obscuring the names does reduce the impact of the breach, exactly by preventing the attacker from gaining the names of the variables and tables. It is a perfect mitigation tactic, because even though the attacker has just found an SQL injection hole in one of your scripts, but he cannot use it in any meaningful way, because he needs to know the name of your login table (for example).
The Phoenix wrote: Worse, it does cost something to the programmer. By memory, I can say that username=bob while coding. On the other hand, in your scheme, I'd likely need to check to see if username is actually field 'sirguhgh' or field 'guhffed'. If you chose fields with similar schemes (field names ooo1, ooo2, ooo3, etc), it would make it likely you would mistype at some point. Its relatively easy to see when you've used the wrong fieldname on username, not so with ooo2 v. ooo3.
Maybe this will make things clear. No additional effort is required from the programmer.

Code: Select all

$g_sLoginTable = $g_sPrefix . 'login_j4mfgbsl';  //the suffix is randomly generated at install-time.
$g_sMessagesTable = $g_sPrefix . 'messages_93nmvb3';
//... later that day ...
$nMessageId = $_GET['mid']; //vulnerable?
$sQuery = "SELECT * FROM $g_sMessagesTable WHERE mid='$nMessageId'"; //exploitable?
The Phoenix wrote:
Mordred wrote:
Security withstands attack - it doesn't avoid it.
The whole article deals with mitigation - what to do [s]if[/s] when ordinary security measures fail. Your stance that "security shouldn't fail" is not helpful, and - alas - not based in the real world, sorry.
On the contrary, while much of your article deals with mitigation, that section did not. It was out of place. As to being based in the real world, I have sufficient experience to refute that claim.
I do not mean to offend you when I'm speaking of experience (everyone's experience is limited after all ;) ), but "Security shouldn't fail" is not a way of thinking that comes from the voice of experience, it is a good wish written on water. In reality security measures fail and aditional mitigation steps may be your last line of defense. I already explained (hopefully) how obfuscation is a good and efficient mitigation strategy. Yes, the coder needs to properly validate and escape data, yes, SQL injection is preventable with a realistic amount of effort, yes, one must be always careful when he writes code. But it doesn't always happen that way. This is what the article is about (in the limited realm of storing login credentials of course)
The Phoenix wrote:
Mordred wrote: 1. One-time pads cannot be used in a practical/secure way for storage. They will either be non-one-time, not practical or not more secure.
2. "sending unique salt" is maybe about secure transport of login data, something this paper is not about. Maugrim has a nice tutorial about that.
I think you understand my meaning. You generate a per-request salt. Your argument that they cannot be used in a practical or secure way is inaccurate. Thats the exact method that has been used for over twenty years in CHAP logins. The server sends a one-time pad, and the user receives it. The user combines the OTP and a hashed version of their password, and sends it back. The server does the same and compares it. If they match, its in.
Its stored securely - the password is hashed. Its transmitted securely - the hashed password itself isn't sent. It's one time, and it's secure.
1. There is a terminology mixup, what you talk about is not a one-time pad (I'm too lazy to explain it, go check wikipedia).
2. It is a chalenge token and you talk about using it in a challenge-response protocol for transmitting login data. This is what Maugrim's article deals with, and is what this article explicitly states that it doesn't deal with.
3. As far as secure storage is concerned (which is what the article is about), storing an unsalted hash of the password is not secure enough.
The Phoenix wrote: I reiterate, I think the article was good, but its not the place for a security v. obscurity argument. It reduces the value of the rest of the information presented. It goes substantially against the real-world information security community, and as such, if it is to be covered, should be its own topic.
I would like to ask you to either present texts from the said "real-world information security community" which refute my argument or just refute it yourself. This is not a generic "security v. obscurity argument", it is a concrete application which does - I repeat - a tremendously good job.

I hope you don't find my spech offensive (excuse me if so), I am attacking your arguments, not yourself.

Posted: Tue Jul 24, 2007 1:16 pm
by The Phoenix
Mordred wrote:You don't seem to follow what I'm saying. Obscuring the names does reduce the impact of the breach, exactly by preventing the attacker from gaining the names of the variables and tables. It is a perfect mitigation tactic, because even though the attacker has just found an SQL injection hole in one of your scripts, but he cannot use it in any meaningful way, because he needs to know the name of your login table (for example).
I do follow, we simply disagree. I feel there are sufficient alternative methods available to the attacker to find the names. You instead offer ways to prevent that (an additional and separate security measure), which proves the point that there are alternative methods to do so (in fact, many beyond the methods listed).

Ask it the opposite way: If there is a method available to the attacker to get the table names, will a complicated naming scheme improve security, or simply increase difficulty in coding?
The Phoenix wrote: Worse, it does cost something to the programmer. By memory, I can say that username=bob while coding. On the other hand, in your scheme, I'd likely need to check to see if username is actually field 'sirguhgh' or field 'guhffed'. If you chose fields with similar schemes (field names ooo1, ooo2, ooo3, etc), it would make it likely you would mistype at some point. Its relatively easy to see when you've used the wrong fieldname on username, not so with ooo2 v. ooo3.
Mordred wrote:Maybe this will make things clear. No additional effort is required from the programmer.
Thats an interesting approach I wasn't thinking of in terms of development. I'm not convinced on the debugging side of things that it would be simple, but I'll concede that on the development side its elegant.
Mordred wrote:I do not mean to offend you when I'm speaking of experience (everyone's experience is limited after all ;) ), but "Security shouldn't fail" is not a way of thinking that comes from the voice of experience, it is a good wish written on water.
Four responses, all the last I will say on this portion of the discussion:

1. It is incredibly personal to say "That is not a way of thinking that comes from the voice of experience". If you feel I am not sufficiently experienced, that is an issue you should discuss in private messaging. I feel quite comfortable that at least a few of the moderators are aware of my background, and know that I not only have the experience to speak with the voice of experience, but that I also do not choose to contribute in areas outside of my expertise. Discuss the concept in public. Discuss your opinion about my experience in private.

2. I never made the claim that security shouldn't fail. You rephrased my statements to fit that, and doing so was inaccurate. I said that security should withstand attack, not avoid it. That doesn't mean that security shouldn't fail. Its the difference between fighting a good fight (and losing), and running away. Obscurity runs away. Security stands and fights. Whether it wins is a question of resources and design on each side of the attacker/defender equation.

3. Obscurity is out of place with the rest of this article. It has nothing to do with hashing credentials. It is an unrelated suggestion that I've argued does not add substantial security. You can agree or disagree with that stance, but it still has no direct impact on the hashing of the credentials. If I use normal tables, and you use 'tricky' ones, would our resulting hashes be different? No. As such, whether you agree with it adding or subtracting net security, its not related to the key topic, and should be removed to avoid difficulty.
Mordred wrote:1. There is a terminology mixup, what you talk about is not a one-time pad (I'm too lazy to explain it, go check wikipedia).
In this one, the devil is in the details. If you don't wish to discuss the details, I can't explain why its an accurate use of the phrase. But it doesn't matter, because..
Mordred wrote:2. It is a chalenge token and you talk about using it in a challenge-response protocol for transmitting login data. This is what Maugrim's article deals with, and is what this article explicitly states that it doesn't deal with.
It explicitly states that it is about storage, but the title is misleading, as hashing does have concerns on both the transmission AND the storage side. The fact that multiple replies brought up transmission shows that I'm not the only one that would think so.

That said, it doesn't matter either, because..
Mordred wrote:3. As far as secure storage is concerned (which is what the article is about), storing an unsalted hash of the password is not secure enough.
This is the heart of your article, and what matters. The rest has all been a distraction. One of the problems is that the choice you advocate (not storing unsalted hashes) has its most direct impact on transmission - which you are forcefully asking not to discuss in this topic.

The two are related, and I can't discuss one without the other. Yet, you want to expand the discussion to include security v. obscurity, which is easily an entirely separate discussion which does not have direct bearing on your article. I'm challenged to understand that logic.
Mordred wrote:
The Phoenix wrote: I reiterate, I think the article was good, but its not the place for a security v. obscurity argument. It reduces the value of the rest of the information presented. It goes substantially against the real-world information security community, and as such, if it is to be covered, should be its own topic.
I would like to ask you to either present texts from the said "real-world information security community" which refute my argument or just refute it yourself. This is not a generic "security v. obscurity argument", it is a concrete application which does - I repeat - a tremendously good job.
You are asking me to disprove the generic argument you have put forth (in the context of a specific area). I've made clear that the discussion is considerably larger than this article thread, and every discussion we've had confirms that. It is a sweeping discussion.

I've offered to discuss it in another thread, you declined.
I've offered specific refutations, you've argued, and I've offered to discuss it in another context, you declined.
Mordred wrote:I hope you don't find my spech offensive (excuse me if so), I am attacking your arguments, not yourself.
On the contrary, you've personally attacked my experience and knowledge. It is an issue best dealt with in private, and so I will.

The article was posted as a draft. You said "Comments, thoughts, fact and grammar nitpicks are very much welcomed. "

I provided them with both good intent, reasoned discussion, and no personal attacks of any kind. I think the article is strong, but since you asked "Do you think this needs expanding (or shrinking) in some direction? " The answer I gave was yes - one paragraph should be removed.

I have provided my opinion, and my reasoning. Discussion I'm happy to engage in, but this thread has devolved into arguing, and I have no interest in that.

Good luck with your article. Sincerely.

Posted: Sun Jul 29, 2007 8:01 am
by Mordred
First, I must apologise. My statements about experience had no place in this discussion, and while I can reconstruct for myself how I came as low as this ad hominem attack (concealed even from myself), it is absolutely no excuse for what I said. I also put words in your mouth that you didn't say when I misunderstood what you meant. Please accept my apologies and thanks for staying in the discussion despite that, I am glad and honored to have such an opponent, even that I don't fully agree with you, the dispute itself is very useful.

....
I don't have time right now to continue the debate (although I very much wish to), so I will copy/paste what I've written so far for later and stop at the apology.

Re: [article draft] Password hashing howto and hownotto

Posted: Sat Jan 12, 2008 1:29 am
by jmut
Well, maybe some will consider too old topic, and let it be..but I will still try.
Question is about the random salt per user. Ultimate goal is you get different hashes even if users have same passwords. Does that mean that:
- should add extra column with pure salt -> not hashed or anything, just random for each user.

If so, wouldn't that be same as giving no random salt if someone access the db, as he will see salt in plain and will be able to use. Or trick here is you use some weird logic behind using this salt?... like taking first 5 and last 5 chars of salt or weird like that... so that he will have to take db + source code to successfully make use of it?
Thanks.

Re: [article draft] Password hashing howto and hownotto

Posted: Sat Jan 12, 2008 3:44 am
by Mordred
jmut wrote:Well, maybe some will consider too old topic, and let it be..but I will still try.
Question is about the random salt per user. Ultimate goal is you get different hashes even if users have same passwords. Does that mean that:
- should add extra column with pure salt -> not hashed or anything, just random for each user.

If so, wouldn't that be same as giving no random salt if someone access the db, as he will see salt in plain and will be able to use. Or trick here is you use some weird logic behind using this salt?... like taking first 5 and last 5 chars of salt or weird like that... so that he will have to take db + source code to successfully make use of it?
Thanks.
This is addressed in part 6, about "salting and peppering". You would use two salts (or, in my self-coined terminology, "salt and pepper") - one per user and one per site installation (which should be strong - i.e. long, and resides in your PHP source). Even with full access to the database, the attacker would still need the per site salt ("pepper") to successfully attack the hashes.

Since the main "protection" of the hash comes from the per-site salt, and having another field in the database adds no security if the attacker has full DB access, you can actually use a simpler per-user salt, for example the username. On the other hand there is very little increased hassle if you add another salt field per user, so I would reccommend this variant.

Re: [article draft] Password hashing howto and hownotto

Posted: Thu Sep 18, 2008 8:42 am
by Haravikk
A very good read! I may note also, that salts should pad-out the input. That is; while hashes such as MD5 and SHA are great at what they do, the more information you can give them, the better the resulting hash (and greater the degree of distribution).

For example; md5 produces a 128-bit hash, for this reason you should endeavour to provide it with a MINIMUM of 128-bits of data to process, preferably however you will give it at least 256-bits, especially when we consider that the entropy for typical English text is closer to 5-bits per character than it is to the full 8-bits. Further to this, salts are typically either; integers, hexadecimal, or characters.

Ideal salts would be completely random characters with a full 8-bits of entropy, and sufficiently long that combined with the original input they will bring it to at least 256-bits of raw-data, or preferably 256-bits of entropy. This way you get the best possible distribution of values you can from your hashing function.

Consequently, it is this basic idea that makes double-hashing cryptographically insecure, as every-time you re-hash a value without properly padding it with more data (i.e - adding another salt), you reduce the range of possible hashes. So instead of 2 ^ 128 distribution in hash values you may end up with a 2 ^ 64 and so-on. Double-hashing WITH additional salts however is pointless, as MD5 and SHA are both designed to give well-distributed results anyway, so you're better off (as proposed in the original post) just taking the extra salt(s), concatenating with the input, and hashing only once.

Re: [article draft] Password hashing howto and hownotto

Posted: Fri Sep 19, 2008 4:50 am
by Mordred
Haravikk, you have some very good points, some wrong ones, (and some very wrong numbers :) )

Indeed, salts are meant to increase the entropy of the plaintext password, so they have to be "strong" (i.e. with enough entropy). Whether they should "fill" an entire block of the hash function is a different matter - it would be a problem, only if there are vulnerabilities of the hash function with padded blocks (in which case, you should not use this function altogether, instead of trying to "fix" it).

But yes, "Ideal salts would be completely random characters with a full 8-bits of entropy, and sufficiently long", well said!

About the numbers, almost all of yours are wrong, I'm afraid, and by a huge margin, that's why I will comment on them:
MD5 has a 128-bit output, but works internally on 512-bit blocks, so if the padding thing were applicable, it would have to be about 512-bits wide.
The entropy of typical English is estimated around one bit per character, the numbers vary from 0.6 to 1.5 bits. Your number, 5 bits would roughly correspond to strings of equally distributed lowercase letters, not English text.

Double hashing does not reduce entropy in such a dramatic way (imagine: if you are right, it would mean that after 8 hashings all inputs would produce equal outputs). I don't know the actual numbers, they would also depend on the hash function being used, but it should be something in the order of thousandths of a bit. In fact there are cases (not related to the topic) where double-hashing protects against some attacks, so it is sometimes a valid approach.

The trouble with double hashing in relation to password storage is not that it reduces entropy (yes it does, but only in a tiny way), but that it doesn't add entropy, like salting does. Even further, the point of salt-and-pepper scheme is to distribute the sources of entropy, so that one security breach would not reveal all of them.

Your note on the entropy of the salts is much appreciated though.