Page 3 of 3
Re: Lugubriousness
Posted: Thu Jul 09, 2009 12:18 pm
by kaisellgren
arborint wrote:I don't think you can say that \n \r \t \77 \xFF are is not transformations.
Well, I think I can. Here's my explanation. \n (literal string) -> 0x0a (byte) is a transformation... but the process that happens beyond the interface of mysql_real_escape_string() for instance, is not actually a transformation. You give the parser Input, which remains the same through the entire process, and as you move on, the parser copies data from the Input to the output buffer. Thus, there are never transformations, but just some copying/creation of data to a separate buffer. MySQL will literally
for loop through the bytes, and if it faces 0x27 ('), then it directly creates bytes 0x5c (\) and 0x27 (') into the output buffer (separate). In the end of the escaping process, the input stream is still the same (untouched).
Usually none of this matters to programmers, but it's interesting to discuss though.
Re: Lugubriousness
Posted: Thu Jul 09, 2009 1:27 pm
by Christopher
kaisellgren wrote:Well, I think I can. Here's my explanation. \n (literal string) -> 0x0a (byte) is a transformation... but the process that happens beyond the interface of mysql_real_escape_string() for instance, is not actually a transformation. You give the parser Input, which remains the same through the entire process, and as you move on, the parser copies data from the Input to the output buffer. Thus, there are never transformations, but just some copying/creation of data to a separate buffer. MySQL will literally for loop through the bytes, and if it faces 0x27 ('), then it directly creates bytes 0x5c (\) and 0x27 (') into the output buffer (separate). In the end of the escaping process, the input stream is still the same (untouched).
As bizarre as it may sound, I think we agree that it "is a transformation" but it "is not actually a transformation."
kaisellgren wrote:Usually none of this matters to programmers, but it's interesting to discuss though.
I think it clarifies some of the nuance of the discussion. I think this understanding will help get to the bottom of the problem you state in your first post.
Re: Lugubriousness
Posted: Thu Jul 09, 2009 2:58 pm
by kaisellgren
arborint wrote:I think it clarifies some of the nuance of the discussion. I think this understanding will help get to the bottom of the problem you state in your first post.
Indeed. I asked a couple of experienced programmers who all agreed with the terms. I think we have pretty well covered what do those common terms mean and I'm sure most people can agree with those or at least understand our perspective.
arborint wrote:As bizarre as it may sound, I think we agree that it "is a transformation" but it "is not actually a transformation."


Re: Lugubriousness
Posted: Thu Jul 09, 2009 5:37 pm
by Christopher
Next is to similarly clarify the types of attacks. For example:
- XSS Cross Site Scripting
- SQL Injection
Re: Lugubriousness
Posted: Fri Jul 10, 2009 3:45 am
by kaisellgren
A successful SQLi usually requires to place 0x27 into the query intact unless the place where you place it is not enclosed within quotes (ORDER BY, LIMIT or just a mistake of not using quotes around values). In case of XSS, that's way too broad... usually we want to make the Javascript parser to execute and run some code for us.
Basically, when you are using Encoding as your protection, things are much more complicated. If you are Escaping, then you usually have to take care of a few meta characters.
Re: Lugubriousness
Posted: Fri Jul 10, 2009 11:31 am
by Christopher
kaisellgren wrote:A successful SQLi usually requires to place 0x27 into the query intact unless the place where you place it is not enclosed within quotes (ORDER BY, LIMIT or just a mistake of not using quotes around values).
So if you always quote and always escape you are safe from SQL injection attacks?
kaisellgren wrote:In case of XSS, that's way too broad... usually we want to make the Javascript parser to execute and run some code for us.
Basically, when you are using Encoding as your protection, things are much more complicated. If you are Escaping, then you usually have to take care of a few meta characters.
Leaving the Escaping/Encoding question aside

what are the values that you need to quote?
Re: Lugubriousness
Posted: Fri Jul 10, 2009 2:09 pm
by kaisellgren
If you just could quote... imagine having a dynamic ORDER BY statement, you can't quote the column name with 0x27 or it will not work. That's where prepared statements become useless - they only apply to DML.
Re: Lugubriousness
Posted: Fri Jul 10, 2009 4:09 pm
by Christopher
So the only way to use user input in SQL for something that is not a quoted value would be to filter it with a whitelist character set? The other options is to check it against acceptable values or use it to select among acceptable values.
Re: Lugubriousness
Posted: Sat Jul 11, 2009 12:12 am
by matthijs
Another type of attack could be mail injection. Still a more complex topic then what you'd think at first sight
Re: Lugubriousness
Posted: Sat Jul 11, 2009 1:43 am
by kaisellgren
arborint wrote:So the only way to use user input in SQL for something that is not a quoted value would be to filter it with a whitelist character set? The other options is to check it against acceptable values or use it to select among acceptable values.
Yeah, either filter
Code: Select all
$column = preg_replace('#^[^a-z0-9_]+$#iD','',$column);
or use white listing
Code: Select all
$allowedColumns = array('id','downloads');
if (!in_array($column,$allowedColumns))
$column = 'downloads'; // you could default to something
Those are filtering the column name. You could also use validation approach.
matthijs wrote:Another type of attack could be mail injection. Still a more complex topic then what you'd think at first sight
SQLi is simple, but yeah injection as the subject isn't.
Re: Lugubriousness
Posted: Sat Jul 11, 2009 3:46 am
by Christopher
kaisellgren wrote:Yeah, either filter
Code: Select all
$column = preg_replace('#^[^a-z0-9_]+$#iD','',$column);
or use white listing
Code: Select all
$allowedColumns = array('id','downloads');
if (!in_array($column,$allowedColumns))
$column = 'downloads'; // you could default to something
Those are filtering the column name. You could also use validation approach.
Those are really both white listing. The first we might call Character White Listing and while the second is Value White Listing.
Re: Lugubriousness
Posted: Sat Jul 11, 2009 4:12 am
by kaisellgren
arborint wrote:Those are really both white listing. The first we might call Character White Listing and while the second is Value White Listing.
Imo, white listing and black listing are not methods themselves, but approaches. Filtering, validating, escaping and many others can be categorized into white lists and black lists. For example, mysql_real_escape_string() is black listing. The second example I gave could be said to be validating (rejects non-allowed values) using a white list. Maybe we could define "white lists" and "black lists", too.
What about:
which tries to match bytes in a range of 0-127? And what about
which tries to match bytes that are not in a range of 0-127?
Are these white lists or black lists? Which is which?
Essentially, a filter
that removes those three dangerous characters (in an imaginary situation) like that would be using black listing. All other potentially dangerous characters such as 0x0a, 0x00, etc. are not filtered. On the contrary,
would be using white listing. It filter characters that are not in the needed character set.
Again, I don't think there are any standard terms for these, but this is the way I think of these. Comments? Arguments? Nods?
Re: Lugubriousness
Posted: Sat Jul 11, 2009 6:14 am
by matthijs
kaisellgren wrote:Again, I don't think there are any standard terms for these, but this is the way I think of these. Comments? Arguments? Nods?
Excellent explanation. I think it's really important to really think about things like this. The difference between using one regular expression (whitelisting) and another (blacklisting) can be pretty fundamental, as you show.