ctype_print for email injection prevention

Discussions of secure PHP coding. Security in software is important, so don't be afraid to ask. And when answering: be anal. Nitpick. No security vulnerability is too small.

Moderator: General Moderators

matthijs
DevNet Master
Posts: 3360
Joined: Thu Oct 06, 2005 3:57 pm

Post by matthijs »

AGISB, I totally agree with you about the validating of all user input.

That's why I'm bothering everyone here asking a lot of questions ;)

But I also learned about the defense in depth principle. So even if I (try to) validate the emailaddress a user has entered with a regex, I want or would like to double check and see if anything suspicious is being entered. (like a "multiparts*/s*mixed" or bcc:)

Because, maybe the regex I use is not sufficient? Regex's can be very hard, and I'm no regex expert, so maybe there's a small mistake somewhere in the email validation routine. Or maybe I made a mistake copy-pasting a piece of code (ok, that would be stupid. But we make mistakes, do we?).

For example, if I look at the regex provided by ilovejackdaniels. http://www.ilovejackdaniels.com/php/ema ... alidation/
That was published 1 june 2004. Since then some people have commented on the code and found mistakes or errors. they were acknowledged by the author, and the regex was updated at some point. So at some time, and maybe even still at this time, the regex could be insufficient. In that case I would like to have a backup validation routine, checking for suspicious input or newlines etc.
(p.s. I'm not nitpicking on Dave, just using it as an example)

So, you are correct in saying that the solution "Just validate any user input" is simple. However, "Just validate any user input" is not so simple. As can be seen by the many many threads in which people discuss what's good or not, better or not.
User avatar
Buddha443556
Forum Regular
Posts: 873
Joined: Fri Mar 19, 2004 1:51 pm

Post by Buddha443556 »

Matthijs, when you get done coding this email thing might you consider coming back here and posting it for review. [Just remeber such reviews take time so don't expect an immediate response.]

AGISB, your absolutely correct all user data should be validated.

Merry Christmas!
matthijs
DevNet Master
Posts: 3360
Joined: Thu Oct 06, 2005 3:57 pm

Post by matthijs »

Buddha, I will do that. Really appreciate the valuable feedback I get and have gotten so far. And I promise I will not get angry when I don't get an answer in 5 minutes ;)

First I'm going to my family (in-law) to celebrate Christmas and eat a lot. I'm sure I'm not the only one. So Merry Christmas everyone and speak to you later!

Matthijs
matthijs
DevNet Master
Posts: 3360
Joined: Thu Oct 06, 2005 3:57 pm

Post by matthijs »

Hi, I'm back full of energy (literally...) to tackle this issue.
I set up some test cases, as shown below.

Straight from the manual with a small addition:

Code: Select all

<?php
$strings = array('email'=> 'mymail@mail.com',
                           'string1' => "asdf\n\r\t",
		           'string2' => 'arf12',
			  'string3' => 'LKA#@%.54',
			  'string4'=> "sender@anonymous.www%0ACc:recipient@someothersite.xxx%0ABcc:somebloke@grrrr.xxx,someotherbloke@oooops.xxx"
					  );

foreach ($strings as $name => $testcase) {
   if (ctype_print($testcase)) {
       echo "The string '$name' consists of all printable characters.<br>";
   } else {
       echo "The string '$name' does not consist of all printable characters.<br>";
   }
}
?>
This will print out:

Code: Select all

The string 'email' consists of all printable characters.
The string 'string1' does not consist of all printable characters.
The string 'string2' consists of all printable characters.
The string 'string3' consists of all printable characters.
The string 'string4' consists of all printable characters.
As you can see, the string I have taken from http://securephp.damonkohler.com/index. ... _Injection gets through.
So, Buddha443556 seems to be correct when he says:
ctype_print wouldn't be a safe option because it doesn't catch url escaped characters like "%0A"
I did try some more stuff, using a form with POST fields, like:

Code: Select all

<?php
if (!isset($_POST["send"])){
   // no post data -> display form
?>
   <form method="POST" action="<?=$_SERVER['PHP_SELF'];?>">
   To: webmaster@website.com<br>

   From: <input type="text" name="sender"><br>

   Subject : <input type="text" name="subject"><br>

   Message : <br>

   <textarea name="message" rows="10" cols="60" lines="20"></textarea><br>

   <input type="submit" name="send" value="Send"><br>

   </form>
<?php 
 }else{
   // found post data .. deal with it
   $from=$_POST['sender'];
   if (eregi("\r",$from) || eregi("\n",$from)){
     echo "Not ok<br>$from";

   }
   else
   {
	echo "Ok<br>$from";
   }
?>
But one thing I don't understand is that with all examples given on the damonkohler.com site, it always echo's out "Ok", even with evil input as:
sender@anonymous.www%0ACc:recipient@someothersite.xxx%0ABcc:somebloke@grrrr.xxx,someotherbloke@oooops.xxx

So the eregi("\r", $from) || eregi("\n",$from)) solution given doesn't seem to help much against email injection?

Or should I set up my test case different?

I remember from a couple of months ago when I had to deal with email-injection attacks on a couple of sites, this was the reason I chose for another solution (looking for the occurance of different strings as shown in the function isInjection($text) )

What I do now is a combination of regex to validate the input and then with the email-injection function look for suspicious input.
josh
DevNet Master
Posts: 4872
Joined: Wed Feb 11, 2004 3:23 pm
Location: Palm beach, Florida

Post by josh »

Well it is trivial wether or not url encoded newlines will actually be rendered by your MTA as a new header value, regardless you could try something like

Code: Select all

$email = 'somestring';

$email_to_check = urldecode($email);
// check the decoded email


// send the string to the mail function
the decoded string was checked for newlines, the 'raw' string was sent to sendmail.. this should check encoded characters, but I still wouldn't personally put this code into my app.
User avatar
shiflett
Forum Contributor
Posts: 124
Joined: Sun Feb 06, 2005 11:22 am

Re: ctype_print for email injection prevention

Post by shiflett »

matthijs wrote:

Code: Select all

$_POST['email'] = preg_replace("/\r/", "", $_POST['email']); 
$_POST['email'] = preg_replace("/\n/", "", $_POST['email']);
This particular approach demonstrates a poor use of regular expressions, because both replacements match a literal string, not a pattern. It's best to use str_replace() in these cases.

You're right that ctype_print() can check for these cases, and it has the additional benefit of being locale-aware.
jshpro2 wrote:generally the header the attacker is overwriting is the "to" header
This isn't true, but I don't fault you - almost every example I've seen demonstrates this approach. The "real" attacks in the wild prefer to add BCC headers - lots of them.
Jenk wrote:To: is a field within the header, not it's own header.
The To header is indeed a header, so maybe you're just misunderstanding something?
matthijs wrote:Does it return false in any instance of a /r or /n?
Yes, assuming you mean \r and \n. Try this:

Code: Select all

<?php

var_dump(ctype_print("\r"));
var_dump(ctype_print("\n"));

?>
Buddha443556 wrote:ctype_print wouldn't be a safe option because it doesn't catch url escaped characters like "%0A".
What is there to "catch" in this case? :-)

I think you're getting your contexts confused.
matthijs wrote:But I also learned about the defense in depth principle.
Glad to hear this mentioned. :-)

To answer your original question. using ctype_print() seems like a perfectly reasonable Defense in Depth mechanism.
matthijs wrote:But one thing I don't understand is that with all examples given on the damonkohler.com site, it always echo's out "Ok", even with evil input as:
sender@anonymous.www%0ACc:recipient@someothersite.xxx%0ABcc:somebloke@grrrr.xxx,someotherbloke@oooops.xxx
That's not evil. :-)

Try this:

Code: Select all

<?php

mail('you@example.org',
     'My Evil Test',
     'My Evil Message',
     'From: sender@anonymous.www%0ACc:recipient@someothersite.xxx%0ABcc:somebloke@grrrr.xxx,someotherbloke@oooops.xxx');

?>
Passing a value in the URL offers no additional attack vectors. If you'll tell me where this is suggested, I'll try to email the author and get it corrected.

I hope this was helpful.
matthijs
DevNet Master
Posts: 3360
Joined: Thu Oct 06, 2005 3:57 pm

Post by matthijs »

Chris, that was certainly helpful. Thanks for your explanation.
using ctype_print() seems like a perfectly reasonable Defense in Depth mechanism
Good to know.
Passing a value in the URL offers no additional attack vectors. If you'll tell me where this is suggested, I'll try to email the author and get it corrected.
The examples are from http://securephp.damonkohler.com/index. ... _Injection.
Although I understand the reasoning behing the examples, they do not work on my server.
My guess is that it has something to do with the php settings on my server, because your example doesn't either produce the results I expect.

Code: Select all

<?php

mail('you@example.org',
     'My Evil Test',
     'My Evil Message',
     'From: sender@anonymous.www%0ACc:recipient@someothersite.xxx%0ABcc:somebloke@grrrr.xxx,someotherbloke@oooops.xxx');

?>
After replacing the email addresses with my own, only the 'you@example.org' receives an email. Whatever I try, no extra cc: or bcc: mail is sent. Of course, that's a good thing, but not being able to reproduce the examples is somewhat frustrating.
User avatar
Maugrim_The_Reaper
DevNet Master
Posts: 2704
Joined: Tue Nov 02, 2004 5:43 am
Location: Ireland

Post by Maugrim_The_Reaper »

Above post - surrounded by <strong> ;)

User sends you data - you assume its bad - you make certain its not bad. If all you expect is an email, then all you need do is validate it (filtering emails is basically validation in this instance). The most comprehensive email format checker I know can be found right here at viewtopic.php?t=38863 . This will validate any string as a valid email "form" (i.e. it will not check that the email exists on a remote server). There's a GPL'd (by Roja) and a Perl version of the same script linked from there somewhere.

As mentioned previously RFC compliant regex's are pretty large - but that's a small price to pay for something that works on all email addresses (not just the common variants). I still find applications which refuse to validate a simple "Padraic Brady"@example.com...
Roja
Tutorials Group
Posts: 2692
Joined: Sun Jan 04, 2004 10:30 pm

Post by Roja »

Maugrim_The_Reaper wrote:As mentioned previously RFC compliant regex's are pretty large - but that's a small price to pay for something that works on all email addresses (not just the common variants). I still find applications which refuse to validate a simple "Padraic Brady"@example.com...
Or Pádraic Brady, eh?

I think of email testing as a three step process.. someone correct me if I miss a potential step, please.

1. Validate email against RFC (Syntax checking)
2. Validate domain exists (DNS checking) - (Also, optionally check against spam blacklists)
3. Send the email and check for return or failure

The first I accomplish using the PHP variant of the definitive regex.

The second I accomplish by using a variant of checkdnsrr() and getmxrr(), with a fallback function for Windows to ensure cross-platform compatibility. An article on devshed has a fairly decent implementation.

The third I accomplish using phpmailer. It makes mailing very easy. :)

By the time I get to actually sending the email, I've eliminated a considerable scope of false emails. Can someone get an email address through that doesn't exist? Sure. But the risk is relatively low, and I've rarely seen it in happen in my implementations.

With phpmailer, you have extremely fine-grained control over each element in the email, from the to/cc/bcc fields seperately, down to the encoding of the email. It makes it possible to create a very strong web-driven email sending system.
Charles256
DevNet Resident
Posts: 1375
Joined: Fri Sep 16, 2005 9:06 pm

Post by Charles256 »

what all headers do you include to make sure your e-mails get through? even with phpmailer I've seen..I'd say...30% failure rate of e-mails just getting bounced by the other server..any clues on what headers to include to make sure I get through?
User avatar
Weirdan
Moderator
Posts: 5978
Joined: Mon Nov 03, 2003 6:13 pm
Location: Odessa, Ukraine

Post by Weirdan »

Roja wrote: someone correct me if I miss a potential step, please.
4. After a temporary failure (such as "try again later"), re-send the message until it gets through (or the message could not be delivered in acceptable time-frame, usually 5 days). That's where most of implementation using phpmailer with smtp mailer fail miserably. And that's why I would recommend to use real MTA instead of sending mail directly to recipient's MX.
Charles256
DevNet Resident
Posts: 1375
Joined: Fri Sep 16, 2005 9:06 pm

Post by Charles256 »

MTA and MX...define please for the ignorant..myself included ;)
User avatar
trukfixer
Forum Contributor
Posts: 174
Joined: Fri May 21, 2004 3:14 pm
Location: Miami, Florida, USA

Post by trukfixer »

Charles256 wrote:MTA and MX...define please for the ignorant..myself included ;)
MTA = Mail Transport Agent (what it sounds like - Postfix, Exim, etc)
MX = Mail eXchange (a type of DNS record - there's A, CNAME, MX, etc records for various types of dns lookups)
Roja
Tutorials Group
Posts: 2692
Joined: Sun Jan 04, 2004 10:30 pm

Post by Roja »

Charles256 wrote:what all headers do you include to make sure your e-mails get through? even with phpmailer I've seen..I'd say...30% failure rate of e-mails just getting bounced by the other server..any clues on what headers to include to make sure I get through?
Personally, my rate is *far* lower than that. I'd say below 5%, but it would be a back-of-the-envelope number. Its certainly rare for me, so its not far off.

I make sure to set Address (To:), Subject, Body, From, FromName, Host, CharSet, WordWrap, and Mailer type.

Technically, the last two are used by phpmailer, not technically included in the mail itself.

With those set, and coming from a non-blacklisted email address, I have very few problems getting mail out.
Weirdan wrote:4. After a temporary failure (such as "try again later"), re-send the message until it gets through (or the message could not be delivered in acceptable time-frame, usually 5 days). That's where most of implementation using phpmailer with smtp mailer fail miserably. And that's why I would recommend to use real MTA instead of sending mail directly to recipient's MX.
Thats odd. If you send the mail using phpmailer to via your smtp server, your smtp server should handle the retries if you receive "Try again later" codes. Thats the smtp server's responsibility. However, I've never had an issue with that. Perhaps I've been lucky so far?
Charles256 wrote:MTA and MX...define please for the ignorant..myself included
Mail Transfer Agent, and Mail Exchange. MX is the DNS record for a mail server.
User avatar
Maugrim_The_Reaper
DevNet Master
Posts: 2704
Joined: Tue Nov 02, 2004 5:43 am
Location: Ireland

Post by Maugrim_The_Reaper »

I'd almost surrendered my entirely valid Gaelic name to oblivion on the internet before people started using charsets properly. There are still sites though that replace the gaelic á with the infamous ?. P?draic is not my name... Doesn't suit me...;)

Hotmail still gets it wrong, of course. As well as most of the sites that stick a UTF-8 charset sticker in their HTML but then forget to save the HTML file with a UTF-8 encoding...or use odd charsets on their database...or...

Lots of reasons...actually. Its depressing come to think of it. What does an Irish guy need to do to get a simple name with one á character displayed correctly these days? :roll:

On the email failure rate... What failure rate? Unless you're blacklisted, or you have the headers all messed up (and that's hard with phpMailer), you should be fine. Of course the email addresses may not exist, or you could be hitting some server imposed limit (check your host's conditions of use), but generally I rarely get problems. Based on a complaints level X 100 measure its miniscule.

Anyone think of any other reasons for a 30% failure rate? Kinda curious now.
Post Reply