Hey everyone,
I'm creating a template engine and right now I have it setup using ereg_replace(). I've read around, and some sources say that using a tokenizer engine is faster. I was wondering if that is true, and if so where can I read up on tokenizers and how tokenizing works?
Tokenizing template engine
Moderator: General Moderators
-
php_wiz_kid
- Forum Contributor
- Posts: 181
- Joined: Tue Jun 24, 2003 7:33 pm
-
php_wiz_kid
- Forum Contributor
- Posts: 181
- Joined: Tue Jun 24, 2003 7:33 pm
- EvilWalrus
- Site Admin
- Posts: 209
- Joined: Thu Apr 18, 2002 3:21 pm
- Location: Springmont, PA USA
This code was originally for my AIM library, where i had to be able to reload PHP functions without having to shutdown and restart the script...
What the following function does ( Tokenizer::tokenize_module() ) is it tokenizes the requested file into 'function' and 'code'. This was custom created just for PHP's create_function....
What the following function does ( Tokenizer::tokenize_module() ) is it tokenizes the requested file into 'function' and 'code'. This was custom created just for PHP's create_function....
Code: Select all
<?php
/**
* Module Tokenizer Class
*/
class Tokenizer
{
private $file;
public function tokenize_module($file)
{
$this->file = file_get_contents($file);
$tokens = token_get_all($this->file);
$temp = '';
$func = array();
$code = '';
$i = 0;
$b = 0;
if (is_array($tokens)) {
foreach($tokens as $c) {
if ($i == 0 && is_array($c) && token_name($c[0]) == 'T_FUNCTION') {
$i++;
}
if ($i == 1 && is_array($c) && token_name($c[0]) == 'T_STRING') {
$tmp = explode('_', $c[1], 2);
if (!empty($tmp[1])) {
$temp = $tmp[0];
$func[$temp]['callback'] = $tmp[1];
$i++;
} else {
$i = 0;
}
}
if ($i == 2 && $c == '{') {
$i++;
}
if ($i == 3) {
if (is_array($c)) {
$code .= $c[1];
} else {
$code .= $c;
if ($c == '{') $b++;
if ($c == '}') $b--;
}
if ($b == 0) {
$func[$temp]['code'] = substr_replace(substr_replace($code,'', -1), '',0,1);
// important to reset the variables, or you might
// end up debugging your code for 5 days *cough*
$i = 0; $b = 0; $code = '';
}
}
}
return $func;
}
return false;
}
}
?>