php – Profiling Regex Lexer

private function tokenize($pattern)
{
    $rules = array(
        self::OPEN_PAREN_TYPE  => '/^(()/',self::CLOSE_PAREN_TYPE => '/^())/',self::VARIABLE_TYPE    => '/^:([a-z0-9_]+)/',self::TEXT_TYPE        => '/^([^:()]+)/',);

    $cursor = 0;
    $tokens = array();
    $buffer = $pattern;
    $buflen = strlen($buffer);

    while ($cursor < $buflen)
    {
        $chunk = substr($buffer,$cursor);

        $matched = false;
        foreach ($rules as $type => $rule)
        {
            if (preg_match($rule,$chunk,$matches))
            {
                $tokens[] = array(
                    'type'  => $type,'value' => $matches[1],);

                $matched = true;
                $cursor += strlen($matches[0]);
            }
        }

        if (!$matched)
        {
            throw new Exception(sprintf('Problem parsing route "%s" at char "%d".',$pattern,$cursor));
        }
    }

    return $tokens;
}

我缺少哪些明显的加速？有什么方法可以放弃preg_ *,或者将正则表达式组合成一种模式等等？

这是xhprof调用图(基准测试使用~255个独特的测试路径)：

我知道最好的解决方案是不要为每个请求调用这个(我计划使用APC进行缓存等),但是对于使用没有启用APC的库的人来说,尽可能高效.

编辑：

我还写了一个快速的状态机版本,它似乎表现更好.我仍然愿意接受前面的建议,因为我相信第一个代码更优雅.

private function tokenize2($pattern)
{
    $buffer = '';
    $invariable = false;

    $tokens = array();
    foreach (str_split($pattern) as $char)
    {
        switch ($char)
        {
            case '(':
                if ($buffer)
                {
                    $tokens[] = array(
                        'type'  => $invariable ? self::VARIABLE_TYPE : self::TEXT_TYPE,'value' => $buffer,);
                    $buffer = '';
                    $invariable = false;
                }

                $tokens[] = array(
                    'type' => self::OPEN_PAREN_TYPE,);
                break;
            case ')':
                if ($buffer)
                {
                    $tokens[] = array(
                        'type'  => $invariable ? self::VARIABLE_TYPE : self::TEXT_TYPE,);
                    $buffer = '';
                    $invariable = false;
                }

                $tokens[] = array(
                    'type' => self::CLOSE_PAREN_TYPE,);
                break;
            case ':':
                if ($buffer)
                {
                    $tokens[] = array(
                        'type'  => $invariable ? self::VARIABLE_TYPE : self::TEXT_TYPE,);
                    $buffer = '';
                    $invariable = false;
                }

                $invariable = true;
                break;
            default:
                if ($invariable && !(ctype_alnum($char) || '_' == $char ))
                {
                    $invariable = false;
                    $tokens[] = array(
                        'type'  => self::VARIABLE_TYPE,);

                    $buffer = '';
                    $invariable = false;
                }

                $buffer .= $char;
                break;
        }
    }

    if ($buffer)
    {
        $tokens[] = array(
        'type'  => $invariable ? self::VARIABLE_TYPE : self::TEXT_TYPE,);
        $buffer = '';
    }

    return $tokens;

我最后只是出于性能原因使用状态机,并使用APC缓存整个lexing过程(因为……为什么不这样做).

如果有人有任何贡献,我会很乐意回答.