May 09, 2002

Regex Compiler Experiments

After further testing the new sexpr-generating backend, I think I'm gonna have to abandon it. While it works just fine, the compile times get incredibly long for complicated patterns. The problem seems to be in the Lispworks compiler itself, trying to chew on the large number of internal functions, and the sheer size of the code.

Given the relatively small improvements in matching speed, I think my next tack is to rewrite it to simply generate code to build a closure-based matcher.

Posted by: mparker762 at 04:58 AM
Post contains 85 words, total size 1 kb.

May 08, 2002

New Regex Compiler Backend

It looks like the new back-end to my common-lisp regular expression engine is about finished. This one returns lisp s-exprs instead of closures, so it's suitable for use in macros like deflexer. It still needs a bit more brushing up before I'm ready to make it available, but it looks good so far. At the moment it doesn't seem to match any faster than the closure-based code. Using Lispworks 4.2, turning on inlining seems to improve the speed by 30% or so, but increases compile times exponentially and unacceptably. The old sexpr-generating regex compiler had the same unpleasant exploding-compile-time behavior with the Symbolics compiler, although there it gave an order-of-magnitude improvement in matching speed. At any rate, given the specialized nature of the code involved, I'm pretty sure I can hand-roll a customized inliner that that is linear-time.

Posted by: mparker762 at 03:48 AM
Post contains 142 words, total size 1 kb.

<< Page 1 of 1 >>
12kb generated in CPU 0.0085, elapsed 0.0378 seconds.
41 queries taking 0.0319 seconds, 101 records returned.
Powered by Minx 1.1.6c-pink.