Devel::Tokenizer::C 0.05

Devel::Tokenizer::C is a Perl module that can generate C source for fast keyword tokenizer. SYNOPSIS use Devel::Tokenizer::C; $t = new Devel::Tokenizer::C TokenFunc => sub { "return U$_[0];n" }; $t->add_tokens(qw( bar baz ))->add_tokens([for]); $t->add_tokens([qw( foo )], defined DIRECTIVE); print $t->generate; The Devel::Tokenizer::C module provides a small class for creating the essential ANSI C source code for a fast keyword tokenizer. The generated code is optimized for speed. On the ANSI-C keyword set, its 2-3 times faster than equivalent code generated with the gprof utility. The above example would print the following C source code: switch (tokstr[0]) { case b: switch (tokstr[1]) { case a: switch (tokstr[2]) { case r: if (tokstr[3] == ) { /* bar */ return BAR; } goto unknown; case z: if (tokstr[3] == ) { /* baz */ return BAZ; } goto unknown; default: goto unknown; } default: goto unknown; } case f: switch (tokstr[1]) { case o: switch (tokstr[2]) { #if defined DIRECTIVE case o: if (tokstr[3] == ) { /* foo */ return FOO; } goto unknown; #endif /* defined DIRECTIVE */ case r: if (tokstr[3] == ) { /* for */ return FOR; } goto unknown; default: goto unknown; } default: goto unknown; } default: goto unknown; } So the generated code only includes the main switch statement for the tokenizer. You can configure most of the generated code to fit for your application..

Requirements:	No special requirements
Platforms:	Linux
Keyword:	C Source,  Case,  Develtokenizerc,  Directive,  Goto,  Keyword,  Libraries,  Perl Module,  Programming,  Source,  Unknown
Users rating:	0/10