c-lexer-hack-typedef-semantic-feedback
IN premise — entries/2026/06/21/wiki-Tokenization_lexical_analysis.md
Created 2026-06-21T09:50:11+00:00
The C lexer hack requires semantic feedback from the symbol table to distinguish typedef names from variable names, since they are lexically identical.
Summary
In C, the token-reading phase cannot tell a type alias like "size_t" apart from a plain variable name just by looking at the characters, since they are textually identical. This forces the lexer to reach into the symbol table for context, creating a tight coupling between lexing and semantic analysis that complicates the compiler's modular design and makes it harder to reuse or replace individual front-end stages.
Dependents
These beliefs depend on this one:
- IN disambiguation-requires-semantic-context-across-processing-paradigms — Both compiler lexers and LLM inference face the same fundamental disambiguation problem: the C lexer hack requires semantic feedback from the symbol table to distinguish typedef names from variable names, and prompt injection exploits the model's inability to distinguish developer instructions from user inputs — in both cases, a processing layer cannot disambiguate its inputs without higher-level semantic context that it structurally lacks.