When you write a tree-sitter major mode, it’s tempting to think theparser handles everything. It doesn’t. Tree-sitter drives font-lock,indentation, and structural navigation, but a whole layer of everydayEmacs behavior still runs on the humble syntax table: forward-sexpand friends, electric-pair-mode, delete-pair, forward-word, andanything built on syntax-ppss. (I wrote about that structural layer inEssential Structured Navigation and Editing Commands.)I got a sharp reminder of this while buildingneocaml, my tree-sitter mode forOCaml. The grammar parsed everything beautifully, yet delete-pair andsexp motion kept misbehaving around a couple of OCaml constructs. Thefix turned out to be a classic tool that long predates tree-sitter:syntax-propertize-function. Here’s the story, since it’s usefulknowledge for anyone writing a mode.The problem: characters a static syntax table can’t classifyA syntax table assigns each character a single class: this one opens astring, that one is a comment starter, this is a word constituent. Thatworks until a character means different things in different places. OCamlhas two such troublemakers.Character literals. 'a' is a character, and the single quotesdelimit it. But the same ' also starts a type variable ('a in'a list), and can appear inside identifiers as a prime (x'). So youcan’t just declare ' a string delimiter. And if you leave it as asymbol constituent (the usual choice), a character literal whose contentshappen to be a delimiter wreaks havoc:let x = '"' (* the " opens a string, as far as the syntax table knows *)let y = '(' (* the ( is an unbalanced open paren *)Quoted string literals. OCaml’s {|...|} (and tagged {foo|...|foo})raw strings can contain anything – including ", (*, and friends –which the syntax table reads as real string/comment delimiters:let z = {|a "b" (* not a comment *)|}You can see the damage with syntax-ppss. With nothing but a statictable, point at the end of let x = '"' reports that you’re inside astring:(nth 3 (syntax-ppss)) ;; => 34 (i.e. inside a string opened by ")And every command that consults the syntax table inherits the confusion:C-M-f walks off into nonsense, delete-pair grabs the wrong delimiter,electric-pair-mode autopairs incorrectly.Note that font-lock looks fine here – tree-sitter fontifies theseconstructs correctly from the parse tree. That’s exactly what makes thebug sneaky: the buffer looks right, but the syntactic layer underneath islying.The fix: context-sensitive syntax with syntax-propertizeThe escape hatch issyntax-propertize-function:a buffer-local function that runs lazily over regions of the buffer andapplies syntax-table text properties to override the static tablewhere context demands it. Because the properties are attached to specificpositions, ' can be a string delimiter in 'a' and an ordinary symbolcharacter in 'a list – in the same buffer.The usual way to write one is with syntax-propertize-rules, which mapsregexps to the syntax classes to apply to their capture groups. Here’sthe heart of neocaml’s:(defun neocaml--syntax-propertize (start end) (goto-char start) (funcall (syntax-propertize-rules ;; Character literals: 'a', '\n', '"', '(', ... ;; Mark both quotes as string delimiters so the contents are inert. ;; A closing quote is required, so type variables ('a) are untouched. ("\\_