JavaScript Dissected: P3 - Parsers, Grammars and Differentials

September 29, 2026 25 min read

javascriptinternalsbug-bountywebdeepdivexssmxssparsershtmlurl

Table of Contents

Introduction

In Part 2 we followed one function through V8: source to tokens, tokens to an AST, bytecode, tiers, inline caches and deoptimization. That was the engine’s view of the world, and it starts at the moment source text arrives.

This part is about everything that happens before and around that moment. Source text doesn’t fall out of the sky. It gets carved out of an HTTP response by an HTML tokenizer, assembled from URLs, query strings and JSON, and often built by another program that concatenates strings. The engine can be perfect and you still get bugs, because the browser has several parsers chained together, and a value that looks harmless to one of them can mean something else to the next. Once you learn to see those boundaries, “find a payload” turns into “find where two components disagree about the same input.”

1. Stop thinking in payloads

Say an app gets this value:

/profile?next=%252Fdashboard%253Ftab%253Dsecurity

Before it does anything, it may go through all of this:

A reverse proxy percent-decodes it.
The backend parses it as a query string.
Application code decodes it a second time.
A template JSON-encodes it into an inline script.
The HTML tokenizer reads it as script data.
The JavaScript lexer and parser read it as a string literal.
new URL() parses it.
Something assigns it to location.

Every step consumes one representation and produces another:

R₀ ─decode→ R₁ ─query parse→ R₂ ─JSON encode→ R₃
   ─HTML tokenize/build→ DOM ─JS parse→ AST ─URL parse→ URL record

A check only means something if it validates the same semantic object the final consumer uses. If the backend approves R₁ but the browser navigates using R₂, the app validated the wrong thing.

Here’s the model I use for injection:

attacker-controlled bytes
  + transformation chain
  + parser state
  + interpreting sink
  = security consequence

The payload is one variable of four. In my experience the parser state decides the outcome more often than the payload does.

2. Where this part sits in the pipeline

Part 2 spent its time on the last two rows of this table. This part is about the first three, plus the runtime parsers that show up later.

Stage	Input	Output	Why it matters for security
Byte decoding	Network bytes	Code points	Encoding disagreements can move delimiters
HTML tokenization	Code points	Tokens	Decides whether data becomes text, a tag or an attribute
HTML tree construction	Tokens	DOM	Error recovery, namespaces and insertion modes reshape markup
ECMAScript lexing	Source text	Input elements	Covered in Part 2 (the slash problem, Unicode identifiers)
ECMAScript parsing	Tokens + grammar context	AST + early errors	Covered in Part 2 (lazy vs. eager parsing, AST)

After the engine’s front end, runtime APIs bring more parsers in: JSON.parse, URL, URLSearchParams, DOMParser, innerHTML, CSSOM setters, template engines and framework compilers.

A note on byte decoding: modern browsers follow the WHATWG Encoding Standard, which removed a lot of the exotic encodings that used to cause delimiter tricks (UTF-7 is the famous one). The stage still matters, because a charset mismatch between the Content-Type header, a BOM and a <meta> tag is worth checking. It’s just a much smaller attack surface than it was in 2010.

Rule of thumb: whenever data changes language, assume there’s a new trust boundary.

3. JavaScript grammar traps that matter for security

Part 2 covered how V8 tokenizes and parses. I won’t repeat that. What I want here is the handful of language-level rules that bite other programs: code generators, minifiers, sanitizers, template engines and scanners that handle JavaScript without being V8.

The reason they bite is that JavaScript has no context-free tokenization. The same characters lex differently depending on the grammar path (the slash problem from Part 2 is the standard example, and the spec’s lexical goal symbols such as InputElementDiv and InputElementRegExp are how it is formalized). Any tool that pretends otherwise is guessing.

3.1 Comments and line terminators

Comments vanish before AST analysis. Line terminators don’t:

function getToken() {
  return /* security token */
  generateToken();
}

The newline after the comment triggers the restricted production for return. The function returns undefined, and generateToken() becomes unreachable code. A multi-line comment that contains a line terminator also counts as a line terminator for this purpose.

Comments also matter when generated code gets concatenated:

const a = "doSafeThing(); //";
const b = "doSensitiveThing();";
const combined = a + b;   // the // swallows everything after it on that line

Code generators need to reason about token boundaries after joining fragments, not per fragment.

3.2 Statement vs. expression position

{ x: 1 }      // a block containing a labelled statement
({ x: 1 })    // an object literal

This matters whenever untrusted data lands in generated JavaScript. You need to know the exact production:

const value = USER_DATA;         // expression position
if (USER_DATA) doWork();         // expression position
const obj = { key: USER_DATA };  // value inside an object literal

“JavaScript-escape it” is underspecified until you say which production the data lands in.

3.3 Script vs. module

The host decides whether source is a classic script or a module, and that changes semantics:

Modules are always strict.
import and export are module-only.
Top-level await is only available in modules.
Top-level this is undefined in a module and the global object in a classic script.
In classic scripts, top-level var and function declarations become properties of the global object. In modules, top-level bindings stay module-scoped.

// module
console.log(this);   // undefined

// classic <script>
console.log(this);   // window

Static analysis has to use the source type that’s deployed. Parse a module as a script and you’ll either reject valid code or model bindings wrongly.

3.4 Early errors and parser fidelity

A program can be well-formed at the token level and still be illegal. The spec defines early errors, static-semantics rules that apply before execution: duplicate lexical bindings, break/continue targets that don’t exist, misuse of await, yield or private names.

The practical lesson from Part 2’s lazy parsing section is that an engine is allowed to be clever about how much work it does up front. A security tool has no such freedom. If it builds a tree, it should either apply the relevant static semantics or say clearly which constructs it doesn’t handle. And a tool that commits too early on ambiguous syntax, such as (a, b) versus (a, b) => a + b, will build a different tree from the engine and pass that mistake on to everything downstream.

4. ASI is grammar repair, not “adding missing semicolons”

The usual explanation is too loose. Automatic Semicolon Insertion happens only in specific cases:

A line terminator is followed by a token that the grammar doesn’t allow at that point.
The input ends, or the next token is }.
A restricted production forces a semicolon.

4.1 Restricted productions

Some tokens can’t be separated from what follows by a line terminator:

return
value
// parsed as: return; value;

The same restriction applies to postfix ++/--, continue, break, yield, async before function or an arrow head, and the => in arrow functions. throw is different. A newline after throw is a syntax error, and it doesn’t silently become throw;.

4.2 Concatenation

Two files can each be valid and still change meaning when joined:

// file-a.js
const result = getResult()

// file-b.js
["admin", "billing"].forEach(register)

Nothing forces a semicolon between them, so the result is:

const result = getResult()["admin", "billing"].forEach(register)

That’s legal. The bracket holds a comma expression, so it evaluates to "billing", and you’ve got property access on the return value of getResult(). Build systems should insert explicit separators and parse the final artifact.

4.3 Why it matters

ASI itself rarely produces a bounty-worthy bug. The lesson is methodological: line terminators are semantic input. A minifier, sanitizer, code generator or template engine that changes them can change the program.

5. The HTML parser: a state machine that repairs input

JavaScript usually arrives embedded in HTML, and HTML has its own tokenizer and tree builder:

bytes → decoder → code points → tokenizer → tokens → tree builder → DOM

The tokenizer moves between states such as Data, Tag open, Attribute name, Attribute value (quoted or unquoted), Comment, RCDATA, RAWTEXT and Script data. The tree builder tracks insertion modes (“in body”, “in table”, foreign content and so on) at the same time. Invalid markup isn’t rejected. It’s repaired by well-defined algorithms.

A filter that accepts or rejects strings while the browser builds a DOM is comparing two different objects.

5.1 Context is parser state

These placeholders are not the same destination:

<div>USER_DATA</div>
<div title="USER_DATA"></div>
<a href="USER_DATA">open</a>
<script>const value = "USER_DATA";</script>
<style>.x { background: USER_DATA }</style>

Destination	What interprets it	Preferred defense
HTML text	Data state	`textContent`, or HTML text encoding
Quoted attribute	Attribute-value state	Attribute encoding, plus validation of what the attribute means
URL attribute	HTML, then the URL parser	Build a URL object, allowlist scheme and origin
Inline JS string	Script data, then JS string grammar	Avoid it. Ship data as inert JSON
CSS value	HTML, then CSS grammar	Typed CSS APIs, or a strict allowlist

HTML entity encoding makes sense for the HTML parser. It isn’t a universal JavaScript-string encoder.

Attributes have a semantic layer too. onclick, srcdoc, href and src are code or URL carriers, so encoding them correctly doesn’t make an untrusted value safe.

5.2 `<script>` is a two-parser boundary

In a classic <script> element, the HTML tokenizer decides where the element ends. Only then does the JavaScript parser (the one from Part 2) see any source.

<script>
  const state = "USER_DATA";
</script>

That string still lives inside an outer HTML script-data context. A server-side encoder that knows JavaScript string escaping but not the HTML tokenizer’s rules can be wrong. Script data has its own escape states involving <!-- and <script, and it ends at </script followed by whitespace, / or >.

The robust approach is to avoid executable inline contexts and pass data through an inert channel:

<script type="application/json" id="bootstrap-data">
  {"theme":"dark"}
</script>
<script src="/static/app.js" nonce="..." defer></script>

Inert JSON in HTML still has to be serialized for the HTML context. The standard trick is to escape < as \u003c inside the JSON string values, which neutralizes </script, <!-- and <script in one go. Escaping > and & as well is cheap. U+2028 and U+2029 are legal in JS string literals since ES2019, but some older tooling still trips on them, so escaping them is harmless. Where the architecture allows, fetching the data from a separate application/json endpoint is better.

5.3 Parsing isn’t serialization

The DOM is a tree. innerHTML serializes a subtree back to text. Assigning that text to another innerHTML parses it again.

const once = container.innerHTML;
other.innerHTML = once;

Nothing guarantees the second tree equals the first. Quoting, namespaces, malformed structure and parser repair can all change the result. That lack of idempotence, parse ∘ serialize ≠ identity, is the basis of mutation XSS.

6. Parser differentials

A parser differential exists when two components consume related input and produce different semantic structures. Written out:

P₁(x)     = structure the security control inspects
P₂(T(x))  = structure the consumer executes, after transformations T

Vulnerable when:  Safe(P₁(x)) = true  and  Executable(P₂(T(x))) = true

The attacker’s job is to find an x in that gap.

6.1 Sanitizer vs. browser

A typical sanitizer parses the input into its own tree, removes disallowed nodes and attributes, serializes the result, and returns a string. The browser then parses that string again. If the sanitizer’s parser, serializer or namespace model disagrees with the browser’s, the second tree can contain executable structure that the first one didn’t.

So string replacement isn’t sanitization, and real sanitizers need constant maintenance.

6.2 Mutation XSS

The chain looks like this:

attacker markup → sanitizer DOM S₁ → sanitized string R → browser DOM S₂

The sanitizer wants dangerous(S₂) = false. What it actually established was dangerous(S₁) = false. When S₁ ≠ S₂, that proof doesn’t carry over.

Ingredients that show up again and again:

Foreign content (SVG, MathML) and transitions between namespaces. Namespace confusion is the recurring theme in bypasses of DOMPurify-class sanitizers.
Misnested formatting elements.
Table insertion modes and foster parenting.
Raw-text-like elements.
Serialization that changes quoting or text boundaries.
More than one innerHTML round trip.
A different fragment context in the second parse (for example div first, template or table second).

The lesson isn’t “block SVG.” Minimize reparsing, use a maintained sanitizer in its supported configuration, keep it patched, and test the final DOM at the real sink. If you can, look at Element.setHTML() and the Sanitizer API, but check current browser support before you rely on it.

6.3 URL differentials

A URL is a structured record. It isn’t a string with a trusted prefix. Components may disagree on:

Backslash vs. slash. The WHATWG parser treats \ like / for special schemes.
Userinfo boundaries (https://trusted.example@evil.example/).
Default ports.
Percent-decoding order.
Unicode and punycode.
Scheme-relative references.
Control characters and whitespace. The URL parser strips leading and trailing C0 controls and spaces, and removes ASCII tab and newline anywhere in the input, so java\nscript: is javascript: to the parser.
Dot-segment removal.
Duplicate query parameters.

A fragile check:

if (input.startsWith("https://trusted.example")) {
  location = input;
}

This accepts https://trusted.example.evil.com and https://trusted.example@evil.com/. The structural version parses once and validates the normalized record:

const candidate = new URL(input, location.origin);

if (candidate.protocol === "https:" &&
    candidate.origin === "https://trusted.example") {
  location.assign(candidate.href);
}

A javascript: URL has an opaque origin whose serialization is "null", so it fails the origin comparison. The explicit protocol check makes the intent clear to the next reader. Backend and frontend should share one canonicalization contract. “Both checked the URL” means nothing if they ran different parsers or validated at different points in the decoding chain.

6.4 Query-string differentials

Frameworks disagree on duplicate keys and nested notation:

role=user&role=admin
user[name]=a&user=admin
next=%252Fadmin

Depending on the stack, you get first-wins, last-wins, an array, a nested object, or a parse error. If a WAF validates one interpretation and the app authorizes another, duplicates and structured parameters cross the boundary.

When testing, record what each layer saw: the raw request, the proxy log, the framework’s request object, the application variable, the serialized response, and the browser API result. Don’t infer one from another.

6.5 Build-time vs. runtime

Babel, TypeScript, minifiers, bundlers and framework compilers all sit between what you review and what runs. Risk shows up when a transform reorders evaluation, a helper introduces a sink, dead-code elimination removes a guard on a wrong assumption, source maps point at stale source, plugins parse with different options, or the bundle joins tokens across file boundaries. For serious review, look at both the authored source and the production artifact.

7. From differential to DOM XSS

I model DOM XSS as data flow plus interpretation:

source → transformations → sink → parser → execution

Typical sources: location.href, search, hash, pathname, document.referrer, window.name, postMessage data, web storage, and API responses that contain user-controlled fields.

Typical sinks:

innerHTML, outerHTML, insertAdjacentHTML, setHTMLUnsafe
srcdoc
document.write
eval, Function, and string arguments to timers
Assignments to location and other URL-consuming properties
Script element creation and script.src assignment

The sink picks the next parser:

output.textContent = input;  // text node, no HTML parse
output.innerHTML   = input;  // HTML fragment parse

Same source, same string, different result, because the sink chooses the interpreter.

7.1 Context can change mid-flow

const data = JSON.parse(response);
const markup = `<button data-action="${data.action}">Run</button>`;
panel.innerHTML = markup;

JSON.parse only proves the outer input obeyed JSON grammar. The value is reinterpreted by the HTML parser afterwards, so it says nothing about safety.

7.2 Safe construction removes a parser

const button = document.createElement("button");
button.dataset.action = data.action;
button.textContent = "Run";
panel.replaceChildren(button);

This doesn’t remove every risk, because attribute semantics still matter. It does remove an HTML-fragment parse, which shrinks the state space a lot.

7.3 Two nearby cases worth knowing

DOM clobbering. Markup with id or name attributes can shadow properties on document or window, so injected HTML that can’t run script can still change what your JavaScript reads.
Dangling markup. An unterminated attribute or tag can make the tokenizer swallow trailing page content into a URL or form field. No script runs, but data can leave. It’s a good reminder that tokenizer state matters even when script execution is blocked.

8. Why WAF signatures lose this game

A WAF sees a request. The browser executes a structure that exists only after application-specific transformations. Signatures fail because of alternate encodings, decoding that happens after inspection, token concatenation, computed property access, browser error correction, context invisible at the perimeter, data assembled from several requests or storage locations, and harmless fragments combined client-side.

That doesn’t make WAFs useless. They cut commodity traffic and give you telemetry. They can’t prove a value is safe for a grammar they don’t model.

The primary controls are contextual output handling, safe DOM APIs, structural validation, maintained sanitizers, CSP and Trusted Types. A WAF is defense in depth.

9. A hunting method that isn’t payload roulette

When I test an authorized target, I start with a parser map.

Identify the source. Query parameter, fragment, path, header, message, storage or API field. Write down exactly where control starts.
Preserve the raw representation. Raw bytes if you can get them, the declared charset, the proxy-visible request, the application-visible value and the browser-visible value.
List the transformations. Percent decoding, entity decoding, JSON parse/stringify, Base64, Unicode normalization, case folding, template interpolation, concatenation, sanitization, DOM serialization.
Name every grammar boundary. Don’t write “rendered on the page.” Write query parser → JSON parser → JS template literal → HTML fragment parser → URL parser.
Find the exact parser state. For HTML: text, quoted attribute, unquoted attribute, script data, style data or foreign content. For JS: string, template literal, expression, property name, regex or comment.
Inspect the structure that was produced. Compare the network response, the DOM tree, element attributes versus properties, and serialized outerHTML. View Source and the Elements panel answer different questions.
Test representation drift. Compare structures after each parse/serialize cycle, in a local harness that never touches a live page and never runs hostile content.
Claim impact only after you have control. Prove the smallest useful primitive: markup injection, attribute control, URL navigation, script execution, cross-origin data access, or a privileged action in the victim’s context. Reflection alone isn’t impact.

10. Static analysis: an AST is necessary, not sufficient

Part 2 showed how to walk an AST. Here’s the security catch: an AST scanner can reliably find direct sinks,

element.innerHTML = value;
eval(value);
location.assign(value);

but it misses aliases and computed names:

const sink = element;
sink["inner" + "HTML"] = value;

const run = globalThis["ev" + "al"];
run(value);

A more accurate analyzer stacks several layers:

AST matching → scope resolution → constant folding → alias tracking
  → local data flow → interprocedural flow → framework semantics → sink context

It also has to tell same-spelling identifiers apart. Here’s a case where scope resolution decides the answer:

(function () {
  function eval(value) { return value; }   // lexically scoped
  eval(userInput);                         // calls the local function
})();

Inside the function, eval resolves to the local binding. If you declared function eval at the top level of a classic script instead, you’d be replacing the global property, which is a different (and much nastier) thing. Also remember that direct eval(x) and indirect (0, eval)(x) have different scope semantics.

The AST says “this is a call.” Scope analysis says which function. Data flow says whether attacker data reaches it. Sink modeling says whether the callee interprets its argument. This is why low-false-positive tooling is a semantic-analysis problem, and regexes don’t get you there.

11. Defensive invariants

These are the seven statements I’d want to be true for a codebase:

Parse once. Untrusted markup shouldn’t cycle through parse → serialize → parse.
Validate structured objects. Parse URLs with one agreed algorithm, then check scheme, origin, host, port and path on the record.
Keep data out of executable grammars. Use textContent, createElement, typed properties, structured messages and JSON APIs instead of string-built HTML or JS.
Encode for the immediate destination. The right encoding depends on the next parser state, not on where the data came from.
Inspect production output. Analyze final bundles and the final DOM, not only templates and source.
Constrain dangerous sinks. Use CSP and Trusted Types where you can. They don’t replace correct handling, but they cut the number of paths that can turn a string into executable structure.
Differential-test the boundaries. For parsers, sanitizers and canonicalizers, keep a corpus of malformed and adversarial inputs, and compare semantic outputs across components and versions. Comparing accept/reject decisions alone isn’t enough.

At every arrow, ask:

What exact representation exists here?
Who interprets it next?
Under which parser state?
Can validation and execution disagree?
Can I remove this parse entirely?

Thanks for Reading <3