Formatting & Layout Stress Test

Some Text

Computational predictions are not diagnoses. But better computational tools mean better inputs to clinical decisions – and a more interpretable, genome-wide approach to variant effect prediction is a significant advance in what these tools can do.

Inline Math and Prose

A second inline expression with subscripts and operators: $\mathbb{E}\!\left[\Delta\mathcal{L}_{t+1}\mid\mathcal{F}_t\right]$ should also flow naturally.¹ This starter section tests inline math interleaved with dense prose. An expression like $\mu_n=\frac{\mu_0/\sigma_0^2 + n\bar{x}/\sigma^2}{1/\sigma_0^2 + n/\sigma^2}$ should sit comfortably inside a normal paragraph without breaking the line rhythm.² Computational predictions are not diagnoses. But better computational tools mean better inputs to clinical decisions – and a more interpretable, genome-wide approach to variant effect prediction is a significant advance in what these tools can do.

A Bayesian Update in Code

Inline code references like jax.grad(loss_fn) or torch.compile(model) should stay compact.³

from dataclasses import dataclass

@dataclass
class GaussianPosterior:
    mean: float
    variance: float


def posterior(mu0: float, sigma0: float, xbar: float, sigma: float, n: int) -> GaussianPosterior:
    prior_precision = 1.0 / (sigma0 ** 2)
    data_precision = n / (sigma ** 2)
    mean = (prior_precision * mu0 + data_precision * xbar) / (prior_precision + data_precision)
    variance = 1.0 / (prior_precision + data_precision)
    return GaussianPosterior(mean=mean, variance=variance)


print(posterior(0.0, 1.0, 0.18, 0.55, 60))

Display Math

Display equations should center cleanly and not interfere with the sidenote rail:

$$\log p(\theta \mid D) = \log p(\theta) + \sum_{i=1}^{n} \log p(x_i \mid \theta) - \log p(D)$$

A second display equation with more vertical complexity:⁴

$$\hat{W}_{\text{proj}} = W_{\text{proj}} + \frac{(\mathbf{v}_1 - W_{\text{proj}} \, \mathbf{k}_1) \, \mathbf{k}_1^\top}{\mathbf{k}_1^\top \mathbf{k}_1}$$

Tables

Method	Efficacy	Generalization	Specificity
Fine-tuning	92%	41%	63%
ROME	99%	94%	98%
MEMIT	98%	91%	97%
Constrained FT	95%	72%	85%

Blockquotes and Nested Lists

The MLP layers of a transformer are not merely nonlinear feature extractors. They function as distributed associative memories, with each neuron contributing a fragment of the retrieval.

Nested list structure:

• Causal tracing identifies critical mediators
  • Clean run: record all hidden states
  • Corrupted run: perturb subject embeddings
  • Restoration: patch in clean states one at a time
• ROME modifies the MLP weight matrix
  1. Compute context-invariant key
  2. Optimize target value vector⁵
  3. Apply rank-one update to $W_{\text{proj}}$

Dense Stress Pass

Dense technical prose should feel deliberate, not loose. Every line carries signal: references stay compact, annotations remain local, and long tokens degrade gracefully instead of tearing the layout.⁶ This is exactly why sidenotes matter for research writing; if you have to jump to the bottom of the page constantly, you lose the thread and your local stack of assumptions collapses.

To stress wrapping, this paragraph includes emphasized long tokens like counterfactualtraceabilitywithoutsemanticcompressionandwithoutauditboundaries⁷ and hyperlink text such as modelrevisionpipelinewithoutcheckpointfences.⁸

It also embeds a long inline code reference: stream_validator.collector.chain_stage_without_manual_breakpoints_or_aliases("batch:alpha:reconciliation")⁹ to verify that code wraps naturally.

Zoom and Reflow Behavior

Screen-awareness matters: a wrapping strategy that works at one viewport and fails at another is not robust enough. This paragraph carries a long unbroken token in normal prose, zoomsynchronizedreflowverificationprotocolwithoutescapehatchesoradhoctruncationfallbacks, plus two inline math cases that stress baseline alignment: $\operatorname{PosteriorShift}_{\text{ultralongdiagnosticdescriptorwithoutmanuallinebreaks}}(\theta)$¹⁰ and $\left(\dfrac{\sum_{i=1}^{n}\left(x_i-\bar{x}\right)^2}{\sqrt{\sum_{i=1}^{n}\left(y_i-\bar{y}\right)^2}+\epsilon}\right)^{\!2}$.¹¹

Sidenote Density

This section packs footnote references close together to stress sidenote collision detection. First claim.¹² Second claim.¹³ Third claim.¹⁴ These three should stack neatly in the margin without overlapping.

A paragraph with a reused reference: this claim appears here¹⁵ and again here¹⁵ to verify that each instance gets its own sidenote positioned near its reference.

---

TypeScript Code Block

interface FactualEdit {
  subject: string;
  relation: string;
  oldObject: string;
  newObject: string;
  layer: number;
}

function applyROME(
  model: TransformerModel,
  edit: FactualEdit,
): TransformerModel {
  const key = computeSubjectKey(model, edit.subject, edit.layer);
  const value = optimizeTargetValue(model, edit, key);

  const W = model.layers[edit.layer].mlp.W_proj;
  const residual = subtractMatVec(value, matVec(W, key));
  const delta = outerProduct(residual, key);
  const scale = 1 / dot(key, key);

  model.layers[edit.layer].mlp.W_proj = addMat(W, scaleMat(delta, scale));
  return model;
}

Inline Math and Prose

This starter section tests inline math interleaved with dense prose. An expression like $\mu_n=\frac{\mu_0/\sigma_0^2 + n\bar{x}/\sigma^2}{1/\sigma_0^2 + n/\sigma^2}$ should sit comfortably inside a normal paragraph without breaking the line rhythm.²

A second inline expression with subscripts and operators: $\mathbb{E}\!\left[\Delta\mathcal{L}_{t+1}\mid\mathcal{F}_t\right]$ should also flow naturally.¹

A Bayesian Update in Code

Inline code references like jax.grad(loss_fn) or torch.compile(model) should stay compact.³

from dataclasses import dataclass

@dataclass
class GaussianPosterior:
    mean: float
    variance: float


def posterior(mu0: float, sigma0: float, xbar: float, sigma: float, n: int) -> GaussianPosterior:
    prior_precision = 1.0 / (sigma0 ** 2)
    data_precision = n / (sigma ** 2)
    mean = (prior_precision * mu0 + data_precision * xbar) / (prior_precision + data_precision)
    variance = 1.0 / (prior_precision + data_precision)
    return GaussianPosterior(mean=mean, variance=variance)


print(posterior(0.0, 1.0, 0.18, 0.55, 60))

Display Math

Display equations should center cleanly and not interfere with the sidenote rail:

$$\log p(\theta \mid D) = \log p(\theta) + \sum_{i=1}^{n} \log p(x_i \mid \theta) - \log p(D)$$

A second display equation with more vertical complexity:⁴

$$\hat{W}_{\text{proj}} = W_{\text{proj}} + \frac{(\mathbf{v}_1 - W_{\text{proj}} \, \mathbf{k}_1) \, \mathbf{k}_1^\top}{\mathbf{k}_1^\top \mathbf{k}_1}$$

Tables

Method	Efficacy	Generalization	Specificity
Fine-tuning	92%	41%	63%
ROME	99%	94%	98%
MEMIT	98%	91%	97%
Constrained FT	95%	72%	85%

Blockquotes and Nested Lists

The MLP layers of a transformer are not merely nonlinear feature extractors. They function as distributed associative memories, with each neuron contributing a fragment of the retrieval.

Nested list structure:

• Causal tracing identifies critical mediators
  • Clean run: record all hidden states
  • Corrupted run: perturb subject embeddings
  • Restoration: patch in clean states one at a time
• ROME modifies the MLP weight matrix
  1. Compute context-invariant key
  2. Optimize target value vector⁵
  3. Apply rank-one update to $W_{\text{proj}}$

Dense Stress Pass

Dense technical prose should feel deliberate, not loose. Every line carries signal: references stay compact, annotations remain local, and long tokens degrade gracefully instead of tearing the layout.⁶ This is exactly why sidenotes matter for research writing; if you have to jump to the bottom of the page constantly, you lose the thread and your local stack of assumptions collapses.

To stress wrapping, this paragraph includes emphasized long tokens like counterfactualtraceabilitywithoutsemanticcompressionandwithoutauditboundaries⁷ and hyperlink text such as modelrevisionpipelinewithoutcheckpointfences.⁸

It also embeds a long inline code reference: stream_validator.collector.chain_stage_without_manual_breakpoints_or_aliases("batch:alpha:reconciliation")⁹ to verify that code wraps naturally.

Zoom and Reflow Behavior

Screen-awareness matters: a wrapping strategy that works at one viewport and fails at another is not robust enough. This paragraph carries a long unbroken token in normal prose, zoomsynchronizedreflowverificationprotocolwithoutescapehatchesoradhoctruncationfallbacks, plus two inline math cases that stress baseline alignment: $\operatorname{PosteriorShift}_{\text{ultralongdiagnosticdescriptorwithoutmanuallinebreaks}}(\theta)$¹⁰ and $\left(\dfrac{\sum_{i=1}^{n}\left(x_i-\bar{x}\right)^2}{\sqrt{\sum_{i=1}^{n}\left(y_i-\bar{y}\right)^2}+\epsilon}\right)^{\!2}$.¹¹

Sidenote Density

This section packs footnote references close together to stress sidenote collision detection. First claim.¹² Second claim.¹³ Third claim.¹⁴ These three should stack neatly in the margin without overlapping.

A paragraph with a reused reference: this claim appears here¹⁵ and again here¹⁵ to verify that each instance gets its own sidenote positioned near its reference.

---

TypeScript Code Block

interface FactualEdit {
  subject: string;
  relation: string;
  oldObject: string;
  newObject: string;
  layer: number;
}

function applyROME(
  model: TransformerModel,
  edit: FactualEdit,
): TransformerModel {
  const key = computeSubjectKey(model, edit.subject, edit.layer);
  const value = optimizeTargetValue(model, edit, key);

  const W = model.layers[edit.layer].mlp.W_proj;
  const residual = subtractMatVec(value, matVec(W, key));
  const delta = outerProduct(residual, key);
  const scale = 1 / dot(key, key);

  model.layers[edit.layer].mlp.W_proj = addMat(W, scaleMat(delta, scale));
  return model;
}

Inline Math and Prose

This starter section tests inline math interleaved with dense prose. An expression like $\mu_n=\frac{\mu_0/\sigma_0^2 + n\bar{x}/\sigma^2}{1/\sigma_0^2 + n/\sigma^2}$ should sit comfortably inside a normal paragraph without breaking the line rhythm.²

A second inline expression with subscripts and operators: $\mathbb{E}\!\left[\Delta\mathcal{L}_{t+1}\mid\mathcal{F}_t\right]$ should also flow naturally.¹

A Bayesian Update in Code

Inline code references like jax.grad(loss_fn) or torch.compile(model) should stay compact.³

from dataclasses import dataclass

@dataclass
class GaussianPosterior:
    mean: float
    variance: float


def posterior(mu0: float, sigma0: float, xbar: float, sigma: float, n: int) -> GaussianPosterior:
    prior_precision = 1.0 / (sigma0 ** 2)
    data_precision = n / (sigma ** 2)
    mean = (prior_precision * mu0 + data_precision * xbar) / (prior_precision + data_precision)
    variance = 1.0 / (prior_precision + data_precision)
    return GaussianPosterior(mean=mean, variance=variance)


print(posterior(0.0, 1.0, 0.18, 0.55, 60))

Display Math

Display equations should center cleanly and not interfere with the sidenote rail:

$$\log p(\theta \mid D) = \log p(\theta) + \sum_{i=1}^{n} \log p(x_i \mid \theta) - \log p(D)$$

A second display equation with more vertical complexity:⁴

$$\hat{W}_{\text{proj}} = W_{\text{proj}} + \frac{(\mathbf{v}_1 - W_{\text{proj}} \, \mathbf{k}_1) \, \mathbf{k}_1^\top}{\mathbf{k}_1^\top \mathbf{k}_1}$$

Tables

Method	Efficacy	Generalization	Specificity
Fine-tuning	92%	41%	63%
ROME	99%	94%	98%
MEMIT	98%	91%	97%
Constrained FT	95%	72%	85%

Blockquotes and Nested Lists

The MLP layers of a transformer are not merely nonlinear feature extractors. They function as distributed associative memories, with each neuron contributing a fragment of the retrieval.

Nested list structure:

• Causal tracing identifies critical mediators
  • Clean run: record all hidden states
  • Corrupted run: perturb subject embeddings
  • Restoration: patch in clean states one at a time
• ROME modifies the MLP weight matrix
  1. Compute context-invariant key
  2. Optimize target value vector⁵
  3. Apply rank-one update to $W_{\text{proj}}$

Dense Stress Pass

Dense technical prose should feel deliberate, not loose. Every line carries signal: references stay compact, annotations remain local, and long tokens degrade gracefully instead of tearing the layout.⁶ This is exactly why sidenotes matter for research writing; if you have to jump to the bottom of the page constantly, you lose the thread and your local stack of assumptions collapses.

To stress wrapping, this paragraph includes emphasized long tokens like counterfactualtraceabilitywithoutsemanticcompressionandwithoutauditboundaries⁷ and hyperlink text such as modelrevisionpipelinewithoutcheckpointfences.⁸

It also embeds a long inline code reference: stream_validator.collector.chain_stage_without_manual_breakpoints_or_aliases("batch:alpha:reconciliation")⁹ to verify that code wraps naturally.

Zoom and Reflow Behavior

Screen-awareness matters: a wrapping strategy that works at one viewport and fails at another is not robust enough. This paragraph carries a long unbroken token in normal prose, zoomsynchronizedreflowverificationprotocolwithoutescapehatchesoradhoctruncationfallbacks, plus two inline math cases that stress baseline alignment: $\operatorname{PosteriorShift}_{\text{ultralongdiagnosticdescriptorwithoutmanuallinebreaks}}(\theta)$¹⁰ and $\left(\dfrac{\sum_{i=1}^{n}\left(x_i-\bar{x}\right)^2}{\sqrt{\sum_{i=1}^{n}\left(y_i-\bar{y}\right)^2}+\epsilon}\right)^{\!2}$.¹¹

Sidenote Density

This section packs footnote references close together to stress sidenote collision detection. First claim.¹² Second claim.¹³ Third claim.¹⁴ These three should stack neatly in the margin without overlapping.

A paragraph with a reused reference: this claim appears here¹⁵ and again here¹⁵ to verify that each instance gets its own sidenote positioned near its reference.

---

TypeScript Code Block

interface FactualEdit {
  subject: string;
  relation: string;
  oldObject: string;
  newObject: string;
  layer: number;
}

function applyROME(
  model: TransformerModel,
  edit: FactualEdit,
): TransformerModel {
  const key = computeSubjectKey(model, edit.subject, edit.layer);
  const value = optimizeTargetValue(model, edit, key);

  const W = model.layers[edit.layer].mlp.W_proj;
  const residual = subtractMatVec(value, matVec(W, key));
  const delta = outerProduct(residual, key);
  const scale = 1 / dot(key, key);

  model.layers[edit.layer].mlp.W_proj = addMat(W, scaleMat(delta, scale));
  return model;
}

Inline Math and Prose

This starter section tests inline math interleaved with dense prose. An expression like $\mu_n=\frac{\mu_0/\sigma_0^2 + n\bar{x}/\sigma^2}{1/\sigma_0^2 + n/\sigma^2}$ should sit comfortably inside a normal paragraph without breaking the line rhythm.²

A second inline expression with subscripts and operators: $\mathbb{E}\!\left[\Delta\mathcal{L}_{t+1}\mid\mathcal{F}_t\right]$ should also flow naturally.¹

A Bayesian Update in Code

Inline code references like jax.grad(loss_fn) or torch.compile(model) should stay compact.³

from dataclasses import dataclass

@dataclass
class GaussianPosterior:
    mean: float
    variance: float


def posterior(mu0: float, sigma0: float, xbar: float, sigma: float, n: int) -> GaussianPosterior:
    prior_precision = 1.0 / (sigma0 ** 2)
    data_precision = n / (sigma ** 2)
    mean = (prior_precision * mu0 + data_precision * xbar) / (prior_precision + data_precision)
    variance = 1.0 / (prior_precision + data_precision)
    return GaussianPosterior(mean=mean, variance=variance)


print(posterior(0.0, 1.0, 0.18, 0.55, 60))

Display Math

Display equations should center cleanly and not interfere with the sidenote rail:

$$\log p(\theta \mid D) = \log p(\theta) + \sum_{i=1}^{n} \log p(x_i \mid \theta) - \log p(D)$$

A second display equation with more vertical complexity:⁴

$$\hat{W}_{\text{proj}} = W_{\text{proj}} + \frac{(\mathbf{v}_1 - W_{\text{proj}} \, \mathbf{k}_1) \, \mathbf{k}_1^\top}{\mathbf{k}_1^\top \mathbf{k}_1}$$

Tables

Method	Efficacy	Generalization	Specificity
Fine-tuning	92%	41%	63%
ROME	99%	94%	98%
MEMIT	98%	91%	97%
Constrained FT	95%	72%	85%

Blockquotes and Nested Lists

The MLP layers of a transformer are not merely nonlinear feature extractors. They function as distributed associative memories, with each neuron contributing a fragment of the retrieval.

Nested list structure:

• Causal tracing identifies critical mediators
  • Clean run: record all hidden states
  • Corrupted run: perturb subject embeddings
  • Restoration: patch in clean states one at a time
• ROME modifies the MLP weight matrix
  1. Compute context-invariant key
  2. Optimize target value vector⁵
  3. Apply rank-one update to $W_{\text{proj}}$

Dense Stress Pass

Dense technical prose should feel deliberate, not loose. Every line carries signal: references stay compact, annotations remain local, and long tokens degrade gracefully instead of tearing the layout.⁶ This is exactly why sidenotes matter for research writing; if you have to jump to the bottom of the page constantly, you lose the thread and your local stack of assumptions collapses.

To stress wrapping, this paragraph includes emphasized long tokens like counterfactualtraceabilitywithoutsemanticcompressionandwithoutauditboundaries⁷ and hyperlink text such as modelrevisionpipelinewithoutcheckpointfences.⁸

It also embeds a long inline code reference: stream_validator.collector.chain_stage_without_manual_breakpoints_or_aliases("batch:alpha:reconciliation")⁹ to verify that code wraps naturally.

Zoom and Reflow Behavior

Screen-awareness matters: a wrapping strategy that works at one viewport and fails at another is not robust enough. This paragraph carries a long unbroken token in normal prose, zoomsynchronizedreflowverificationprotocolwithoutescapehatchesoradhoctruncationfallbacks, plus two inline math cases that stress baseline alignment: $\operatorname{PosteriorShift}_{\text{ultralongdiagnosticdescriptorwithoutmanuallinebreaks}}(\theta)$¹⁰ and $\left(\dfrac{\sum_{i=1}^{n}\left(x_i-\bar{x}\right)^2}{\sqrt{\sum_{i=1}^{n}\left(y_i-\bar{y}\right)^2}+\epsilon}\right)^{\!2}$.¹¹

Sidenote Density

This section packs footnote references close together to stress sidenote collision detection. First claim.¹² Second claim.¹³ Third claim.¹⁴ These three should stack neatly in the margin without overlapping.

A paragraph with a reused reference: this claim appears here¹⁵ and again here¹⁵ to verify that each instance gets its own sidenote positioned near its reference.

---

TypeScript Code Block

interface FactualEdit {
  subject: string;
  relation: string;
  oldObject: string;
  newObject: string;
  layer: number;
}

function applyROME(
  model: TransformerModel,
  edit: FactualEdit,
): TransformerModel {
  const key = computeSubjectKey(model, edit.subject, edit.layer);
  const value = optimizeTargetValue(model, edit, key);

  const W = model.layers[edit.layer].mlp.W_proj;
  const residual = subtractMatVec(value, matVec(W, key));
  const delta = outerProduct(residual, key);
  const scale = 1 / dot(key, key);

  model.layers[edit.layer].mlp.W_proj = addMat(W, scaleMat(delta, scale));
  return model;
}

Inline Math and Prose

This starter section tests inline math interleaved with dense prose. An expression like $\mu_n=\frac{\mu_0/\sigma_0^2 + n\bar{x}/\sigma^2}{1/\sigma_0^2 + n/\sigma^2}$ should sit comfortably inside a normal paragraph without breaking the line rhythm.²

A second inline expression with subscripts and operators: $\mathbb{E}\!\left[\Delta\mathcal{L}_{t+1}\mid\mathcal{F}_t\right]$ should also flow naturally.¹

A Bayesian Update in Code

Inline code references like jax.grad(loss_fn) or torch.compile(model) should stay compact.³

from dataclasses import dataclass

@dataclass
class GaussianPosterior:
    mean: float
    variance: float


def posterior(mu0: float, sigma0: float, xbar: float, sigma: float, n: int) -> GaussianPosterior:
    prior_precision = 1.0 / (sigma0 ** 2)
    data_precision = n / (sigma ** 2)
    mean = (prior_precision * mu0 + data_precision * xbar) / (prior_precision + data_precision)
    variance = 1.0 / (prior_precision + data_precision)
    return GaussianPosterior(mean=mean, variance=variance)


print(posterior(0.0, 1.0, 0.18, 0.55, 60))

Display Math

Display equations should center cleanly and not interfere with the sidenote rail:

$$\log p(\theta \mid D) = \log p(\theta) + \sum_{i=1}^{n} \log p(x_i \mid \theta) - \log p(D)$$

A second display equation with more vertical complexity:⁴

$$\hat{W}_{\text{proj}} = W_{\text{proj}} + \frac{(\mathbf{v}_1 - W_{\text{proj}} \, \mathbf{k}_1) \, \mathbf{k}_1^\top}{\mathbf{k}_1^\top \mathbf{k}_1}$$

Tables

Method	Efficacy	Generalization	Specificity
Fine-tuning	92%	41%	63%
ROME	99%	94%	98%
MEMIT	98%	91%	97%
Constrained FT	95%	72%	85%

Blockquotes and Nested Lists

The MLP layers of a transformer are not merely nonlinear feature extractors. They function as distributed associative memories, with each neuron contributing a fragment of the retrieval.

Nested list structure:

• Causal tracing identifies critical mediators
  • Clean run: record all hidden states
  • Corrupted run: perturb subject embeddings
  • Restoration: patch in clean states one at a time
• ROME modifies the MLP weight matrix
  1. Compute context-invariant key
  2. Optimize target value vector⁵
  3. Apply rank-one update to $W_{\text{proj}}$

Dense Stress Pass

Dense technical prose should feel deliberate, not loose. Every line carries signal: references stay compact, annotations remain local, and long tokens degrade gracefully instead of tearing the layout.⁶ This is exactly why sidenotes matter for research writing; if you have to jump to the bottom of the page constantly, you lose the thread and your local stack of assumptions collapses.

To stress wrapping, this paragraph includes emphasized long tokens like counterfactualtraceabilitywithoutsemanticcompressionandwithoutauditboundaries⁷ and hyperlink text such as modelrevisionpipelinewithoutcheckpointfences.⁸

It also embeds a long inline code reference: stream_validator.collector.chain_stage_without_manual_breakpoints_or_aliases("batch:alpha:reconciliation")⁹ to verify that code wraps naturally.

Zoom and Reflow Behavior

Screen-awareness matters: a wrapping strategy that works at one viewport and fails at another is not robust enough. This paragraph carries a long unbroken token in normal prose, zoomsynchronizedreflowverificationprotocolwithoutescapehatchesoradhoctruncationfallbacks, plus two inline math cases that stress baseline alignment: $\operatorname{PosteriorShift}_{\text{ultralongdiagnosticdescriptorwithoutmanuallinebreaks}}(\theta)$¹⁰ and $\left(\dfrac{\sum_{i=1}^{n}\left(x_i-\bar{x}\right)^2}{\sqrt{\sum_{i=1}^{n}\left(y_i-\bar{y}\right)^2}+\epsilon}\right)^{\!2}$.¹¹

Sidenote Density

This section packs footnote references close together to stress sidenote collision detection. First claim.¹² Second claim.¹³ Third claim.¹⁴ These three should stack neatly in the margin without overlapping.

A paragraph with a reused reference: this claim appears here¹⁵ and again here¹⁵ to verify that each instance gets its own sidenote positioned near its reference.

---

TypeScript Code Block

interface FactualEdit {
  subject: string;
  relation: string;
  oldObject: string;
  newObject: string;
  layer: number;
}

function applyROME(
  model: TransformerModel,
  edit: FactualEdit,
): TransformerModel {
  const key = computeSubjectKey(model, edit.subject, edit.layer);
  const value = optimizeTargetValue(model, edit, key);

  const W = model.layers[edit.layer].mlp.W_proj;
  const residual = subtractMatVec(value, matVec(W, key));
  const delta = outerProduct(residual, key);
  const scale = 1 / dot(key, key);

  model.layers[edit.layer].mlp.W_proj = addMat(W, scaleMat(delta, scale));
  return model;
}

Inline Math and Prose

This starter section tests inline math interleaved with dense prose. An expression like $\mu_n=\frac{\mu_0/\sigma_0^2 + n\bar{x}/\sigma^2}{1/\sigma_0^2 + n/\sigma^2}$ should sit comfortably inside a normal paragraph without breaking the line rhythm.²

A second inline expression with subscripts and operators: $\mathbb{E}\!\left[\Delta\mathcal{L}_{t+1}\mid\mathcal{F}_t\right]$ should also flow naturally.¹

A Bayesian Update in Code

Inline code references like jax.grad(loss_fn) or torch.compile(model) should stay compact.³

from dataclasses import dataclass

@dataclass
class GaussianPosterior:
    mean: float
    variance: float


def posterior(mu0: float, sigma0: float, xbar: float, sigma: float, n: int) -> GaussianPosterior:
    prior_precision = 1.0 / (sigma0 ** 2)
    data_precision = n / (sigma ** 2)
    mean = (prior_precision * mu0 + data_precision * xbar) / (prior_precision + data_precision)
    variance = 1.0 / (prior_precision + data_precision)
    return GaussianPosterior(mean=mean, variance=variance)


print(posterior(0.0, 1.0, 0.18, 0.55, 60))

Display Math

Display equations should center cleanly and not interfere with the sidenote rail:

$$\log p(\theta \mid D) = \log p(\theta) + \sum_{i=1}^{n} \log p(x_i \mid \theta) - \log p(D)$$

A second display equation with more vertical complexity:⁴

$$\hat{W}_{\text{proj}} = W_{\text{proj}} + \frac{(\mathbf{v}_1 - W_{\text{proj}} \, \mathbf{k}_1) \, \mathbf{k}_1^\top}{\mathbf{k}_1^\top \mathbf{k}_1}$$

Tables

Method	Efficacy	Generalization	Specificity
Fine-tuning	92%	41%	63%
ROME	99%	94%	98%
MEMIT	98%	91%	97%
Constrained FT	95%	72%	85%

Blockquotes and Nested Lists

The MLP layers of a transformer are not merely nonlinear feature extractors. They function as distributed associative memories, with each neuron contributing a fragment of the retrieval.

Nested list structure:

• Causal tracing identifies critical mediators
  • Clean run: record all hidden states
  • Corrupted run: perturb subject embeddings
  • Restoration: patch in clean states one at a time
• ROME modifies the MLP weight matrix
  1. Compute context-invariant key
  2. Optimize target value vector⁵
  3. Apply rank-one update to $W_{\text{proj}}$

Dense Stress Pass

Dense technical prose should feel deliberate, not loose. Every line carries signal: references stay compact, annotations remain local, and long tokens degrade gracefully instead of tearing the layout.⁶ This is exactly why sidenotes matter for research writing; if you have to jump to the bottom of the page constantly, you lose the thread and your local stack of assumptions collapses.

To stress wrapping, this paragraph includes emphasized long tokens like counterfactualtraceabilitywithoutsemanticcompressionandwithoutauditboundaries⁷ and hyperlink text such as modelrevisionpipelinewithoutcheckpointfences.⁸

It also embeds a long inline code reference: stream_validator.collector.chain_stage_without_manual_breakpoints_or_aliases("batch:alpha:reconciliation")⁹ to verify that code wraps naturally.

Zoom and Reflow Behavior

Screen-awareness matters: a wrapping strategy that works at one viewport and fails at another is not robust enough. This paragraph carries a long unbroken token in normal prose, zoomsynchronizedreflowverificationprotocolwithoutescapehatchesoradhoctruncationfallbacks, plus two inline math cases that stress baseline alignment: $\operatorname{PosteriorShift}_{\text{ultralongdiagnosticdescriptorwithoutmanuallinebreaks}}(\theta)$¹⁰ and $\left(\dfrac{\sum_{i=1}^{n}\left(x_i-\bar{x}\right)^2}{\sqrt{\sum_{i=1}^{n}\left(y_i-\bar{y}\right)^2}+\epsilon}\right)^{\!2}$.¹¹

Sidenote Density

This section packs footnote references close together to stress sidenote collision detection. First claim.¹² Second claim.¹³ Third claim.¹⁴ These three should stack neatly in the margin without overlapping.

A paragraph with a reused reference: this claim appears here¹⁵ and again here¹⁵ to verify that each instance gets its own sidenote positioned near its reference.

---

TypeScript Code Block

interface FactualEdit {
  subject: string;
  relation: string;
  oldObject: string;
  newObject: string;
  layer: number;
}

function applyROME(
  model: TransformerModel,
  edit: FactualEdit,
): TransformerModel {
  const key = computeSubjectKey(model, edit.subject, edit.layer);
  const value = optimizeTargetValue(model, edit, key);

  const W = model.layers[edit.layer].mlp.W_proj;
  const residual = subtractMatVec(value, matVec(W, key));
  const delta = outerProduct(residual, key);
  const scale = 1 / dot(key, key);

  model.layers[edit.layer].mlp.W_proj = addMat(W, scaleMat(delta, scale));
  return model;
}

Inline Math and Prose

This starter section tests inline math interleaved with dense prose. An expression like $\mu_n=\frac{\mu_0/\sigma_0^2 + n\bar{x}/\sigma^2}{1/\sigma_0^2 + n/\sigma^2}$ should sit comfortably inside a normal paragraph without breaking the line rhythm.²

A second inline expression with subscripts and operators: $\mathbb{E}\!\left[\Delta\mathcal{L}_{t+1}\mid\mathcal{F}_t\right]$ should also flow naturally.¹

A Bayesian Update in Code

Inline code references like jax.grad(loss_fn) or torch.compile(model) should stay compact.³

from dataclasses import dataclass

@dataclass
class GaussianPosterior:
    mean: float
    variance: float


def posterior(mu0: float, sigma0: float, xbar: float, sigma: float, n: int) -> GaussianPosterior:
    prior_precision = 1.0 / (sigma0 ** 2)
    data_precision = n / (sigma ** 2)
    mean = (prior_precision * mu0 + data_precision * xbar) / (prior_precision + data_precision)
    variance = 1.0 / (prior_precision + data_precision)
    return GaussianPosterior(mean=mean, variance=variance)


print(posterior(0.0, 1.0, 0.18, 0.55, 60))

Display Math

Display equations should center cleanly and not interfere with the sidenote rail:

$$\log p(\theta \mid D) = \log p(\theta) + \sum_{i=1}^{n} \log p(x_i \mid \theta) - \log p(D)$$

A second display equation with more vertical complexity:⁴

$$\hat{W}_{\text{proj}} = W_{\text{proj}} + \frac{(\mathbf{v}_1 - W_{\text{proj}} \, \mathbf{k}_1) \, \mathbf{k}_1^\top}{\mathbf{k}_1^\top \mathbf{k}_1}$$

Tables

Method	Efficacy	Generalization	Specificity
Fine-tuning	92%	41%	63%
ROME	99%	94%	98%
MEMIT	98%	91%	97%
Constrained FT	95%	72%	85%

Blockquotes and Nested Lists

The MLP layers of a transformer are not merely nonlinear feature extractors. They function as distributed associative memories, with each neuron contributing a fragment of the retrieval.

Nested list structure:

• Causal tracing identifies critical mediators
  • Clean run: record all hidden states
  • Corrupted run: perturb subject embeddings
  • Restoration: patch in clean states one at a time
• ROME modifies the MLP weight matrix
  1. Compute context-invariant key
  2. Optimize target value vector⁵
  3. Apply rank-one update to $W_{\text{proj}}$

Dense Stress Pass

Dense technical prose should feel deliberate, not loose. Every line carries signal: references stay compact, annotations remain local, and long tokens degrade gracefully instead of tearing the layout.⁶ This is exactly why sidenotes matter for research writing; if you have to jump to the bottom of the page constantly, you lose the thread and your local stack of assumptions collapses.

To stress wrapping, this paragraph includes emphasized long tokens like counterfactualtraceabilitywithoutsemanticcompressionandwithoutauditboundaries⁷ and hyperlink text such as modelrevisionpipelinewithoutcheckpointfences.⁸

It also embeds a long inline code reference: stream_validator.collector.chain_stage_without_manual_breakpoints_or_aliases("batch:alpha:reconciliation")⁹ to verify that code wraps naturally.

Zoom and Reflow Behavior

Screen-awareness matters: a wrapping strategy that works at one viewport and fails at another is not robust enough. This paragraph carries a long unbroken token in normal prose, zoomsynchronizedreflowverificationprotocolwithoutescapehatchesoradhoctruncationfallbacks, plus two inline math cases that stress baseline alignment: $\operatorname{PosteriorShift}_{\text{ultralongdiagnosticdescriptorwithoutmanuallinebreaks}}(\theta)$¹⁰ and $\left(\dfrac{\sum_{i=1}^{n}\left(x_i-\bar{x}\right)^2}{\sqrt{\sum_{i=1}^{n}\left(y_i-\bar{y}\right)^2}+\epsilon}\right)^{\!2}$.¹¹

Sidenote Density

This section packs footnote references close together to stress sidenote collision detection. First claim.¹² Second claim.¹³ Third claim.¹⁴ These three should stack neatly in the margin without overlapping.

A paragraph with a reused reference: this claim appears here¹⁵ and again here¹⁵ to verify that each instance gets its own sidenote positioned near its reference.

---

TypeScript Code Block

interface FactualEdit {
  subject: string;
  relation: string;
  oldObject: string;
  newObject: string;
  layer: number;
}

function applyROME(
  model: TransformerModel,
  edit: FactualEdit,
): TransformerModel {
  const key = computeSubjectKey(model, edit.subject, edit.layer);
  const value = optimizeTargetValue(model, edit, key);

  const W = model.layers[edit.layer].mlp.W_proj;
  const residual = subtractMatVec(value, matVec(W, key));
  const delta = outerProduct(residual, key);
  const scale = 1 / dot(key, key);

  model.layers[edit.layer].mlp.W_proj = addMat(W, scaleMat(delta, scale));
  return model;
}

Wrapping & Scroll Test

Short Display Math (should NOT scroll)

$$E = mc^2$$

Long Display Math (should NOT scroll, just render naturally)

$$\mathcal{L}(\theta) = -\frac{1}{N}\sum_{i=1}^{N}\left[\sum_{k=1}^{K} y_{i,k} \log\!\left(\frac{\exp\!\bigl(W_k^\top h_i + b_k\bigr)}{\sum_{j=1}^{K}\exp\!\bigl(W_j^\top h_i + b_j\bigr)}\right)\right] + \frac{\lambda}{2}\sum_{\ell=1}^{L}\left\|W^{(\ell)}\right\|_F^2 + \beta \sum_{\ell=1}^{L}\operatorname{KL}\!\left(\hat{\rho}\;\big\|\;\rho^{(\ell)}\right)$$

Tall Display Math (vertically large, should NOT get a vertical scrollbar)

$$\mathbf{J} = \begin{pmatrix} \frac{\partial f_1}{\partial x_1} & \frac{\partial f_1}{\partial x_2} & \cdots & \frac{\partial f_1}{\partial x_n} \\[6pt] \frac{\partial f_2}{\partial x_1} & \frac{\partial f_2}{\partial x_2} & \cdots & \frac{\partial f_2}{\partial x_n} \\[6pt] \vdots & \vdots & \ddots & \vdots \\[6pt] \frac{\partial f_m}{\partial x_1} & \frac{\partial f_m}{\partial x_2} & \cdots & \frac{\partial f_m}{\partial x_n} \end{pmatrix}, \quad \nabla_\theta \mathcal{L} = \begin{pmatrix} \frac{\partial \mathcal{L}}{\partial \theta_1} \\[8pt] \frac{\partial \mathcal{L}}{\partial \theta_2} \\[8pt] \vdots \\[8pt] \frac{\partial \mathcal{L}}{\partial \theta_p} \end{pmatrix}, \quad \mathbf{H} = \begin{pmatrix} \frac{\partial^2 \mathcal{L}}{\partial \theta_1^2} & \frac{\partial^2 \mathcal{L}}{\partial \theta_1 \partial \theta_2} & \cdots & \frac{\partial^2 \mathcal{L}}{\partial \theta_1 \partial \theta_p} \\[6pt] \frac{\partial^2 \mathcal{L}}{\partial \theta_2 \partial \theta_1} & \frac{\partial^2 \mathcal{L}}{\partial \theta_2^2} & \cdots & \frac{\partial^2 \mathcal{L}}{\partial \theta_2 \partial \theta_p} \\[6pt] \vdots & \vdots & \ddots & \vdots \\[6pt] \frac{\partial^2 \mathcal{L}}{\partial \theta_p \partial \theta_1} & \frac{\partial^2 \mathcal{L}}{\partial \theta_p \partial \theta_2} & \cdots & \frac{\partial^2 \mathcal{L}}{\partial \theta_p^2} \end{pmatrix}$$

Extremely Tall Display Math (should get a vertical scrollbar at 75vh)

$$f(\mathbf{x}, \theta) = \begin{cases} \displaystyle\sum_{i=1}^{n} \alpha_i \exp\!\left(-\frac{\|x - \mu_i\|^2}{2\sigma_i^2}\right) & \text{if } x \in \mathcal{D}_1 \\[12pt] \displaystyle\prod_{j=1}^{m} \left(1 + \frac{\beta_j}{1 + e^{-\gamma_j^\top x}}\right)^{-1} & \text{if } x \in \mathcal{D}_2 \\[12pt] \displaystyle\int_0^\infty \frac{t^{s-1}}{e^t - 1}\,dt \cdot \Gamma(s)\,\zeta(s) & \text{if } x \in \mathcal{D}_3 \\[12pt] \displaystyle\frac{\partial}{\partial \theta_k}\left[\sum_{\ell=1}^{L}\left\|W^{(\ell)}\right\|_F^2 + \lambda\sum_{i}\operatorname{KL}\!\left(q_i \| p_i\right)\right] & \text{if } x \in \mathcal{D}_4 \\[12pt] \displaystyle\det\begin{pmatrix} a_{11} & a_{12} & \cdots & a_{1n} \\ a_{21} & a_{22} & \cdots & a_{2n} \\ \vdots & \vdots & \ddots & \vdots \\ a_{n1} & a_{n2} & \cdots & a_{nn} \end{pmatrix} & \text{if } x \in \mathcal{D}_5 \\[12pt] \displaystyle\oint_{\partial \Omega} \mathbf{F} \cdot d\mathbf{s} - \iint_{\Omega} \nabla \times \mathbf{F}\,dA & \text{if } x \in \mathcal{D}_6 \\[12pt] \displaystyle\sum_{k=0}^{\infty}\frac{(-1)^k}{(2k+1)!}x^{2k+1} + \sum_{k=0}^{\infty}\frac{(-1)^k}{(2k)!}x^{2k} & \text{if } x \in \mathcal{D}_7 \\[12pt] \displaystyle\mathbb{E}\!\left[\left(\frac{\partial \log p(x|\theta)}{\partial \theta}\right)^{\!2}\right] + \operatorname{Var}\!\left[\frac{\partial \log p(x|\theta)}{\partial \theta}\right] & \text{if } x \in \mathcal{D}_8 \\[12pt] \displaystyle\lim_{n\to\infty}\left(1+\frac{1}{n}\right)^n \cdot \prod_{p\text{ prime}}\frac{1}{1-p^{-s}} & \text{if } x \in \mathcal{D}_9 \\[12pt] \displaystyle\sum_{n=1}^{\infty}\frac{\mu(n)}{n^s} \cdot \left(\sum_{n=1}^{\infty}\frac{1}{n^s}\right)^{-1} & \text{if } x \in \mathcal{D}_{10} \\[12pt] \displaystyle\frac{d}{dt}\begin{pmatrix} x_1 \\ x_2 \\ x_3 \\ x_4 \\ x_5 \\ x_6 \end{pmatrix} = \begin{pmatrix} 0 & 1 & 0 & 0 & 0 & 0 \\ -k_1 & -c_1 & k_1 & c_1 & 0 & 0 \\ 0 & 0 & 0 & 1 & 0 & 0 \\ k_1 & c_1 & -(k_1+k_2) & -(c_1+c_2) & k_2 & c_2 \\ 0 & 0 & 0 & 0 & 0 & 1 \\ 0 & 0 & k_2 & c_2 & -k_2 & -c_2 \end{pmatrix}\begin{pmatrix} x_1 \\ x_2 \\ x_3 \\ x_4 \\ x_5 \\ x_6 \end{pmatrix} & \text{otherwise} \end{cases} \\ f(\mathbf{x}, \theta) = \begin{cases} \displaystyle\sum_{i=1}^{n} \alpha_i \exp\!\left(-\frac{\|x - \mu_i\|^2}{2\sigma_i^2}\right) & \text{if } x \in \mathcal{D}_1 \\[12pt] \displaystyle\prod_{j=1}^{m} \left(1 + \frac{\beta_j}{1 + e^{-\gamma_j^\top x}}\right)^{-1} & \text{if } x \in \mathcal{D}_2 \\[12pt] \displaystyle\int_0^\infty \frac{t^{s-1}}{e^t - 1}\,dt \cdot \Gamma(s)\,\zeta(s) & \text{if } x \in \mathcal{D}_3 \\[12pt] \displaystyle\frac{\partial}{\partial \theta_k}\left[\sum_{\ell=1}^{L}\left\|W^{(\ell)}\right\|_F^2 + \lambda\sum_{i}\operatorname{KL}\!\left(q_i \| p_i\right)\right] & \text{if } x \in \mathcal{D}_4 \\[12pt] \displaystyle\det\begin{pmatrix} a_{11} & a_{12} & \cdots & a_{1n} \\ a_{21} & a_{22} & \cdots & a_{2n} \\ \vdots & \vdots & \ddots & \vdots \\ a_{n1} & a_{n2} & \cdots & a_{nn} \end{pmatrix} & \text{if } x \in \mathcal{D}_5 \\[12pt] \displaystyle\oint_{\partial \Omega} \mathbf{F} \cdot d\mathbf{s} - \iint_{\Omega} \nabla \times \mathbf{F}\,dA & \text{if } x \in \mathcal{D}_6 \\[12pt] \displaystyle\sum_{k=0}^{\infty}\frac{(-1)^k}{(2k+1)!}x^{2k+1} + \sum_{k=0}^{\infty}\frac{(-1)^k}{(2k)!}x^{2k} & \text{if } x \in \mathcal{D}_7 \\[12pt] \displaystyle\mathbb{E}\!\left[\left(\frac{\partial \log p(x|\theta)}{\partial \theta}\right)^{\!2}\right] + \operatorname{Var}\!\left[\frac{\partial \log p(x|\theta)}{\partial \theta}\right] & \text{if } x \in \mathcal{D}_8 \\[12pt] \displaystyle\lim_{n\to\infty}\left(1+\frac{1}{n}\right)^n \cdot \prod_{p\text{ prime}}\frac{1}{1-p^{-s}} & \text{if } x \in \mathcal{D}_9 \\[12pt] \displaystyle\sum_{n=1}^{\infty}\frac{\mu(n)}{n^s} \cdot \left(\sum_{n=1}^{\infty}\frac{1}{n^s}\right)^{-1} & \text{if } x \in \mathcal{D}_{10} \\[12pt] \displaystyle\frac{d}{dt}\begin{pmatrix} x_1 \\ x_2 \\ x_3 \\ x_4 \\ x_5 \\ x_6 \end{pmatrix} = \begin{pmatrix} 0 & 1 & 0 & 0 & 0 & 0 \\ -k_1 & -c_1 & k_1 & c_1 & 0 & 0 \\ 0 & 0 & 0 & 1 & 0 & 0 \\ k_1 & c_1 & -(k_1+k_2) & -(c_1+c_2) & k_2 & c_2 \\ 0 & 0 & 0 & 0 & 0 & 1 \\ 0 & 0 & k_2 & c_2 & -k_2 & -c_2 \end{pmatrix}\begin{pmatrix} x_1 \\ x_2 \\ x_3 \\ x_4 \\ x_5 \\ x_6 \end{pmatrix} & \text{otherwise} \end{cases} \\ f(\mathbf{x}, \theta) = \begin{cases} \displaystyle\sum_{i=1}^{n} \alpha_i \exp\!\left(-\frac{\|x - \mu_i\|^2}{2\sigma_i^2}\right) & \text{if } x \in \mathcal{D}_1 \\[12pt] \displaystyle\prod_{j=1}^{m} \left(1 + \frac{\beta_j}{1 + e^{-\gamma_j^\top x}}\right)^{-1} & \text{if } x \in \mathcal{D}_2 \\[12pt] \displaystyle\int_0^\infty \frac{t^{s-1}}{e^t - 1}\,dt \cdot \Gamma(s)\,\zeta(s) & \text{if } x \in \mathcal{D}_3 \\[12pt] \displaystyle\frac{\partial}{\partial \theta_k}\left[\sum_{\ell=1}^{L}\left\|W^{(\ell)}\right\|_F^2 + \lambda\sum_{i}\operatorname{KL}\!\left(q_i \| p_i\right)\right] & \text{if } x \in \mathcal{D}_4 \\[12pt] \displaystyle\det\begin{pmatrix} a_{11} & a_{12} & \cdots & a_{1n} \\ a_{21} & a_{22} & \cdots & a_{2n} \\ \vdots & \vdots & \ddots & \vdots \\ a_{n1} & a_{n2} & \cdots & a_{nn} \end{pmatrix} & \text{if } x \in \mathcal{D}_5 \\[12pt] \displaystyle\oint_{\partial \Omega} \mathbf{F} \cdot d\mathbf{s} - \iint_{\Omega} \nabla \times \mathbf{F}\,dA & \text{if } x \in \mathcal{D}_6 \\[12pt] \displaystyle\sum_{k=0}^{\infty}\frac{(-1)^k}{(2k+1)!}x^{2k+1} + \sum_{k=0}^{\infty}\frac{(-1)^k}{(2k)!}x^{2k} & \text{if } x \in \mathcal{D}_7 \\[12pt] \displaystyle\mathbb{E}\!\left[\left(\frac{\partial \log p(x|\theta)}{\partial \theta}\right)^{\!2}\right] + \operatorname{Var}\!\left[\frac{\partial \log p(x|\theta)}{\partial \theta}\right] & \text{if } x \in \mathcal{D}_8 \\[12pt] \displaystyle\lim_{n\to\infty}\left(1+\frac{1}{n}\right)^n \cdot \prod_{p\text{ prime}}\frac{1}{1-p^{-s}} & \text{if } x \in \mathcal{D}_9 \\[12pt] \displaystyle\sum_{n=1}^{\infty}\frac{\mu(n)}{n^s} \cdot \left(\sum_{n=1}^{\infty}\frac{1}{n^s}\right)^{-1} & \text{if } x \in \mathcal{D}_{10} \\[12pt] \displaystyle\frac{d}{dt}\begin{pmatrix} x_1 \\ x_2 \\ x_3 \\ x_4 \\ x_5 \\ x_6 \end{pmatrix} = \begin{pmatrix} 0 & 1 & 0 & 0 & 0 & 0 \\ -k_1 & -c_1 & k_1 & c_1 & 0 & 0 \\ 0 & 0 & 0 & 1 & 0 & 0 \\ k_1 & c_1 & -(k_1+k_2) & -(c_1+c_2) & k_2 & c_2 \\ 0 & 0 & 0 & 0 & 0 & 1 \\ 0 & 0 & k_2 & c_2 & -k_2 & -c_2 \end{pmatrix}\begin{pmatrix} x_1 \\ x_2 \\ x_3 \\ x_4 \\ x_5 \\ x_6 \end{pmatrix} & \text{otherwise} \end{cases}$$

Short Code Block (should wrap, no scrollbar)

# print(f"LR at step 50000: {cosine_schedule(50000, train_config.warmup_steps, train_config.learning_rate):.6f}")
def fibonacci(n: int) -> int:
    if n <= 1:
        return n
    a, b = 0, 1
    for _ in range(2, n + 1):
        a, b = b, a + b
    return b

print([fibonacci(i) for i in range(20)])
# print(f"LR at step 50000: {cosine_schedule(50000, train_config.warmup_steps, train_config.learning_rate):.6f}")

Long Code Block (30+ lines, should scroll)

import numpy as np
from dataclasses import dataclass, field
from typing import Optional

@dataclass
class TransformerConfig:
    d_model: int = 512
    n_heads: int = 8
    n_layers: int = 6
    d_ff: int = 2048
    vocab_size: int = 32000
    max_seq_len: int = 2048
    dropout: float = 0.1
    layer_norm_eps: float = 1e-5

@dataclass
class TrainingConfig:
    batch_size: int = 64
    learning_rate: float = 3e-4
    warmup_steps: int = 4000
    max_steps: int = 100000
    weight_decay: float = 0.01
    grad_clip: float = 1.0
    eval_interval: int = 500
    save_interval: int = 5000
    log_interval: int = 10

def scaled_dot_product_attention(Q, K, V, mask=None):
    d_k = Q.shape[-1]
    scores = np.matmul(Q, K.transpose(-2, -1)) / np.sqrt(d_k)
    if mask is not None:
        scores = np.where(mask == 0, -1e9, scores)
    weights = softmax(scores, axis=-1)
    return np.matmul(weights, V), weights

def softmax(x, axis=-1):
    e_x = np.exp(x - np.max(x, axis=axis, keepdims=True))
    return e_x / np.sum(e_x, axis=axis, keepdims=True)

def layer_norm(x, gamma, beta, eps=1e-5):
    mean = np.mean(x, axis=-1, keepdims=True)
    var = np.var(x, axis=-1, keepdims=True)
    return gamma * (x - mean) / np.sqrt(var + eps) + beta

def feed_forward(x, W1, b1, W2, b2):
    hidden = np.maximum(0, np.matmul(x, W1) + b1)
    return np.matmul(hidden, W2) + b2

def positional_encoding(seq_len, d_model):
    pos = np.arange(seq_len)[:, np.newaxis]
    dim = np.arange(d_model)[np.newaxis, :]
    angles = pos / np.power(10000, (2 * (dim // 2)) / d_model)
    encoding = np.zeros((seq_len, d_model))
    encoding[:, 0::2] = np.sin(angles[:, 0::2])
    encoding[:, 1::2] = np.cos(angles[:, 1::2])
    return encoding

def cosine_schedule(step, warmup_steps, max_lr, min_lr=1e-6):
    if step < warmup_steps:
        return max_lr * step / warmup_steps
    progress = (step - warmup_steps) / (100000 - warmup_steps)
    return min_lr + 0.5 * (max_lr - min_lr) * (1 + np.cos(np.pi * progress))

config = TransformerConfig()
train_config = TrainingConfig()
pe = positional_encoding(config.max_seq_len, config.d_model)
print(f"Config: {config}")
print(f"Positional encoding shape: {pe.shape}")
print(f"LR at step 0: {cosine_schedule(0, train_config.warmup_steps, train_config.learning_rate):.6f}")
print(f"LR at step 4000: {cosine_schedule(4000, train_config.warmup_steps, train_config.learning_rate):.6f}")
print(f"LR at step 50000: {cosine_schedule(50000, train_config.warmup_steps, train_config.learning_rate):.6f}")

---

Open Questions

How distributed is factual storage really?

Causal tracing reveals dominant sites of mediation, but a long tail of smaller contributions from other components may exist.¹⁶

What is the relationship between factual recall and in-context learning?

Models can acquire new facts from their context window. Whether in-context and parametric facts share retrieval mechanisms remains open.¹⁷

Copyable Units

An inline copyable — the button rides beside the text and copies it: reach me at kaushikreddyxyz@gmail.com anytime. And the fully detached form — display text is ordinary markdown, the button carries its own payload: displayed 1@email.com but copies 2@email.com . A second one with a custom payload (displays one thing, copies another): the install command.

The block form, for standalone copy targets:

Images and Figures

A full-width markdown image with no caption, exactly as ![...](...) renders it:

The same mechanics with an explicit figure and caption, the recommended form for anything that deserves a label:

Odd Sizes

A square figure narrower than the measure should center itself rather than hug the left edge:

A tall, portrait-orientation figure — the awkward case for vertical rhythm:

And a small inline-scale image dropped mid-paragraph flow, uncaptioned:

Prose continues immediately after the small image to check the spacing above and below bare images against paragraph rhythm.

Video

A locally hosted clip served from the site’s own files, with controls and a poster-free default frame:

An external YouTube embed, responsive at 16:9 regardless of viewport width:

Footnotes

1. Conditional expectations with calligraphic filtration symbols are a good test of KaTeX’s font rendering across different baseline alignments. ↩ ↩² ↩³ ↩⁴ ↩⁵ ↩⁶ ↩⁷

2. Write inline equations with $...$ and display equations with $$...$$. KaTeX renders at build time, so there is no client-side JavaScript cost for math. ↩ ↩² ↩³ ↩⁴ ↩⁵ ↩⁶ ↩⁷

3. Fenced code blocks get syntax highlighting from Shiki with the solarized-light theme. Inline code is styled separately for high-contrast readability. ↩ ↩² ↩³ ↩⁴ ↩⁵ ↩⁶ ↩⁷

4. The Sherman-Morrison formula connection becomes clearer when you consider the constraint that $\hat{W}$ should satisfy $\hat{W}\mathbf{k}_1 = \mathbf{v}_1$ while minimizing $\|\hat{W} - W\|_F$. The solution is exactly a rank-one perturbation. ↩ ↩² ↩³ ↩⁴ ↩⁵ ↩⁶ ↩⁷

5. The target value vector $\mathbf{v}_1$ is optimized via gradient descent so that the model produces the desired new fact. A regularization term $\lambda\|\mathbf{v}\|^2$ prevents drift from the manifold of typical value vectors. ↩ ↩² ↩³ ↩⁴ ↩⁵ ↩⁶ ↩⁷

6. hyperparametercalibrationwithoutregularizationbarriersandwithoutdistributionchecks stays bold and should wrap cleanly because it is plain emphasized prose. ↩ ↩² ↩³ ↩⁴ ↩⁵ ↩⁶ ↩⁷

7. counterfactualtraceabilitywithoutsemanticcompressionandwithoutauditboundaries should follow the same long-word wrapping path as plain text. ↩ ↩² ↩³ ↩⁴ ↩⁵ ↩⁶ ↩⁷

8. modelrevisionpipelinewithoutcheckpointfences checks that hyperlink text keeps link styling while wrapping. ↩ ↩² ↩³ ↩⁴ ↩⁵ ↩⁶ ↩⁷

9. Inline code should wrap without artifacts: stream_validator.collector.chain_stage_without_manual_breakpoints_or_aliases("batch:alpha:reconciliation"). ↩ ↩² ↩³ ↩⁴ ↩⁵ ↩⁶ ↩⁷

10. Inline math with a long subscript descriptor: $\operatorname{PosteriorShift}_{\text{ultralongdiagnosticdescriptorwithoutmanuallinebreaks}}(\theta)$. ↩ ↩² ↩³ ↩⁴ ↩⁵ ↩⁶ ↩⁷

11. Tall inline math: $\left(\dfrac{\sum_{i=1}^{n}\left(x_i-\bar{x}\right)^2}{\sqrt{\sum_{i=1}^{n}\left(y_i-\bar{y}\right)^2}+\epsilon}\right)^{\!2}$. This exercises vertical rhythm when math is taller than surrounding text. ↩ ↩² ↩³ ↩⁴ ↩⁵ ↩⁶ ↩⁷

12. First densely-packed sidenote. Short and to the point. ↩ ↩² ↩³ ↩⁴ ↩⁵ ↩⁶ ↩⁷

13. Second sidenote immediately below. Tests collision detection between closely-spaced margin notes. ↩ ↩² ↩³ ↩⁴ ↩⁵ ↩⁶ ↩⁷

14. Third sidenote in the cluster. If all three render without overlap, the collision avoidance logic is working correctly. ↩ ↩² ↩³ ↩⁴ ↩⁵ ↩⁶ ↩⁷

15. This footnote is referenced twice in the text. Each reference should produce its own sidenote positioned near the reference point, not a single shared sidenote. ↩ ↩² ↩³ ↩⁴ ↩⁵ ↩⁶ ↩⁷ ↩⁸ ↩⁹ ↩¹⁰ ↩¹¹ ↩¹² ↩¹³ ↩¹⁴

16. The distinction between mediation and storage remains open. A component can mediate a prediction without being the sole site of storage. ↩

17. Recent work suggests parametric and in-context knowledge may use partially overlapping circuits, but the mechanistic picture is still incomplete. ↩