Dynamic Masking

Dynamic masking hides values during training so the model learns to reconstruct them from surrounding context. Leaf fields already support local stochastic masking with p_mask, pruning with p_prune, and supervised targets with target=True. Branch masks add one more control: they select slices of a repeated context and apply that selection to descendant tensorfields.

Use branch masks when the hiding rule is about a repeated coordinate rather than one leaf value. Common examples are “mask half of the last 10 retained events,” “mask three early line items,” or “mask the amount and merchant for selected transactions while leaving country visible.”

Leaf Masking Versus Branch Masking

Use the narrowest mechanism that matches the modeling intent.

Mechanism	Configure on	What gets hidden
target=True	leaf tensorfield	Every instance of that leaf, for supervised prediction.
p_prune=...	leaf tensorfield	Whole leaf instances sampled independently.
p_mask=...	leaf tensorfield	Observed leaf values sampled independently.
jv.Mask(...)	Branch	Repeated branch coordinates, projected onto descendant leaves.

Leaf masking is enough when every value can be sampled independently. Branch masking is better when one repeated row should be hidden consistently across several fields, or when the sampling window is defined by recency inside a branch.

Branch Masks

Attach jv.Mask(...) to a Branch(...) with mask= or masks=:

import json2vec as jv

model = jv.Model(
    name="customer",
    d_model=64,
    n_layers=2,
    n_heads=4,
    events=jv.Branch(
        length=64,
        overflow="tail",
        mask=jv.Mask(rate=0.50, window=10),
        event_type=jv.Category(size=128),
        amount=jv.Number,
    ),
)

model

This selects the last 10 retained, non-padded events slots for each observation and masks about half of those slots. The same selected event rows are projected onto each active descendant leaf unless the mask excludes that leaf.

mask= is shorthand for a one-item list. Use masks= when one branch needs more than one masking policy:

jv.Branch(
    name="line_items",
    length=128,
    masks=[
        jv.Mask(name="recent", rate=0.25, window=32),
        jv.Mask(name="early", count=3, window=20, start=True),
    ],
    sku=jv.Category(size=4096),
    quantity=jv.Number,
)

Mask Options

Each Mask chooses a candidate window, then samples from that window.

Option	Default	Meaning
rate	None	Bernoulli probability for each candidate slot. Mutually exclusive with count.
count	None	Exact number of candidate slots to sample per parent coordinate, capped by available candidates. Mutually exclusive with rate.
window	None	Candidate span length. None means all slots in the selected coordinate space.
branch	False	Use fixed padded branch coordinates instead of observed non-padded coordinates.
start	False	Take the window from the start instead of the end.
offset	0	Skip slots away from the selected edge before taking the window.
exclude	None	One predicate, or a list of predicates, for descendant leaves that should not receive this mask.

Exactly one of rate or count is required.

The default coordinate mode is branch=False, start=False, which means “the end of the retained real values.” This is usually the right choice for recency histories, especially with overflow="tail".

Window Coordinates

Mask windows are resolved after preprocessing, query extraction, overflow handling, and padding. Padding is never selected.

For a branch with length=8, window=3, and retained values shaped like this:

slot:   0 1 2 3 4 5 6 7
state:  V V V V V P P P

The candidate windows are:

Configuration	Candidate slots
jv.Mask(rate=0.2, window=3, start=True)	0, 1, 2
jv.Mask(rate=0.2, window=3)	2, 3, 4
jv.Mask(rate=0.2, window=3, branch=True, start=True)	0, 1, 2
jv.Mask(rate=0.2, window=3, branch=True)	none, because fixed slots 5, 6, 7 are padded

offset skips away from the chosen edge. With the same values and offset=2:

Configuration	Candidate slots
jv.Mask(rate=0.2, window=3, start=True, offset=2)	2, 3, 4
jv.Mask(rate=0.2, window=3, offset=2)	0, 1, 2

Nested Branches

A branch mask is resolved at the branch that owns it. If that branch is nested, the mask is computed independently for each parent coordinate.

This example masks two letters from the last two retained letters in each word:

from __future__ import annotations

import random
import string

from rich import print

import json2vec as jv

ALPHABET = string.ascii_uppercase
ADDRESS = "record/words/letters/letter"


def consecutive_letters(rng: random.Random) -> list[dict[str, str]]:
    length = rng.randint(3, 8)
    start = rng.randint(0, len(ALPHABET) - length)
    return [{"letter": value} for value in ALPHABET[start : start + length]]


def words(rng: random.Random) -> list[dict[str, list[dict[str, str]]]]:
    return [{"letters": consecutive_letters(rng)} for _ in range(rng.randint(2, 4))]


rng = random.Random(7)
data = [{"words": words(rng)} for _ in range(5)]

model = jv.Model(
    d_model=16,
    n_layers=1,
    n_heads=4,
    words=jv.Branch(
        length=3,
        letters=jv.Branch(
            length=8,
            mask=jv.Mask(count=2, window=2),
            letter=jv.Category(size=len(ALPHABET), p_unavailable=0.0),
        ),
    ),
)

inputs = model.encode(data, strata=jv.Strata.train, mask=False)
masked = model.encode(data, strata=jv.Strata.train, mask=True)

print(inputs[ADDRESS])
print(masked[ADDRESS])

The tensorfield display shows the first batch item and the generated root singleton dimension. Nested branches are printed with one row per inner branch:

TensorField [tensorfield] state=(5, 1, 3, 8) device=cpu trainable=0
 counts V=54 N=0 P=66 M=0 O=0
 state V V V V P P P P
       V V V V V V V V
       V V V P P P P P

After masking, selected values become M and cached targets are present:

TensorField [tensorfield] state=(5, 1, 3, 8) device=cpu trainable=22
 counts V=32 N=0 P=66 M=22 O=0
 state V V M M P P P P
       V V V V V V M M
       V M M P P P P P
 targets=content, state

Randomness is intentionally stochastic, so exact masked positions can differ between runs.

Debugging Encodes

Model.encode(...) applies masking by default. Pass mask=False when you want to inspect the raw tensorized state before any dynamic masks, p_mask, or target=True pruning are applied:

unmasked = model.encode(records, strata=jv.Strata.train, mask=False)
masked = model.encode(records, strata=jv.Strata.train, mask=True)

This is most useful when validating mask windows for ragged branches. In the rendered tensorfield state:

Symbol	Token
V	observed valued input
N	explicit null
P	padded slot
M	masked hidden input
O	other reserved state

The first tensor dimension is batch. The second dimension is the generated root singleton. The display previews state[0, 0].

Excluding Leaves

Use exclude= when a row should be selected for some descendant fields but not others:

jv.Branch(
    name="transactions",
    length=128,
    mask=jv.Mask(
        rate=0.25,
        window=32,
        exclude=jv.where("name") == "country",
    ),
    merchant=jv.Category(size=4096),
    amount=jv.Number,
    country=jv.Category(size=256),
)

exclude accepts one predicate or a list of predicates. Predicates are resolved after the schema is bound, relative to the active descendant leaves of the branch that owns the mask:

jv.Mask(
    rate=0.25,
    window=32,
    exclude=[
        jv.where("type") == "entity",
        jv.where("name") == "country",
    ],
)

Prediction Placeholders

During prediction, every tensorfield accepts the literal string "<MASK>" as a placeholder for “decode this value”:

model.predict(
    [
        {
            "events": [
                {"event_type": "login", "amount": 0.0},
                {"event_type": "<MASK>", "amount": "<MASK>"},
            ]
        }
    ]
)

"<MASK>" is predict-only. It raises in train, validate, and test strata, and vocabulary-backed fields never learn it as an observed token.

For structured leaves such as Set or Vector, a scalar "<MASK>" masks the whole field. A collection containing "<MASK>" as one element is invalid.

Validation

Mask configuration is validated when the schema is bound:

• Mask requires exactly one of rate or count.
• rate must be between 0 and 1.
• count must be non-negative.
• window must be positive when provided.
• offset must be non-negative and smaller than the owning branch’s length.
• Passing both mask= and masks= to one branch is invalid.
• Masks on the generated root branch are not supported.
• A masked branch must have at least one active descendant leaf.
• exclude cannot remove every active descendant leaf.
• Duplicate non-null mask names on the same branch are invalid.

Performance Notes

Mask resolution happens once per active leaf tensorfield. rate masks are fully vectorized and usually add little overhead. count masks sample an exact number of slots per parent coordinate and are more expensive, especially for large nested batches or many descendant leaves.

For very large schemas, prefer rate when exact counts are not required. If multiple leaves share the same branch mask, expect cost to scale with the number of affected leaves.

Where Next

• Use Learning Modes & Embeddings for target=True, p_mask, p_prune, and embedding export patterns.
• Use Branch for repeated-context configuration, length, and overflow.
• Use Data Modules to see where preprocessing, queries, overflow, and masking run in the batch path.