Text

Use Text for string fields encoded by a frozen Hugging Face model. The text encoder produces a dense representation that json2vec projects into the model dimension.

{
  "body": "Customer reported a delayed international transfer."
}

import json2vec as jv

body = jv.Text(
    "body",
    max_length=128,
    encoder_pooling="mean",
)

body

body [text] active
 pooling=query weight=1 p_mask=0 p_prune=0 n_heads=4 n_linear=1
 model=google/bert_uncased_L-2_H-128_A-2 max_length=128 encoder_batch_size=32 encoder_pooling=mean objective=l2

Text is semantic feature encoding, not text generation. Use Category for bounded labels such as merchant names or product codes when exact identity is the desired signal.

Dependency

The text tensorfield requires the optional transformers dependency.

uv sync --extra text

Without that extra, using type: text raises an import error when json2vec tries to load the tokenizer or encoder.

Hugging Face may download model assets the first time a model is used. Use a local model path when documentation, training, or serving must run from cached files only.

Input Values

Text expects string values. None is encoded as a null state, and missing branch positions are encoded as padded state.

Text is tokenized with the configured Hugging Face tokenizer using max-length padding and truncation. Token IDs and attention masks are stored as tensorfield content.

Examples

Common text fields include:

• Titles, descriptions, reviews, tickets, notes, or messages.
• Merchant names, product names, search queries, or support subjects.
• Short explanations or free-form metadata attached to structured events.
• Text that should be encoded as a semantic feature rather than generated as output.

Text can be nested like any other leaf:

ticket_events = jv.Branch(
    name="ticket_events",
    length=32,
    message=jv.Text(max_length=128),
    event_type=jv.Category(size=64),
)

ticket_events

ticket_events [branch] length=32 overflow=head attention=mha n_layers=1 n_heads=4 n_linear=1
|-- message [text] active
|    pooling=query weight=1 p_mask=0 p_prune=0 n_heads=4 n_linear=1
|    model=google/bert_uncased_L-2_H-128_A-2 max_length=128 encoder_batch_size=32 encoder_pooling=cls objective=l2
`-- event_type [category] active
     pooling=query weight=1 p_mask=0 p_prune=0 n_heads=4 n_linear=1
     size=64 p_unavailable=0.01 topk=[]

Tiny BERT Local Smoke Test

For a quick local check, keep the json2vec model shallow and use the default small BERT miniature. The first run downloads the Hugging Face model into the local cache; later Text fields with the same model reuse the same frozen encoder module inside the process.

uv sync --extra text

import json2vec as jv

model = jv.Model(
    d_model=32,
    n_layers=1,
    n_heads=4,
    batch_size=2,
    embed=True,
    message=jv.Text(
        max_length=32,
        encoder_batch_size=4,
        encoder_pooling="mean",
    ),
)

records = [
    {"message": "Refund was approved after manual review."},
    {"message": "Customer asked for shipment tracking details."},
]

predictions = model.predict(records)
record_embedding = predictions[jv.Address("record")]["embedding"]

print(record_embedding.shape)

This is a wiring smoke test, not a trained classifier. It exercises tokenizer loading, the frozen BERT encoder, the Text projection, and root embedding output.

Configuration

Option	Default	Notes
model	"google/bert_uncased_L-2_H-128_A-2"	Hugging Face model or local path. Must be a non-empty string.
max_length	128	Tokenizer max length. Must be positive.
encoder_batch_size	32	Number of flattened text values encoded per Hugging Face forward pass.
encoder_pooling	"cls"	Pooling mode: "cls", "mean", or "pooler".
objective	"l2"	Hidden-embedding reconstruction objective: "l1" or "l2".

encoder_pooling="pooler" requires the selected Hugging Face model to expose pooler_output. The "cls" and "mean" modes require last_hidden_state.

Target Behavior

When Text is masked or used as a target, the decoder predicts:

• state: probabilities for valued, null, padded, and masked.
• content: the frozen encoder hidden embedding for the original text.

The decoder does not generate text tokens. It reconstructs the frozen text embedding used as the target representation.

Prediction Output

Text currently trains and reports losses and metrics, but it does not emit generated text in Model.predict(...). Configure it as an input feature or embedding-reconstruction target, not as a generated-text output. Configure embed=True when you want the field address to emit an embedding payload.

Notes

The Hugging Face encoder is cached, run in evaluation mode, and not fine-tuned by json2vec. Use encoder_batch_size to manage memory when many text values are flattened from nested branches.