typed decisions · one encoder pass

Ask ten questions.
Read the text once.

cbjev answers choice, score and yes/no questions about a ticket, an e-mail or a JSON document with calibrated probabilities — no generation, nothing to parse. It is a self-hosted successor to Laya that speaks TypeSafe Jev's wire format.

source on GitHub weights on Hugging Face all benchmark tables

Laya · one sequence per question · 5,500 tokens75.8 ms

cbjev · one row per call · 1,000 tokens11.4 ms

Ten questions over a 500-token document. One square is 50 tokens: solid squares are question text, faded squares the document. Median latency on an RTX 4090.

01The layout

Every question gets its own segment. The document is read once.

Laya places each question in front of a full copy of the document, so ten questions mean ten passes over the same text. cbjev packs a call into one row and shapes the attention mask instead: questions never see each other, while the document reads all of them.

Each segment restarts its positions right after [CLS]. With a single question the row is token for token, position for position, what a Laya checkpoint was trained on — which is why cbjev could be fine-tuned from Laya's weights instead of learning the task from scratch.

Attention mask for three questions: each row reads the columns that are filled. Solid is a question's own segment, faded the shared document, empty is masked out.

Around the layout. A from-scratch ModernBERT / mmBERT forward: bf16 matmuls over an fp32 residual stream, each layer fused with torch.compile, and the whole forward replayed as one CUDA graph per 32-token shape bucket. The document is tokenized once per call; question text is tokenized once and cached.

Option-order voting. Every choice and score question is also asked with its options reversed, and the two answers are averaged — one more short segment, not another pass. Answers change on reordered options 0.2 % of the time, against 7.8 % for Laya.

02Speed

Faster than both Laya paths in all ten cases measured.

End-to-end predict() latency, median of 60 calls, engines interleaved call by call on one RTX 4090. “Laya fast” is Laya's TileLang kernel path. The gap grows with the length of the document and the number of questions.

cbjev Laya fast Laya

03Accuracy

Ahead on 11 of 15 suites — and here are the other four.

Accuracy on 400 cases per suite, cbjev against the better of Laya's two English checkpoints on each suite, with byte-identical inputs. Held-out suites were never trained on. Sorted by the difference.

cbjev Laya, best checkpoint difference in points

0.741vs 0.710

mean accuracy over the 15 English suites

0.117vs 0.125

mean calibration error (ECE); Jev's published figure is 0.246

45 / 51languages

MASSIVE intent where cbjev-multilingual matches or beats Laya (0.436 vs 0.401 macro)

04Use it

Install, load, ask.

Python 3.10+ and PyTorch 2.4+. Weights download from Hugging Face on first use (about 800 MB for the English checkpoint). A Router sends other languages to the multilingual checkpoint. The answers below are this exact call's real output.

pythonpredict.py

import cbjev

agent = cbjev.load()  # GPU if available

state = {
    "subject": "Duplicate charge on invoice #4411",
    "body": "Billed twice for March. "
            "Refund it today or we cancel.",
}
questions = {
    "team": {
        "type": "choice",
        "instructions": "Which team should handle this?",
        "criteria": {"billing": "invoices, refunds",
                     "technical": "bugs, outages",
                     "other": "everything else"},
    },
    "urgency": {
        "type": "score",
        "instructions": "How urgent is it?",
        "criteria": ["can wait", "this week",
                     "today", "blocking"],
    },
    "churn": {
        "type": "noul",
        "instructions": "Does the customer threaten "
                        "to cancel their subscription?",
    },
}
res = agent.predict(state, questions)
print(res["answers"])

jsonres["answers"]

{
  "team": {
    "type": "choice",
    "choice": "billing",
    "probabilities": {"billing": 0.9916,
                      "technical": 0.0034,
                      "other": 0.005},
    "confidence": 0.9508
  },
  "urgency": {
    "type": "score",
    "score": 2.0983,
    "probabilities": {"0": 0.0171, "1": 0.091,
                      "2": 0.6685, "3": 0.2234},
    "confidence": 0.3568
  },
  "churn": {
    "type": "noul",
    "noul": 0.6728,
    "confidence": 0.6728
  }
}

shellinstall, cli, server

pip install "cbjev[serve] @ git+https://github.com/tomek7667/cbjev"

cbjev "I was charged twice, please refund" --predict   # one-off answers as JSON
CBJEV_DEVICE=cuda CBJEV_PRELOAD=1 cbjev-serve          # POST /v1/systemone on 127.0.0.1:8000

curl -s localhost:8000/v1/systemone -H 'content-type: application/json' -d '{
  "state": {"body": "billed twice, refund please or we cancel"},
  "questions": {"team": {"type": "choice", "instructions": "Which team?",
                         "criteria": {"billing": "refunds", "tech": "bugs"}}}}'

05Limits

Where it still loses.

Seven training rounds and several weight averages moved these by a point or two at most, so treat them as real.

−4.0Support triage — 10-way queue routing, 0.470 against 0.510.
−3.5Prompt injection — 0.698 against 0.733; most misses are German prompts. Route non-English text to the multilingual checkpoint.
−2.5DAIR emotion — 0.573 against 0.598, held out from training.
−0.8AG News — 0.945 against 0.953: three cases out of 400.
−2577 labels in one question — Banking77 at 0.620 against Jev's published 0.870 (Laya: 0.497). Keep choice questions under about 30 options.

Ask ten questions.Read the text once.