Introducing Wazn-2B: weighing hundreds of runtime labels without generating a token

Today I’m releasing Wazn-2B v0.1, an experimental decision model built on Qwen3.5-2B-Base. Wazn takes a context, an instruction, and a runtime-defined set of candidate labels—optionally with definitions and few-shot examples—and returns probabilities over those candidates. It does not generate text.

The current checkpoint was trained on roughly 5,000 diverse examples spanning retrieval, classification, and decision tasks. The training set intentionally does not include well-known public classification benchmarks such as BANKING77 or CLINC150.

Wazn uses a different architecture from approaches that derive class decisions from next-token logits, or that serialize every candidate and its examples into a single causal prompt. Instead, the shared context and instruction are computed once, while each candidate is encoded through its own branch. These branches produce candidate representations that are then compared by a small relational judge. A separate NONE gate estimates whether any supplied candidate is valid at all.

This separation has several useful properties. Candidate ordering is not part of the decision signal: permuting the candidates simply permutes their scores. Candidate-specific examples do not have to share one increasingly large causal sequence. And once candidate representations have been computed, they can be reused by the decision head without running the backbone again.

This last property makes large-$K$ classification especially interesting. Rather than placing hundreds of classes into one prompt, Wazn can divide them into small groups, retain the top candidates from each group, and repeatedly compare the survivors in a tournament-like procedure. Later rounds operate entirely on already-computed candidate representations, so they require no candidate re-prefill.

Even though the current inference implementation is not yet heavily optimized, Wazn can already handle requests with more than 200 candidate classes in under 200 ms on an RTX 4090, and in around 1 s on a MacBook M3. The rest of this post explains the architecture, why I chose a relational decision head, and what this design buys compared with conventional LLM classification.

Separating representation from competition

A straightforward way to turn a decoder LLM into a classifier is to place every candidate in the prompt and either generate the answer or score candidate tokens directly. This works well when the candidate set is small, but it couples several things that do not necessarily need to be coupled: understanding the input, representing each candidate, and making candidates compete with one another.

In a causal decoder, candidates placed in the same sequence are also not represented symmetrically. An early candidate cannot attend to candidates appearing after it, while later candidates can attend to everything before them. Reordering the choices therefore changes the computation performed by the backbone, even though the underlying classification problem has not changed.

Wazn instead treats candidate representation and candidate competition as two separate stages. The expensive language model is responsible for understanding the state, instruction, candidate definition, and any candidate-specific examples. It produces one representation $p_i$ for each candidate $a_i$, given the input $x$:

Candidate competition happens only afterwards, in a much smaller relational network. Each candidate is evaluated relative to the alternatives rather than assigned an independent absolute class logit.

This separation gives Wazn several useful properties. Candidate order does not carry semantic information, candidate-specific examples remain local to their candidate, and candidate representations can be reused after the backbone has finished running. This becomes particularly useful when the candidate set is large: once the representations exist, Wazn can repeatedly regroup and compare candidates without re-running the language model.

It also allows Wazn to separate two questions that are often conflated by a softmax classifier: which candidate is best? and is any candidate valid at all? The former is handled by the relational judge; the latter by a separate validity gate.

Architecture

From hidden states to candidate representations

Wazn does not use the language-model vocabulary head. For each candidate $a_i$, the backbone processes the shared context and instruction followed by that candidate’s branch. We take the final hidden state at the candidate’s closing </choice> token:

The </choice> position acts as a readout point. Because the backbone is causal, its hidden state has already incorporated the shared context, the instruction, the candidate definition, and any candidate-specific examples that precede it.

Importantly,

There is no fixed output neuron corresponding to "billing", "refund", "network failure", or any other class. Wazn can therefore receive a new set of labels at runtime.

The candidate hidden state is then mapped into the judge’s representation space:

For Wazn-2B v0.1, $d_j = 768$ (see the released config). The same projection is applied to every candidate.

At this point, if there are $K$ candidates, inference has produced the set

Still, none of them has an absolute class logit.

Scoring by comparison

For every ordered pair of candidates $(i, j)$, Wazn constructs a relational representation:

These three terms expose different relationships to the judge: $p_i - p_j$ captures directional differences, $p_i \odot p_j$ captures dimension-wise interaction, and $\cos(p_i, p_j)$ provides an explicit measure of semantic alignment. The released v0.1 configuration uses exactly these diff, prod, and cos relational features.

A small shared network $\psi$ then evaluates the ordered pair. Instead of directly using its output, Wazn antisymmetrizes it:

This simple operation gives an important property by construction:

If comparing candidate $i$ against candidate $j$ gives $i$ an advantage of $+0.7$, the exact same comparison contributes $-0.7$ to $j$. There is no need for the model to separately learn both facts.

A continuous round-robin tournament

Each candidate then plays against every other candidate. Its final score is its mean relational advantage:

I like describing this as a continuous round-robin tournament. For two candidates, $s_1 = \phi_{12}$ and $s_2 = -\phi_{12}$. For five candidates, each candidate receives four pairwise margins and its score is their average.

The mean rather than the sum is useful because the natural scale of the score does not automatically increase just because more candidates were supplied. The released configuration uses mean relational aggregation.

The conditional probability over candidates is finally:

No text is generated. The language-model head never participates in the decision; the released model is explicitly a probability-over-runtime-labels model rather than a generator.

Scores are zero-sum

The antisymmetric construction implies

To see why, consider the numerator $\sum_i \sum_{j \neq i} \phi_{ij}$. Every term $\phi_{ij}$ appears exactly once together with $\phi_{ji} = -\phi_{ij}$, so every pair cancels.

The judge therefore cannot increase the scores of every candidate simultaneously. Candidate scores describe relative advantage, not an arbitrary collection of independent logits.

I think this is one of the nicest aspects of Wazn:

That is also where the name Wazn—“weight” or “weighing”—fits the architecture quite naturally.

Candidate order is not part of the model

There is another useful consequence. Let $\pi$ be any permutation of the $K$ candidates. If we reorder $(p_1, \ldots, p_K)$ to $(p_{\pi(1)}, \ldots, p_{\pi(K)})$, the exact same pairwise computation takes place, merely with the rows and columns renamed. Therefore:

The scorer is permutation equivariant: reordering the candidates simply reorders their scores. The semantic prediction is unchanged.

This property does not have to be learned from candidate shuffling. It follows from the architecture.

The classes do not live in the weights

A conventional classifier typically ends with something like

where row $W_i$ permanently represents class $i$. Wazn has no such matrix. Instead, the meaning of a class comes from its text, while the judge learns how to compare meanings.

The same relational parameters can therefore operate on $K = 3$, $K = 17$, or $K = 237$ without changing the architecture or introducing new class parameters.

This is also what makes tournament inference possible. Once $p_1, \ldots, p_K$ have been produced by the expensive backbone, they can be regrouped and compared repeatedly by the small judge,

without computing the candidate branches again.

There is, however, a deliberate limitation to this formulation. A relative tournament always produces a winner. Softmax cannot tell us that none of the candidates should win. Wazn therefore treats candidate selection and candidate validity as two separate inference problems.

The NONE gate: a winner is not necessarily an answer

The relational judge has one intentional limitation: it always produces a ranking. Given any candidate set $\{a_1, \ldots, a_K\}$, some candidate will obtain the highest relational score, even when none of the candidates is actually correct.

Wazn therefore separates which candidate is best? from should any candidate be selected at all?

A separate NONE gate operates on the candidate representations and predicts

At inference time, the decision is simply:

where $q_i$ is the conditional probability produced by the relational judge.

This distinction is important because relative confidence is not the same as validity. A softmax must assign its probability mass somewhere. Even if every candidate is poor, one of them will still look best relative to the others.

The gate gives Wazn a second axis—selection plus existence—rather than trying to make one score serve both purposes.

If full probabilities are needed, the two components combine naturally:

The judge decides who wins. The gate decides whether the tournament should have a winner at all.

Training in two phases

Wazn is trained in two stages. The first stage is ordinary supervised training on the decision task. The second is a short invariance post-training stage whose purpose is different: it teaches the model that changing the wording of an instruction or a label should not change the underlying decision.

Phase 1: task learning

Phase 1 trains the model normally on the classification, retrieval, and decision mixture.

For a labelled example with gold candidate $y^*$, the judge is optimized with the usual negative log-likelihood:

When the NONE gate is trained on a batch, its binary loss is added separately, giving the Phase 1 objective:

For NONE rows there is no correct candidate, so the choice cross-entropy is masked and only the gate term remains:

At this stage, every row is treated independently.

That is enough to teach the model how to make decisions, but it does not directly teach an important property of a semantic classifier: two equivalent ways of expressing the same question should ideally result in the same distribution over answers.

Consider:

Which department should handle this request?

and:

Route this request to the appropriate department.

Or candidate labels such as:

payment authentication failure

and:

card verification problem

If these expressions are semantically equivalent in a particular training group, Phase 1 has no explicit objective saying that their predictions should agree. Each version simply receives its own cross-entropy loss.

This is what Phase 2 addresses.

Phase 2: semantic paraphrase invariance

Phase 2 is a short post-training stage starting from the Phase 1 checkpoint. Its objective is to make the decision distribution stable under meaning-preserving changes to the surface form of both the instruction and the candidate labels.

The central idea is simple:

even if the text used to express that problem changes.

Training examples are therefore organized into invariance groups. Every member of a group describes the same underlying decision problem:

• the state is unchanged;

• the correct semantic answer is unchanged;

• the candidate slots correspond to the same classes;

• only their wording changes.

The current recipe uses four variants per group:

The paraphrases deliberately vary in form. Instructions can be rewritten in different styles, while candidate labels may appear as short phrases, longer class descriptions, snake_case identifiers, or single-word labels.

This matters because Wazn is intended to operate on semantic classes, rather than memorize one canonical string for each class.

For example, a model should ideally behave similarly when presented with:

or:

provided that the two are defined to represent the same candidate in that invariance group.

Matching distributions, not just answers

The Phase 2 objective does not merely require all variants to have the same argmax. It attempts to align their entire probability distributions.

Suppose two members of the same group produce

Both select the same candidate, but they are clearly not equally certain. A loss based only on top-1 agreement would consider them identical. The invariance objective should not.

For every group, Wazn compares the predicted distributions of its members using Jensen–Shannon divergence. For two variants,

For a group containing several variants, the default objective averages the JS divergence over every unordered pair.

The full Phase 2 loss becomes

where $\mathcal{L}_{\text{inv}}$ is the semantic invariance loss. Setting $\gamma = 0$ recovers the original Phase 1 behavior.

An alternative formulation compares every group member against the group’s mean distribution rather than evaluating all pairs. This is cheaper, although it penalizes a single deviant member less strongly.

Training on whole groups

Invariance is fundamentally a property of sets of equivalent examples, so the batching strategy changes slightly during Phase 2.

All members of an invariance group are kept in the same batch. They are flattened into ordinary model inputs for the forward pass, while a group_index records which rows belong to the same semantic example.

This allows the ordinary task loss to be calculated per row while the invariance loss is calculated across rows belonging to the same group.

Replay examples from Phase 1 are also mixed into Phase 2. They have no invariance partners and are therefore treated as groups of size one ($|\mathcal{G}| = 1$). These examples participate in the ordinary supervised objective but are skipped by the invariance loss.

The replay data serves as a regression constraint: Phase 2 should improve semantic stability without erasing behavior learned during Phase 1. The current recipe mixes invariance groups with Phase 1 replay rows, with the replay share configurable during training.

Freezing the comparator

One detail of Phase 2 is particularly important: not every part of Wazn is allowed to adapt.

The trainable components include:

• the LoRA adapters;

• the choice readout embedding;

• the normalization immediately before the judge;

• the candidate-local representation layers of the judge.

The relational comparator $\psi$ itself is frozen.

This follows directly from the purpose of the invariance stage. A paraphrase changes how a candidate is represented, not the meaning of comparison itself. Ideally, “payment authentication failure” and “card verification problem” should be mapped to compatible candidate representations before they reach the comparator. The comparator should not need to learn a new notion of preference for every wording variation.

There is also a practical reason for freezing it. If the relational comparator were trainable, there would be a degenerate way to reduce the invariance loss: make the comparator increasingly flat.

A comparator that outputs almost the same value for every pair would produce highly consistent distributions across paraphrases—but it would no longer compare candidates effectively. In other words, $\mathcal{L}_{\text{inv}} \rightarrow 0$ could be achieved by destroying useful decision structure.

Freezing the relational component prevents that shortcut and forces Phase 2 to solve the intended problem—make equivalent wording produce equivalent representations—rather than make the judge stop caring about its inputs.

Measuring invariance

Accuracy alone is not sufficient to evaluate this stage. Phase 2 therefore tracks several complementary quantities.

The first is the invariance divergence $\mathcal{L}_{\text{inv}}$ itself. The second is the mean pairwise total variation distance:

This provides an intuitive measure of how much probability mass moves when the wording changes.

Finally, group agreement measures the fraction of invariance groups for which every variant produces the same top-1 prediction.

Together, these metrics distinguish three different outcomes: the same answer, the same confidence distribution, and the same performance on the original task.

The final evaluation therefore uses two separate test sets. Held-out invariance groups measure robustness to paraphrases and label rewrites that were not seen during Phase 2, while the original Phase 1 evaluation set is rerun as a regression check.

The goal of the second stage is not to teach Wazn new classes. It is to make the decision function depend more on meaning and less on the particular string used to express that meaning.

Large-$K$ inference

Wazn was trained with candidate sets of at most $K = 10$. Despite that, the relational judge generalizes reasonably well beyond the training regime. In direct full-set inference, groups with $K \approx 20$ remain robust, and accuracy can stay good even as $K$ grows further.

The main problem at large $K$ is not that the judge suddenly stops ranking candidates correctly. Two different scaling effects appear first.

The first is computational. A full relational comparison evaluates every candidate against every other candidate:

This requires $O(K^2)$ pairwise evaluations. For $K = 10$, this is negligible. For $K = 200$, it means tens of thousands of pairwise relations. The relational head is small compared with the backbone, but eventually the quadratic structure becomes unnecessary overhead.

The second effect is probabilistic. The candidate probabilities are obtained by applying a softmax over the relational scores:

As more plausible candidates are added, probability mass is distributed across a larger set. In practice, the correct candidate can remain the highest-ranked candidate while its absolute probability decreases. This is an important distinction:

A candidate that receives $0.8$ probability in a four-way problem should not necessarily be expected to receive the same probability when competing against 200 semantically related alternatives.

For this reason, Wazn does not need to perform one global $K \times K$ comparison when $K$ becomes large.

Tournament inference

Once the backbone has produced the candidate representations $p_1, p_2, \ldots, p_K$, the expensive language-model computation is finished.

The representations can then be divided into small groups, scored by the relational judge, and only the strongest candidates retained. For example, using groups of eight while retaining the top two candidates:

The final surviving candidates are then compared together in one full relational round. A simplified version is:

The important point is that the tournament operates on the already-computed candidate representations. No candidate text is prefilled again, and no candidate branch is passed through Qwen3.5 again. Only the small relational head is reused.

This makes the tournament fundamentally different from repeatedly asking an LLM to choose from smaller and smaller subsets of text.

Why keep more than one candidate?

A strict knockout tournament could use $8 \rightarrow 1$ at each stage. That is efficient, but brittle: a strong candidate placed in a particularly difficult group can be eliminated by a single local mistake.

Wazn instead works better as a high-recall pruning system. Keeping the top two candidates gives $8 \rightarrow 2$ at each stage. The goal of an early round is therefore not necessarily to identify the final answer. It is only to avoid eliminating it.

This is especially useful because the relational score of a candidate depends on the competitors present in its group $S$:

A different competitor set can produce a slightly different score. Candidates can therefore be reshuffled between rounds so that survival does not depend too heavily on one particular bracket.

From quadratic to approximately linear scaling

Suppose groups contain $G$ candidates and retain $R$, with $\rho = R / G$. The number of candidates over successive rounds is approximately

Each group requires only $O(G^2)$ relational work, and $G$ is fixed. The total work therefore follows a geometric series and scales approximately as $O(K)$ for fixed $G$ and $R$, rather than $O(K^2)$ for a single global round-robin comparison.

This gives Wazn two inference regimes:

This distinction is useful because the model does not need to have been trained with hundreds of simultaneous candidates to operate over a class universe of that size. The backbone still solves the same local problem it saw during training: understand one candidate in relation to the input. The large-$K$ machinery is mostly an inference-time operation over the resulting representations.

That is one of the reasons Wazn can scale beyond the candidate-set sizes used during training without requiring the full class inventory to be loaded into one monolithic prompt.

Preliminary evaluation

Wazn-2B v0.1 is still experimental, and a more systematic evaluation will follow in a separate release. The first tests on established datasets are nevertheless encouraging, particularly given that these datasets were not part of the training mixture.

These results should be treated as initial measurements rather than optimized benchmark numbers.

Wazn is slightly unusual in that a class is not represented by a fixed output neuron. Its meaning comes from the text supplied for that candidate, and the model can also consume candidate-specific few-shot examples. Consequently, benchmark performance depends partly on how the candidate semantics are presented.

For example, a class may be supplied simply as:

card_payment_wrong_exchange_rate

or as a semantic description:

The customer was charged an incorrect exchange rate for a card payment.

The second formulation gives the model considerably more information about what the class means. Similarly, few-shot examples can be attached directly to an individual candidate.

The numbers above therefore should not be interpreted as the ceiling of the architecture. A future evaluation will separately measure the effects of label descriptions, candidate-specific demonstrations, paraphrase invariance, NONE detection, and large-$K$ tournament inference.

Examples handled by the model

Benchmarks are useful, but they do not show some of the behaviors Wazn was designed for. The following are real requests and responses from the experimental checkpoint.

The JSON shown is the request sent to the inference server and the distribution returned by the model. Inference was done using https://github.com/numid-ai/wazn-experimental on an Nvidia RTX 4090.

1. Incident postmortem → root cause

Here the model receives a paragraph-length incident report and chooses among six runtime-defined root-cause categories.

Request

{
  "state": {
    "context": "Checkout latency rose sharply at 09:40, five minutes after a routine change to the load balancer. The change reduced the idle connection timeout from 60 to 5 seconds as part of a cleanup. Long-lived connections from the payment service were then dropped and had to reconnect on every request, adding about two seconds per call. No application code was deployed that day. Rolling back the timeout value at 10:15 restored normal latency immediately."
  },
  "questions": {
    "q": {
      "type": "choice",
      "instructions": "What was the root cause of this incident?",
      "criteria": {
        "code_bug": "A defect in code released by the team",
        "config_change": "A faulty configuration or infrastructure change",
        "capacity": "Load exceeded the available resources",
        "dependency_failure": "A third-party or upstream service failed",
        "process_error": "A mistake in a manual or organisational procedure",
        "security_attack": "A malicious attack on the system"
      }
    }
  }
}

Output

{
  "choice": "config_change",
  "confidence": 0.9987,
  "none_probability": 0.0025,
  "is_none": false,
  "probabilities": {
    "config_change": 0.9987,
    "process_error": 0.0012,
    "code_bug": 0.0,
    "security_attack": 0.0,
    "capacity": 0.0,
    "dependency_failure": 0.0
  }
}

Processing time: 0.11s server-side, 167 input tokens, 6 candidates.

This is a useful example because the correct answer is not simply a keyword match. The report contains references to application code, an upstream payment service, latency, and operational changes. The decision comes from assembling the causal sequence: a configuration change was made, connections began failing immediately afterward, and rolling the configuration back restored normal behavior.

2. Applying a business rule

Wazn can also condition a decision on rules supplied at inference time.

Request

{
  "state": {
    "context": "Customer bought a jacket 45 days ago. It is unworn with tags and the receipt is available. They ask for a refund.",
    "rules": "Refunds are not available after the first month; offer store credit instead."
  },
  "questions": {
    "q": {
      "type": "choice",
      "instructions": "What should we do?",
      "criteria": {
        "approve_refund": "Give the money back",
        "deny_refund": "Refuse the refund",
        "store_credit": "Offer store credit instead of money"
      }
    }
  }
}

Output

{
  "choice": "store_credit",
  "confidence": 0.9953,
  "none_probability": 0.0031,
  "is_none": false,
  "probabilities": {
    "store_credit": 0.9953,
    "deny_refund": 0.0046,
    "approve_refund": 0.0
  }
}

Processing time: 0.09s server-side, 83 input tokens, 3 candidates.

The interesting part is not this individual classification, but what happens when only the rule changes.

For the same customer request and the same three candidates:

no rule
→ approve_refund     0.99

"Unworn items with tags and a receipt are always refundable."
→ approve_refund     1.00

"Refunds are not available after the first month; offer store credit instead."
→ store_credit       0.995

The candidate set is unchanged. The decision follows the policy supplied with the request.

3. Knowing when none of the answers fit

A conventional softmax classifier must choose something.

Wazn’s NONE gate is intended to distinguish:

from:

Consider a support knowledge base containing ten FAQ entries.

Request

{
  "state": {
    "context": "can I apply for a job with you?"
  },
  "questions": {
    "q": {
      "type": "choice",
      "instructions": "Which FAQ entry answers this message?",
      "criteria": {
        "shipping_time": "How long does delivery take?",
        "return_policy": "How do I return an item?",
        "order_tracking": "How can I track my order?",
        "payment_methods": "Which payment methods do you accept?",
        "change_address": "Can I change my delivery address after ordering?",
        "cancel_order": "How do I cancel an order?",
        "invoice": "Where can I download my invoice?",
        "warranty": "Is my product covered by a warranty?",
        "password_reset": "I forgot my password, how do I log in?",
        "gift_cards": "How do I use a gift card or promo code?"
      }
    }
  }
}

Output

{
  "choice": "change_address",
  "confidence": 0.258,
  "none_probability": 0.9582,
  "is_none": true,
  "probabilities": {
    "change_address": 0.258,
    "warranty": 0.1978,
    "cancel_order": 0.1176,
    "return_policy": 0.0864,
    "invoice": 0.0778,
    "payment_methods": 0.0711,
    "password_reset": 0.0563,
    "order_tracking": 0.0493,
    "shipping_time": 0.0471,
    "gift_cards": 0.0386
  }
}

The relational judge still identifies the least bad candidate, because a ranking always has a winner:

But the separate gate assigns

The application can therefore hand the request to another system or a human instead of treating the forced-choice winner as an answer.

4. 219 classes with tournament inference

The previous examples use small candidate sets. The same interface can also operate over much larger runtime-defined ontologies.

The following request contains 219 DBpedia level-3 classes. Instead of running a complete $219 \times 219$ relational comparison, tournament inference uses groups of ten and retains the top two candidates from each group.

Request, with most labels omitted here for readability:

{
  "state": {
    "context": "Crawford v. Marion County Election Board, 553 U.S. 181 (2008), is a United States Supreme Court case holding that an Indiana law requiring voters to provide photo IDs did not violate the Constitution of the United States."
  },
  "questions": {
    "q": {
      "type": "choice",
      "instructions": "Which class does this Wikipedia article describe?",
      "criteria": {
        "AcademicJournal": "Academic Journal",
        "AdultActor": "Adult Actor",
        "Airline": "Airline",
        "Airport": "Airport",
        "... (215 more labels)": "..."
      },
      "tournament": {
        "group_size": 10,
        "top_k": 2,
        "seed": 0
      }
    }
  }
}

The tournament reduces the candidate population over three rounds:

Only the final ten candidates are compared together.

Output

{
  "choice": "SupremeCourtOfTheUnitedStatesCase",
  "confidence": 0.9927,
  "none_probability": 0.0024,
  "is_none": false,
  "tournament": {
    "rounds": 3,
    "groups_per_round": [
      22,
      5,
      1
    ],
    "labels_in_final_group": 10
  },
  "final_group_probabilities": {
    "SupremeCourtOfTheUnitedStatesCase": 0.9927,
    "Single": 0.0039,
    "Election": 0.0025,
    "Dam": 0.0005,
    "Cave": 0.0002,
    "Moss": 0.0001,
    "Model": 0.0001,
    "Fish": 0.0001,
    "Noble": 0.0,
    "OfficeHolder": 0.0
  }
}

Processing time: 0.32s server-side, 1,741 input tokens, 219 candidates.

This example is important because Wazn was never trained on 219-way decision problems—the largest training candidate sets had only $K = 10$.

The model is instead being used compositionally: the backbone computes the candidate representations once, while the relational judge repeatedly compares small subsets of those representations.

No candidate needs to be re-prefilled between tournament rounds.

Weaknesses and limitations

Wazn-2B v0.1 is an experimental checkpoint, and several failure modes are already clear.

The first is rule following when rules contain exceptions. The model usually handles straightforward policies well, but performance is less reliable when a rule has nested conditions, carve-outs, or exception clauses. In these cases, Wazn can over-apply the general rule and miss the exception that should dominate the decision.

A second weakness appears in fine-grained class hierarchies. When both a broad category and a more specialized category are plausible, the judge can be underconfident and prefer the more general class. For example, given both:

database_issue

and:

database_connection_pool_exhaustion

the model may recognize the correct semantic area but still assign too much probability to the broader category.

This suggests that the current judge is sometimes better at identifying the right semantic neighborhood than at resolving very fine-grained distinctions inside that neighborhood.

The model is also weaker on tasks that require precise counting or temporal reasoning. Questions that depend on exact quantities, event ordering, elapsed time, or combining several numerical facts are less reliable than ordinary semantic classification and retrieval.

Another limitation is language coverage. The current checkpoint was trained only on English data. While the Qwen3.5 backbone itself is multilingual, Wazn’s decision behavior and calibration outside English have not been trained or evaluated systematically.

Finally, performance degrades as the input context becomes longer. The experimental training mixture focused mostly on short and medium contexts and did not exceed roughly 4,096 tokens.

The underlying backbone can accept longer sequences, but that does not mean the decision head was trained to make reliable use of them. Long-context classification, retrieval, and rule application should therefore be considered out of distribution for this checkpoint.

These limitations are important because they mostly reflect the current training distribution, not hard architectural constraints. Future checkpoints will need more data targeting exceptions, fine-grained class boundaries, numerical and temporal composition, multilingual inputs, and substantially longer contexts.

Conclusion and next steps

Wazn-2B v0.1 is still an experimental release, but the first results suggest that separating candidate representation, candidate comparison, and candidate validity is a useful direction for decision models.

The current model already supports runtime-defined classes, explicit NONE detection, candidate-order equivariance, candidate-specific few-shot examples, and tournament-style inference over candidate sets far larger than those seen during training.

There is still a lot to validate properly.

The next release will focus on four areas:

• A full benchmark suite. I plan to publish results on established classification and retrieval datasets, together with calibration and invariance metrics such as accuracy, ECE, TVD, NONE detection, and sensitivity to candidate reordering and paraphrasing.
• A larger and more diverse training mixture. The current checkpoint was trained on only a few thousand rows. The next dataset will expand the range of nominal decision behaviors, structured inputs, code, logs, JSON, aggregation, and large candidate spaces.
• Scaling the backbone. Wazn-2B is the first checkpoint. The same architecture will be tested at larger parameter counts, initially around 4B and 8B, to understand how much semantic decision quality improves with backbone capacity.
• A dedicated inference stack. The current implementation is not optimized for the architecture. I plan to fork or extend vLLM so that shared-prefix execution, candidate branching, representation reuse, and tournament inference can be handled natively rather than emulated on top of a standard generation-oriented runtime.