Strands Decider 2B: A Model That Scores Menus, Not Prose

2026-10-02
Brian

· Updated
Subscribe
Get New Posts by Email
New RepoSquare posts in your inbox: comparisons, setup guides and explainers on open-source tools and AI coding agents. No spam, and you can unsubscribe anytime.

Strands Decider 2B is a small open model that scores options you already listed. It does not write paragraphs, code or summaries. AWS’s Strands Labs team announced it on 1 October 2026 under Apache-2.0, launching with the hobson-v19 checkpoint. The current checkpoint to download is StrandsAgents/strands-decider-2B-qwen3.5-v1-2610, which Hugging Face lists as created on 9 October. The older hobson-v19 and hobson-v21 checkpoints stay published, but the README now demos v1.

Checked on 23 October 2026 against the GitHub README and evaluation/results.md, the launch post and the Hugging Face model files. I installed strands-decider 0.2.0 on this box and ran the payout example on CPU. The numbers match the README’s recorded sample.

What shipped

Simple mindmap of Strands Decider 2B: scores not writes, three question types, agent gate, 0.9 rule, Apache-2.0
ItemDetail
ProductStrands Decider 2B (decision model / “system one”)
Released checkpointstrands-decider-2B-qwen3.5-v1-2610
BaseQwen/Qwen3.5-2B-Base; the README puts the model at 1.9 billion parameters
LicenseApache-2.0 (weights, training data notes, and scripts)
JobScore listed choices; attach a confidence
Not a jobWrite free-form text

A language model guesses the next token, appends it, and guesses again. That loop is writing. A decider deletes that last step. You give it a state (the text to judge) and a menu. It returns scores and a confidence.

Language model writes text; decider scores listed choices

How the pointer head replaces the writing head

Transformer torso kept; language head removed; pointer head scores options

A transformer has a torso that turns tokens into hidden states, and a language-model head that turns the last state into the next-token guess. Strands kept the torso, removed the speaking head and attached a pointer head of about a million parameters that scores the options you listed. It compares the hidden state at the answer position with the hidden state at each option’s last token, in one forward pass with no decoding loop. Training uses LoRA: a small adapter (rank 16 per training run) on a frozen torso, plus the new head. The Hugging Face repo for v1 holds the LoRA adapter, head.safetensors and tokenizer files, not a full copy of the base model; the CLI pulls the Qwen base as well.

LoRA adapter stuck on a frozen 2B torso

Because the head has no per-option weights, the label set lives in your request, not in the model. One masked softmax is read back three ways:

Three question types: choice, noul, and score from the payout example
TypeWhat you askWhat you get
choiceWhich of these options?One pick plus per-option scores and a confidence
noulYes or no (the project’s name for this type)A number near 1 for yes, near 0 for no
scoreWhere on this ordered scale?A score plus per-rung probabilities

What I ran on this box, and the 0.9 rule

I used the README’s payout ticket on CPU. The first attempt was killed by the out-of-memory killer (exit 137) on this 16 GB machine, which had roughly 8 GB free at the time. After adding 8 GB of swap, the same command finished. I re-ran it on 23 October with the model already cached; it took 1 minute 45 seconds of wall time on CPU and printed the same numbers:

pip install "strands-decider[cpu]"
strands-decider ask StrandsAgents/strands-decider-2B-qwen3.5-v1-2610 \
  --device cpu \
  --state "Help! My payouts have been failing for 3 days! " \
  --choice "Which team should handle this?=billing,sales,retail" \
  --noul "Does this convey urgency?" \
  --score "How frustrated is the writer?=calm,frustrated,depressed"

Billing wins at confidence 0.659. Urgency is 0.858. Frustration lands near “frustrated” with confidence 0.509. Those match the README sample within rounding. The confidence numbers matter more than the labels.

The README says that on short classification tasks the model has never seen, answers at confidence 0.9 or higher are right about 95% of the time. Below that, confirm or ask a person. The demo itself is at 0.659 for the team choice, so under their own rule it is not an automatic route. The README adds that frontier LLM inference APIs do not expose anything equivalent.

Where it sits in an agent loop

Agent loop: high-confidence menu acts; writing jobs go to large model
StepWho does it
Read the ticket / page / logYour harness
Ask a menu question with a confidence thresholdDecider
If confidence ≥ your thresholdAct (route, tag, allow, deny)
If confidence is low, or the job needs wordsCall a large writing model
Write the reply, patch or summaryLarge model

That split is the point. You stop paying a frontier model to pick “billing” versus “sales” on every ticket. For auditing the larger agent around it, see how to audit an AI agent with iFixAi. For local VRAM planning on bigger models, see running large models on a 12 GB GPU.

The CLI also has strands-decider serve, which exposes POST /v1/systemone. For an agent built with the Strands Agents SDK, the natural pattern is: harness prepares state → decider answers typed questions → harness branches on confidence → only then call a writer model. Keep the threshold and the fallback path in your code, not in a prompt.

Published numbers and how v1 was trained

Published JevBench scorecard snapshot for Strands Decider 2B

For the released soup checkpoint, measured with the code on main:

MeasureSoup (released)
JevBench public, window 4096180 / 231 tasks right
JevBench Brier0.280
JevBench ECE0.072
HotpotQA (never trained on)0.832
held-out short tasks (6,000)0.653
code tasks, held out (1,996)0.836
Decision Index (balanced skill)30.82

The release checks re-ran JevBench through the code on main on one NVIDIA L4 (180/231 again) and report that the README examples give the same answers on --device cpu and --device cuda, with probabilities within 0.006. By tier, v1 scores 1.000 easy, 0.931 standard and 0.586 hard. I re-ran only the README example, not the 231-task suite. If you are weighing it against other decision models, our Jev vs Laya bake-off covers the same JevBench scorecards.

What will fail if you trust it too early

Four failure modes: ignored questions, long docs, threshold mismatch, poor transfer
  1. You ignore the confidence. A 0.66 pick is a suggestion, not a gate.
  2. Your menu is wrong. The model can only score options you listed. It will not invent ” Tier-2 escalations”.
  3. The document is long and the question is subtle. Held-out short-task accuracy in the table is 0.653 overall, and the hard JevBench tier is 0.586.
  4. You needed writing. Summaries, patches and emails still need a language model.
  5. Threshold mismatch. If your product needs 99% precision, a 0.9 rule that yields ~95% on short tasks is not enough.

The evaluation write-up says v1 is a soup of three training runs (seeds 0, 1 and 2) of one recipe. Each layer’s LoRA update is the exact mean of the three runs, stored as one rank-48 adapter, and the three readout heads are stacked so the soup’s score for an option is the mean of theirs. Training data is a balanced mix of 246,678 rows — about twice v21 — including code-task rows and 22,168 rule-application rows where gemma-4-31B-it wrote the policy text and cases, code computed the gold answers, and Claude was used only as an independent filter. A KL term (weight 0.3) keeps answers close to the older hobson-v19 release. The builders of that data mix are not all on main yet, so you cannot rebuild the exact training set from the public repo alone.

That matters if you care about reproducibility. You can run the released weights. You cannot, today, recreate every training row from the GitHub tree.

Decider versus a classical classifier versus a chat model

PropertyClassical classifierStrands DeciderChat LLM
OutputOne label from a fixed setScores over a menu you pass at request timeFree text
New labelsNeeds a retrainChange the menu stringPrompt change
ConfidenceDepends on the model and calibrationCalibrated confidence; 0.9 rule publishedNot exposed by frontier APIs, per the README
Speed / costUsually fastestMedian 115 ms per JevBench question on an RTX 3090 (v19, same architecture)A full generation call per decision
WritingNoneNoneYes

Use a classical classifier when the label set is stable and you have enough labelled data to train. Use the decider when the menu changes by product area but the job is still “pick or score”. Use a chat model when the answer is words.

Troubleshooting a local install

SymptomLikely causeWhat to try
Process killed with exit 137 on CPUOut of memory while loading the modelAdd swap, close other processes, or use --device cuda / mps / mlx
Answers differ from README in the last digitsExpected across devicesThe README says other devices differ in the last digits; the release check is within 0.006
Model writes a sentenceWrong productYou are not calling strands-decider; check the CLI
Confidence always low on your dataDomain shift or bad menusTighten options; collect a labelled sample; raise the human-review band
First run is slow to startThe CLI fetches the base model plus the adapter and headLet the first download finish; later runs use the cache

When to try it, and when to skip

Decision guide: try Strands Decider when you have a menu; skip when you need writing
Try DeciderSkip Decider
Routing, tagging, allow/deny with a fixed menuOpen-ended chat
You can set a confidence threshold and a human fallbackYou need a paragraph, diff or plan
You want Apache-2.0 weights on your own machineYou expected a 2B chatbot
Your agent already calls a big model for every micro-decisionYour labels change every day and you have no menu

Install with pip install "strands-decider[cpu]" (or the cuda / mps extras; the README says the mlx extra ships with the next release). On a 16 GB machine with other work running, plan for swap or a GPU. CPU latency is a separate question from the GPU figures in launch posts; we measured the same gap for another decision model in running Laya without a GPU.

Common questions about Strands Decider 2B

Does it write answers?

No. It scores options you provide. There is no next-token writing head in the released design.

Which checkpoint should I download?

StrandsAgents/strands-decider-2B-qwen3.5-v1-2610, unless you have a reason to pin the older hobson-v19.

Is confidence 0.66 good enough to auto-route?

Not under their 0.9 rule. Treat it as a suggestion and confirm.

Can I fine-tune it?

Yes, under Apache-2.0. The repo includes training notes; rebuilding the exact v1 mix from main alone is incomplete according to results.md.

Get New Posts by Email
New RepoSquare posts in your inbox: comparisons, setup guides and explainers on open-source tools and AI coding agents. No spam, and you can unsubscribe anytime.

1 comment

Leave a comment