When to use this local approach
JevStyle is a local option for scoring predefined choices on Apple silicon with MLX, or on CPU/CUDA with PyTorch. Choose it when you need typed option probabilities; keep a generative model when you also need a written reply. See the named-tool comparison and Semantic Router integration.
Match the method to the decision
If you have representative labeled text and stable categories, build a supervised classifier as a baseline. If routes are described by example utterances, evaluate an embedding router. If the decision depends on instructions and a fixed set of options, evaluate a typed decision model such as JevStyle. If you also need a written answer or flexible extraction, keep a local generative model in the comparison.
These are alternatives to test, not a universal ordering. A model that succeeds on a public benchmark can still fail on your languages, labels, documents, or unusual inputs. Keep the input set fixed when comparing methods.
Four ways to make local decisions
| Approach | You provide | You receive | Useful starting condition | Check before adopting |
|---|---|---|---|---|
| Supervised text classifier | Labeled examples; text features such as TF–IDF | A class prediction; some estimators also expose scores or probabilities | Stable labels and enough representative training data | Class imbalance, vocabulary/domain shift, and probability calibration |
| Embedding router | A local encoder and route examples | Similarity scores and a proposed route | Routes can be represented by example utterances | Similarity is not automatically a probability; test ambiguous and out-of-scope requests |
| Typed decision model | Text or JSON, instructions, and predefined options | Probabilities for yes/no, choices, or ordered score levels | The application needs a bounded decision rather than a generated answer | Task-specific accuracy, calibration, option wording, and fallback coverage |
| Generative LLM with structured output | A prompt and an output schema or grammar | Generated text or a structured object | You need extraction, explanations, or a response as well as a label | Schema compliance is not semantic correctness; validate fields and measure the full request path |
The scikit-learn text feature guide describes bag-of-words and TF–IDF representations. Semantic Router's local quickstart demonstrates routes built from utterances and a local embedding model. These offer useful baselines alongside instruction-driven decision models.
For generated output, llama.cpp's grammar documentation explains output constraints. A valid JSON object still needs application-level checks: a syntactically valid {"team":"billing"} can select the wrong team.
When fixed-option scoring is a useful fit
Consider a service request: “I was charged twice.” A classifier might assign billing; an embedding router might match a billing example; a typed decision model can score your defined destinations and a separate refund question. A generative model can also propose a label, but generating a paragraph is unnecessary if the next component only accepts a route.
JevStyle is one open-source implementation of the typed-decision approach. Its local API returns probabilities over supplied options; your application decides whether to accept the result or send it for review. It does not execute a selected action. Multiple questions may share one input state, for example team, refund intent, and urgency.
Use the local text classification tutorial for a Python implementation, the routing guide for a fallback workflow, or the Claude Code guard guide for an additional tool-call review signal. A probabilistic guard is not a sandbox or authorization system.
Running local decisions on Apple silicon
A runtime and a model are different choices. MLX LM provides local language-model generation and fine-tuning on Apple silicon. llama.cpp supports local inference across hardware, including Apple silicon. JevStyle supplies matching decision runtimes for its own models; its probability outputs do not come from a generic text-generation command.
| Input and machine | JevStyle starting point | Important boundary |
|---|---|---|
| Text on Apple silicon | 0.8B v3 or 2B v4 with MLX | Both MLX builds are text-only. Evaluate quality and memory on the actual input lengths. |
| Images on an Apple silicon Mac or CUDA machine | 2B v4 with PyTorch | Explicitly select PyTorch on a Mac; automatic backend selection otherwise favors MLX for text. |
| CPU-only text | PyTorch with --device cpu | GPU benchmark latency does not predict CPU latency. |
| GGUF decision weights | The matching JevStyle scorer | jev-score for 0.8B v3; jev-score-v2 for 2B v3/v4. Follow the release-specific build instructions. |
The model comparison separates download sizes from runtime memory and lists the input limits for each release. Weight files are only part of the memory budget; framework state, context, and image processing also need space.
Start a local server and score a choice
Use Python 3.10 or newer in a virtual environment. Choose one setup below for the same local API. The first start downloads model files; later requests can use the downloaded assets without the hosted JevStyle service.
python -m pip install "jev-style[mlx]"
jev-style serve --release 0.8b --backend mlxpython -m pip install "jev-style[torch]"
jev-style serve --release 2b --backend torchpython -m pip install "jev-style[torch]"
jev-style serve --release 0.8b --backend torch --device cpuLeave the server running. In a second terminal with the same environment:
jev-style decide "I was charged twice" \
--url http://127.0.0.1:8765 \
--choice "Which team should handle this request?::billing,shipping,account"The command prints the actual model result; this page does not promise a particular prediction. Open the local Playground to inspect options and scores. For GGUF, use the package's matching scorer instructions instead of substituting a generic chat runner.
Make the comparison on your workload
- Define the correct decision and collect representative examples, including ambiguous, rare, and out-of-scope inputs.
- Fix a final test set before tuning prompts, route descriptions, or thresholds. Record model revision, backend, precision, and hardware.
- Compare errors per class, calibration, accepted-request coverage, and end-to-end latency. Include serialization, queueing, and fallback requests if they are part of production.
- Send low-confidence outputs and runtime failures to a person or existing LLM. Test how that fallback behaves before automating downstream actions.
A threshold such as 0.90 is not a general safety rule. An embedding similarity, a model's option probability, and a transformed confidence field are different quantities. Choose the threshold using the actual score exposed by the method you deploy.
Follow the evaluation and threshold guide. The published JevStyle benchmark page records existing self-runs and their limitations; it is not a head-to-head test of the four approaches on this page.
Common selection questions
Do I need a generative LLM to return a label?
No. A supervised classifier, embedding router, or typed decision model may fit that step. A generative model remains useful when the task also needs an explanation, extraction, or open-ended response. Compare against a simple baseline before replacing an existing system.
Can I score options locally without sending private inputs to a hosted API?
Yes, if the model, encoder, and inference path run locally. Download the required dependencies and model assets first, and check the configuration of every component in the workflow. JevStyle's hosted API and Hugging Face demos are separate remote services.
Are embedding scores and decision-model probabilities interchangeable?
No. Similarity measures match in an embedding space; option probabilities belong to a particular model, input, and choice set. Neither guarantees correctness on a new domain. Calibrate and validate the acceptance rule for the score you actually use.
Can stock Ollama or MLX generation commands return JevStyle decision scores?
Use those tools for compatible generative models. JevStyle's current decision releases require their matching decision runtime to expose option probabilities. Use the package's MLX, PyTorch, or GGUF setup rather than treating decision weights as an ordinary chat model.
Is the largest model always the right choice?
No. Accuracy, memory, warm and cold latency, and fallback frequency depend on the workload and backend. Compare them on the same held-out inputs. A model's download size or public benchmark result alone does not settle that choice.