Sampler Settings for Local LLMs on Apple Silicon: Temperature, Top-p, Min-p, and Repetition Penalty Measured (2026)

At every generation step the model emits a vector of logits, one score per token in the vocabulary, and none of it is text yet. The sampler is the small deterministic or stochastic procedure that collapses that vector into the single token you actually see. Teams spend weeks choosing a model and a quantization and then accept whatever sampler defaults the runner ships, which is backwards, because the sampler is the setting that decides whether your extraction pipeline returns valid JSON, whether your agent picks the right tool, and whether a long summary loops on itself. On Apple Silicon the tuning is close to free: the forward pass is the cost, and sampling operates on the logits after it, so there is no memory or throughput penalty for getting it right. This post covers what each knob does, how they interact, and where to set them for real work on an M5 Max.

What Each Knob Actually Does to the Distribution

Temperature is applied first, and it rescales the logits before they become probabilities. A temperature below 1.0 sharpens the distribution, concentrating mass on the already likely tokens; a temperature above 1.0 flattens it, giving unlikely tokens a larger share. At exactly 0 the sampler degenerates to greedy decoding, always taking the single highest logit, which is deterministic but not always correct. The important thing is that temperature happens before any truncation, so it changes the shape that the truncation step then operates on.

Top-p, also called nucleus sampling, sorts tokens by probability and keeps the smallest set whose cumulative probability reaches p, discarding the rest before sampling. It is adaptive in one direction: when the model is confident the nucleus is tiny, and when the model is uncertain the nucleus is large. Its weakness is that it has no floor on individual token probability. In a flat distribution top-p at 0.95 can admit a long tail of individually implausible tokens simply because collectively they sum to five percent, and any one of them can be sampled.

Min-p attacks that weakness directly. Instead of a cumulative threshold, it sets a floor relative to the top token: keep every token whose probability is at least min_p times the probability of the most likely token. If the top token sits at 0.6 and min_p is 0.1, the floor is 0.06, and anything below that is cut regardless of how the tail sums. This makes min-p scale-aware in a way top-p is not. When the model is confident, the floor is high and the candidate set is small; when the model is genuinely uncertain, the floor drops and more tokens survive. In practice min-p lets you run a higher temperature for variety without admitting the garbage tokens that a raw top-p would let through.

Repetition penalty is a different kind of tool. It divides or subtracts from the logits of tokens that have already appeared, discouraging the model from repeating them. It fixes the visible failure of a model looping on a phrase, but it is a blunt instrument: it penalizes a token for being present, not for being wrong, so a high penalty can push the model off a token that was correct precisely because it needed to repeat, such as a variable name in code, a key in JSON, or a proper noun in a summary.

How They Interact, Which Is the Part That Bites

The parameters are not independent, and treating them as independent is the most common mistake. Because temperature reshapes the distribution before top-p or min-p truncate, raising temperature widens the nucleus that top-p then keeps, so a temperature and top-p that behaved well together break when you change one alone. The modern recommendation, and the reason min-p has largely displaced top-p for local work, is that min-p tolerates a higher temperature gracefully: you can push temperature to 1.0 or above for creative variety and let a min_p floor of 0.05 to 0.1 keep the sampling honest, because the floor rescales with the model's confidence at each step.

Stacking a repetition penalty on top of an already tight sampler is where correctness quietly dies. If min-p has already narrowed the field to two or three plausible tokens and a repetition penalty then demotes the correct one because it appeared earlier, the sampler is forced onto a worse token. For anything structured, the safer pattern is a low or zero repetition penalty and a tight min-p or low temperature, and if repetition is a real problem, reach for a presence-style penalty with a small value rather than a large frequency penalty. The quantization you run interacts here too, because a heavily quantized model has a noisier logit distribution, which makes an aggressive sampler more likely to tip into a wrong token; we covered that precision side in the quantization quality writeup.

Settings by Job, Measured on an M5 Max

These are directional results from a 128GB M5 Max running a 30B class model in MLX, framed as observations rather than universal constants, because the exact numbers shift with model and quantization. The pattern is what transfers, not the decimals.

Job	Temperature	Truncation	Repetition penalty	Observed behavior
Structured extraction / JSON	0.0 to 0.2	min_p 0.1 or greedy	none	highest parse rate, near deterministic
Agentic tool calls	0.2 to 0.4	min_p 0.05	none or 1.05 presence	reliable tool selection, minor variety
Code generation	0.3 to 0.6	min_p 0.05	none	correct and varied, no loop with min-p
Long form writing	0.8 to 1.1	min_p 0.05 to 0.1	1.1 frequency	fluent, varied, low repetition
Brainstorming / ideation	1.0 to 1.3	min_p 0.02 to 0.05	1.1 frequency	high diversity, min-p floor blocks garbage

Two observations stand out. First, for structured extraction and tool calling, near-greedy decoding wins outright: the moment you need the output to parse or to name a real tool, variety is a liability, and pushing temperature toward zero raised the valid-output rate more than any other single change. That is consistent with what we found building local tool calling with strict JSON schema, where the sampler and the schema constraint reinforce each other. Second, across the creative jobs, min-p at a modest floor let temperature run higher than top-p safely allowed, producing more interesting output without the occasional incoherent token that a comparable top-p setting let slip through.

The Config, and Why It Costs Nothing to Change

Every mainstream local runner exposes these knobs, and because they operate after the forward pass, changing them per request is free. Here is the shape of a per-job sampler config you can switch on at call time rather than baking one global default.

# Per-job sampler presets. These ride on top of the same loaded model;
# switching presets costs nothing because sampling happens post-forward-pass.
PRESETS = {
    "extract":  {"temp": 0.1, "min_p": 0.1,  "rep_penalty": 1.0},
    "tools":    {"temp": 0.3, "min_p": 0.05, "rep_penalty": 1.0},
    "code":     {"temp": 0.5, "min_p": 0.05, "rep_penalty": 1.0},
    "prose":    {"temp": 1.0, "min_p": 0.08, "rep_penalty": 1.1},
}

def sample_params(job: str) -> dict:
    p = PRESETS[job]
    # top_p left at 1.0 on purpose: min_p is doing the truncation.
    return {"temperature": p["temp"], "min_p": p["min_p"],
            "top_p": 1.0, "repetition_penalty": p["rep_penalty"]}

Notice top-p is pinned at 1.0 in every preset. That is deliberate. Running min-p and top-p together stacks two truncation rules whose interaction is hard to reason about, and for local work min-p alone is the cleaner primitive. If your runner does not support min-p, a top-p of 0.9 to 0.95 paired with a lower temperature is the fallback, but the scale-aware behavior is what you give up.

For extraction and agent work you can go further and remove sampling from the equation entirely with a grammar or schema constraint that masks invalid tokens before the sampler ever sees them. That combines with a low temperature rather than replacing it, and it is the most reliable way to guarantee parseable output; the mechanics are in the structured outputs writeup.

When This Applies To Your Stack

Tune the sampler per job, not once globally, because the correct setting for a JSON extractor and the correct setting for a marketing draft are almost opposites. Default to near-greedy with a min-p floor for anything that has to parse or name a tool, let temperature climb with a min-p floor for anything that has to read well, and keep repetition penalty near zero unless you actually observe looping, because it demotes correct tokens as readily as wrong ones. Prefer min-p over top-p for the truncation step, and do not run both at once.

The reason to care is leverage: this is the rare tuning lever that changes output quality materially and costs nothing at inference time, so ignoring it means leaving accuracy on the table for free. If you are standing up local inference where the output feeds a parser, an agent, or a customer-facing surface, the sampler is part of the reliability surface, not a cosmetic setting, and getting it right is the kind of AI infrastructure work we do at Contra Collective. The model gives you a distribution; whether that distribution becomes something you can trust is a decision you make in the sampler.

FAQ

Should I use top-p or min-p for local models? Prefer min-p for local work. It sets a floor relative to the top token's probability, so it stays sensible as the model's confidence changes and tolerates a higher temperature without admitting implausible tail tokens. Use top-p only as a fallback when your runner lacks min-p, and do not run both together.

What temperature should I use for structured or JSON output? Near zero, between 0.0 and 0.2, ideally paired with a schema or grammar constraint. When output has to parse, variety is a liability, and the lower the temperature the higher the valid-output rate. Save higher temperatures for writing and ideation.

Does a repetition penalty hurt code generation? It can. Repetition penalty demotes any token that already appeared, including variable names, JSON keys, and syntax that legitimately repeats, so a high value can push the model onto a wrong token. For code, keep it at or near 1.0 and control diversity with temperature and min-p instead.

Do sampler settings cost extra time or memory on Apple Silicon? No meaningful amount. Sampling operates on the logits after the forward pass, which is where nearly all the compute and memory go, so changing sampler parameters per request is effectively free and there is no throughput reason to leave them at the defaults.

Why do temperature and top-p interact? Temperature reshapes the distribution before top-p truncates it. Raising temperature flattens the distribution, which widens the nucleus top-p keeps, so a pair that worked together breaks when you change one alone. Min-p reduces this coupling because its floor rescales with the model's per-step confidence.