Aug 15, 2026 / AI Models
Most vision benchmarks test whether a model can describe an image, which is the easy half of the problem and rarely the half that matters at work. MMMU tests something harder: whether a model can solve a college level question where the image is the problem, not an illustration of it. The figure might be a circuit diagram you have to trace, a chart whose axes carry the answer, a chemical structure, a medical scan, or a geometry construction, and getting the reasoning right depends entirely on reading the visual correctly first. That is why MMMU is a better predictor of whether a model can do real multimodal work than a captioning score, where a model can sound fluent about an image it only half understood. We ran Gemini 3.5 Pro and Claude Opus 5 across the MMMU disciplines under the same harness, measuring overall accuracy and, more usefully, where each model's errors come from: misreading the image versus reasoning wrongly from an image it read correctly. Those two failure types call for different fixes, and the model that wins on the headline number is not always the one that fails more safely on the questions you care about.
Aug 15, 2026 / Headless Commerce
Picking a helpdesk looks like a support team decision and turns into an architecture decision the moment your storefront goes headless. In a standard Shopify theme the helpdesk apps assume the storefront and the order data live in the same place, and the deep integrations that put a customer's order history in the agent's sidebar mostly just work. Decouple the front end, split order state across Shopify and an OMS, push financial truth into NetSuite or QuickBooks, and that convenient assumption breaks. Now the question is not which helpdesk has the nicest inbox; it is which one can assemble a coherent view of a customer whose data is scattered across four systems, and which one lets an agent take an action, refund, cancel, edit, that actually writes back to the system of record rather than to a copy that drifts. Gorgias, Zendesk, and Kustomer answer that question from three different starting philosophies, and the wrong pick shows up not in the demo but a year in, when agents are alt tabbing between six tabs because the sidebar only ever showed half the truth. This is a comparison of how each platform models the customer and the order when the commerce stack underneath it is decoupled.
Aug 15, 2026 / AI Models
The original MMLU stopped being a useful benchmark once frontier models pushed past ninety percent, because at that point the remaining gap is noise and the four answer choices mean a blind guess is right a quarter of the time. MMLU-Pro was built to fix both problems: it expands each question to ten answer choices instead of four, which cuts the value of a lucky guess, and it deliberately adds harder distractors, the wrong answers that look right until you actually work the problem. That combination turns a knowledge quiz into a reasoning test, because a model can no longer coast on recognizing the topic; it has to reason its way past a plausible trap. We ran Grok 4.5 and Gemini 3.5 Pro across the MMLU-Pro disciplines under the same harness and the same prompt, measuring overall accuracy and, more usefully, where each model's errors come from and which disciplines swing the result. The headline gap is small. The discipline level story is where the two models actually differ, and it tells you which one to trust for the kind of reasoning your work depends on.
Aug 15, 2026 / AI Infrastructure
A single question answered by a local model is easy to reason about: you pay a prefill for the prompt, then decode the answer, and the tokens per second number you benchmarked tells you roughly how long it takes. An agent is a different animal, because it is not one generation, it is a loop: the model emits a tool call, your code runs the tool, the result gets appended to the conversation, and the model reads the whole thing again to decide the next move. Run that five or eight times and the latency you feel has almost nothing to do with the decode speed you measured on a one shot prompt. The cost that dominates a local agent run is the repeated prefill of a conversation that grows with every step, and on a single M5 Max that prefill competes with everything else for the same GPU. Most people profiling a slow agent look at the wrong number, blame the model size, and swap to a smaller model that does not fix the problem because the problem was never decode. This post breaks a real agent loop into its parts, measures where the time goes across the steps, and shows why prefix caching is the single change that turns a sluggish local agent into a usable one.
Aug 15, 2026 / Headless Commerce
Adding third party sellers to a storefront sounds like a merchandising decision and is actually an architecture decision. The moment a single order can contain items from two different sellers, everything downstream changes: the cart has to split, each seller needs their own fulfillment path, the payment has to divide and pay out to multiple parties, and the catalog has to absorb thousands of products you did not create and cannot fully trust. A marketplace platform exists to own that complexity so your commerce core does not have to, and the three that matter for a headless build, Mirakl, Marketplacer, and Nautical Commerce, sit at three different points on the spectrum from heavyweight enterprise operator platform to modern composable transaction engine. The wrong choice shows up not in the demo but eighteen months in, when seller onboarding is a bottleneck, order splitting fights your OMS, or the payout model cannot handle the seller structure your business actually has. This is a comparison of where each platform draws the line between what it owns and what it leaves to your stack, because for a headless marketplace that boundary is the whole decision.
Aug 15, 2026 / Identity
Supabase Auth and Firebase Auth solve the same problem in opposite ways: one leans on Postgres and open standards, the other on a managed Google runtime. This breakdown covers session models, authorization, pricing, and how each fits an AI powered backend.
Aug 15, 2026 / Search
Typesense and Meilisearch are the two open source search engines most teams reach for in 2026, and they feel similar until you push them. This breakdown covers relevance, vector search, scaling, and hosting so you can pick the engine that fits your product.
Aug 11, 2026 / AI Models
Most code benchmarks ask a model to write one function in isolation, which is the easiest shape of the problem and the least like real work. ClassEval asks for something harder and more honest: generate an entire class where methods depend on each other, share instance state, and only work if the whole thing is internally consistent. A class is not a bag of independent functions; it is a small system with an interface, private helpers, and invariants that hold across methods, and getting one method's contract wrong quietly breaks the two methods that call it. That interdependence is exactly what makes ClassEval a better predictor of whether a model can contribute real code than a function level benchmark, where a model can score well by pattern matching common snippets. We ran Claude Opus 5 and GPT-5.6 Sol on ClassEval under the same harness, measuring both the strict class level pass rate, where every method must work together, and the method level rate, where methods are scored individually. The gap between those two numbers, per model, is the interesting part: it tells you whether a model writes coherent classes or just plausible methods that do not add up.
Aug 11, 2026 / AI Infrastructure
On a hosted API you rarely think about what happens when a user closes the tab mid-answer, because the provider eats the wasted compute and you were billed per token anyway. On a single M5 Max serving your own traffic, that same abandoned request is your problem, because the model keeps decoding tokens into a socket nobody is reading, and every one of those tokens is a decode step stolen from the requests that are still waiting. Cancellation is the part of local serving that almost nobody implements until the machine is saturated, at which point it becomes the cheapest throughput win available. A user who reloads, a browser that times out, an upstream gateway that gives up after thirty seconds: each leaves a generation running to its full max_tokens unless something tells the runtime to stop. The fix is not exotic, but it is not automatic either, and the difference between a server that aborts on disconnect and one that does not is the difference between a machine that degrades gracefully under load and one that wedges. This post measures what an abandoned request actually costs on an M5 Max, how cancellation propagates through llama-server and mlx, and where the naive implementation quietly fails to free anything.
Aug 11, 2026 / Headless Commerce
Affiliate platforms were built for the classic Shopify theme, where a tracking script drops on every page and a conversion pixel fires on the order status page, and that quiet assumption breaks the moment you go headless. On a custom storefront there is no theme to inject a snippet into, no Shopify order status page you control, and often a checkout that lives on a different domain than the storefront, so the browser side attribution that these tools lean on has three places to fail before a sale is ever credited to a partner. The consequence is not subtle: an affiliate program with broken attribution pays commissions on sales it did not drive, misses sales it did, and erodes partner trust the first time a creator notices their referrals are not being counted. Refersion, Impact, and PartnerStack solve the same problem, tracking a referral from click to paid commission, but they sit at three different points on the spectrum from lightweight ecommerce tracker to enterprise partnership platform, and their fit for a headless build depends almost entirely on how each one handles conversion tracking when there is no theme and how much of the attribution you are willing to move server side. The right choice depends less on the dashboard than on where the tracking fires and who reconciles it when the browser lies.
Aug 10, 2026 / Headless Commerce
A Shopify Plus headless storefront backed by NetSuite lives or dies on the integration layer between them, and that layer is rarely a single sync. Orders flow one way, inventory and fulfillment flow back, products and pricing flow out, refunds and returns reconcile in both directions, and every one of those flows has to survive a rate limit, a malformed record, and a midnight NetSuite maintenance window without losing data or double posting. Celigo, Workato, and Patchworks are three credible ways to own that layer without hand rolling it, and they sit at three different points on the same spectrum. Celigo ships prebuilt NetSuite and Shopify connectors and a managed integration app, trading flexibility for speed to live. Workato is a general automation platform that will connect almost anything, trading commerce specificity for breadth and governance. Patchworks is commerce native middleware built for exactly this retail data shape, trading ecosystem size for fit. The right choice depends less on the feature grid than on who maintains the flows, how you recover from a failed sync, and what the platform costs once your order volume is real.
Aug 10, 2026 / AI Models
A model can top a reasoning leaderboard and still fail as an assistant, because a real assistant task is rarely one question with one answer; it is a chain of small correct steps where a single wrong turn poisons everything after it. GAIA measures exactly that. Its questions are conceptually simple for a person but require the model to actually do things: search the web, open a file, read a table, do arithmetic on what it found, and carry the result into the next step, often across ten or more actions. The trap is that GAIA questions have a single verifiable answer, so partial credit does not exist; getting nine steps right and the tenth wrong scores identically to getting the first step wrong. That is what makes it a better predictor of assistant reliability than a one shot benchmark, and it is why raw reasoning scores mislead here. We ran Gemini 3.5 Pro and Grok 4.5 across the GAIA validation set with an identical tool harness, one attempt per question, and the headline accuracies are closer than the difference in how they fail. The story, as always, is where each model drops its steps: which tools, which reasoning depth, and whether the failure is a bad plan or a botched tool call.