From the Lab Page 08 / 46

Insights, engineered.

Technical deep-dives, AI strategy, and engineering perspectives from the team building autonomous systems.

Sep 16, 2026 AI Infrastructure

SWE-bench Verified in 2026: What the Score Actually Measures Now

SWE-bench Verified is the coding number every vendor quotes, a 500 task set of real GitHub issues that a model has to fix by producing a patch that passes the hidden tests. In 2026 the headline scores crossed 90 percent, and at almost the same moment the benchmark's credibility fell apart. Berkeley researchers showed a 10 line Python file scoring 100 percent without solving anything. OpenAI deprecated its own copy of the set after an internal audit reported that a large share of tasks had broken tests. Independent work measured how often a passing patch came from the model recalling the answer rather than reasoning to it. None of this means benchmarks are useless, but it does mean a raw SWE-bench Verified percentage is now one of the least informative things you can know about a coding agent. This post works through what the score measures, where it breaks, and what to look at instead.

Sep 15, 2026 AI Infrastructure

Gemini 3.8 Flash vs Claude Opus 5: Coding, Agentic Benchmarks, and the Token Cost Catch (2026)

Google shipped Gemini 3.8 Flash on September 2, 2026, at 0.75 dollars per million input tokens and 3.75 per million output, and it beats Claude Opus 5 on three of the benchmarks Google chose to publish. Opus 5 sits at 5 and 25 dollars per million and holds the top score on SWE-bench Verified. On sticker price the Flash tier looks like an easy win, but 3.8 Flash is built to burn more thinking tokens than the model it replaces, which means the real bill per task can climb well above what the headline rate implies. This post puts the two side by side on the numbers that compare, and works through when the cheaper model is actually cheaper.

Sep 15, 2026 AI Infrastructure

Mac mini M6 for Local LLM Inference: What the Refresh Actually Changes (2026)

Apple announced the M6 Mac mini on August 25, 2026, and for anyone running models locally the interesting numbers are memory bandwidth and total unified memory, not the CPU core count. The M6 lifts bandwidth to 170GB/s on the 24GB and 32GB configurations, up from 153 on the M5, and adds a 12-core GPU and a dual Neural Engine on TSMC's 2 nanometer process. It also caps at 32GB of unified memory, which is the number that decides what you can and cannot run. This post works through what the refresh actually changes for local inference: real but modest decode gains, a hard ceiling on model size, and the point where you should stop looking at the mini and buy a Mac Studio instead.

Sep 15, 2026 AI Infrastructure

Sakana Fugu Ultra v2 vs GPT-6 Astra: Orchestration Against the Flagship on Benchmarks and Cost (2026)

Sakana AI released Fugu Ultra v2 on September 11, 2026, and it is not a language model in the usual sense. It is an orchestration model: a system that decides how a pool of specialized models should split and combine work on a task. Sakana reports best or joint-best results on five of eight benchmarks and top-two placement on seven of eight, including 74.3 on DeepSWE, while pointedly excluding GPT-6 Astra and Claude Fable 5.1 from the pool it draws on. GPT-6 Astra is the opposite bet: one monolithic frontier model at 10 and 50 dollars per million tokens. This post compares the router of specialists against the monolith on benchmarks, cost, and what each means for how you build agent systems.

Sep 15, 2026 Engineering

Shopify Hydrogen Framework vs Hydrogen Core: Picking a Headless Path on Next.js (2026)

Shopify rebuilt Hydrogen in 2026 so it runs on any React stack, including Next.js, and split it into two products that answer different questions. The Hydrogen Framework is the opinionated path: built in routing, commerce hooks, and a project structure that assumes you are starting fresh. Hydrogen Core is the lower level path: a set of components and utilities you drop into an existing React or Next.js app when you already have infrastructure you are not going to throw away. Picking wrong means either fighting an opinionated framework you did not want or rebuilding conveniences Core leaves out. This post is the decision framework, plus what the v2026.4 browser proxy change means for both.

Sep 15, 2026 Engineering

Shopify Storefront MCP: Building an In Store AI Agent for Cart and Checkout (2026)

Shopify's Storefront MCP is how you wire an AI agent into a headless store: a server that exposes catalog search, cart state, and product data as tools an agent can call. The catch, as of September 2026, is that the original Storefront MCP cart tools are deprecated in favor of a new set under the Universal Commerce Protocol, the Cart MCP and Checkout MCP servers, and if you built on the old get_cart and update_cart calls you are now on a migration path. This post walks the current architecture for an in store agent that reads the catalog, builds and iterates a cart, and hands the buyer off to checkout, and is honest about what just changed and why.

Sep 8, 2026 AI Infrastructure

How to Build an MCP Server: A Practical TypeScript Guide (2026)

The Model Context Protocol went from a promising idea to the default way agents reach tools and data, which means building an MCP server is now a routine engineering task rather than a research project. The concepts are small: a server exposes tools, resources, and prompts, and a client running inside an agent calls them over a transport. The hard parts are not the hello world, they are the decisions after it: what to expose as a tool versus a resource, which transport fits local versus remote deployment, how to test the thing without a full agent in the loop, and how to ship it so it is observable and safe. This guide builds a minimal server in TypeScript, then walks the decisions that turn a demo into something you would actually run in production.

Sep 8, 2026 Engineering

Shopify 3PL Integration: Order and Inventory Sync Patterns (2026)

A Shopify 3PL integration looks simple on a whiteboard: orders go out to the warehouse, tracking comes back, inventory stays in sync. In production it is where the most expensive commerce bugs live, because three systems disagree about the same physical pallet and the customer sees the disagreement first. This post lays out the integration patterns that hold up: using the FulfillmentOrder API instead of the legacy fulfillment model, deciding which system owns inventory truth, handling partial and split shipments, and treating tracking and inventory updates as event streams that will drop and duplicate. It covers the build versus connector decision honestly, and the failure modes (oversell, stuck fulfillments, ghost inventory) that separate a demo from a warehouse that ships on time.

Sep 8, 2026 Engineering

Shopify Metafields: Custom Data Modeling for Headless Storefronts (2026)

Shopify metafields are how you extend Shopify's data model without leaving the platform, and in a headless build they are the difference between a storefront that can express your catalog and one that is fighting the schema. The trouble is that teams reach for metafields as a dumping ground, scatter untyped key value pairs across products, and end up with data no query can trust. This guide covers the modeling decisions that actually matter in 2026: metafields versus metaobjects, why definitions and typed validation are not optional, how to structure data so a headless frontend can query it cleanly, and the mistakes (untyped values, reference sprawl, and treating metafields as a database) that make custom data a liability instead of an asset.

Sep 8, 2026 Engineering

Shopify QuickBooks Integration: Sync Architecture and API Patterns (2026)

Most Shopify QuickBooks integrations fail for the same reason: they treat the accounting system as a mirror of the storefront instead of a ledger with its own rules. Push every order in as an invoice and you get a QuickBooks file that no accountant trusts and no reconciliation ever balances. This post lays out the three ways teams connect Shopify to QuickBooks Online in 2026, the data model mismatch that causes most of the pain (orders are not invoices, payouts are not payments, and refunds are not deletions), and the API patterns that keep the two systems agreeing. It covers the prebuilt app route, the iPaaS route, and the custom build route, with an honest read on when each one is the right call and what each one costs you in fidelity, control, and maintenance.

Sep 8, 2026 Engineering

WooCommerce to Shopify Plus Migration: A Technical Playbook (2026)

Migrating from WooCommerce to Shopify Plus is not a data export followed by a data import. It is a translation between two philosophies: a self hosted WordPress plugin where everything is a post in MySQL and every behavior is another plugin, and a hosted platform with an opinionated data model and a fixed checkout. The migrations that fail treat it as a file copy and discover too late that URLs changed, that a critical plugin has no Shopify equivalent, and that subscription customers silently stopped billing. This playbook walks the WooCommerce specific parts that actually decide the outcome: mapping the WordPress data model, preserving search rankings through redirects, achieving plugin parity, migrating the payment gateway without losing tokens, and running a cutover that does not drop orders.

Sep 7, 2026 AI Tooling

Cursor Origin vs GitHub: Agent First Code Hosting After the SpaceX Deal (2026)

SpaceX closed its 60 billion dollar all stock acquisition of Anysphere, the maker of Cursor, in mid August 2026, and within days Cursor shipped Origin, a code hosting platform built directly into the editor. Origin is not trying to be GitHub. It is a git host designed for the way AI agents actually use a repository: constant cloning, pushing, and parallel branches that collide and need merging. The early beta ships four pieces, repositories, pull requests, code browsing, and GitHub sync, and it is built on rearchitected technology from Cursor's December 2025 acquisition of Graphite. For teams already living inside Cursor with agents running most of the day, keeping the repo where the agents work removes a real seam. It also raises a real question, because your source now syncs into infrastructure owned by SpaceX, and the data terms are new. This post covers what Origin does, how it stacks against GitHub, and the trade you are actually making.