From the Lab Page 10 / 46

Insights, engineered.

Technical deep-dives, AI strategy, and engineering perspectives from the team building autonomous systems.

Sep 4, 2026 AI Infrastructure

Gemini 3.8 Flash: Google's Cheap Agentic Tier Tested (September 2026)

Google DeepMind released Gemini 3.8 Flash on September 2, 2026, and the headline is that it costs exactly what 3.7 Flash cost, 0.75 dollars per million input tokens and 3.75 per million output, while beating the older model on every benchmark Google published. It ships a 1 million token context, 64K output, multimodal input across text, image, audio, video, and PDF, and it is tuned for long horizon coding and autonomous agents. On DeepSWE v1.1 it reads 73.7 percent against 65.3 for 3.7 Flash, and on Terminal-Bench 2.1 it posts 89.4 percent. Google also claims it beats Claude Opus 5 on three of the benchmarks it reported. The catch worth naming up front is the price schedule: those rates hold through December 31, 2026, then double on January 1. This post tests whether the cheap tier is now good enough to be the default engine for real agent work, where it holds up, and where the flagship tiers still earn their keep.

Sep 4, 2026 AI Infrastructure

GPT-6 Astra vs Claude Fable 5.1: Agentic Coding Benchmarks Tested (September 2026)

Two frontier flagships shipped inside 72 hours. OpenAI released GPT-6 Astra on September 3, 2026, calling it the most intelligent and aligned model it has published, and Anthropic released Claude Fable 5.1 on September 1 with a 75 percent cut to cache read pricing. Both carry a 1 million token context window and both list at 10 dollars per million input tokens and 50 dollars per million output tokens, so the sticker price will not decide this for you. What decides it is the agentic coding numbers and the cost you actually pay once caching enters the picture. On Terminal-Bench 4.0, Astra reads 57.7 percent against Fable 5.1 at 55.8 percent; on DeepSWE v1.1, Astra is 74.1 percent. Those gaps are real but narrow, and narrow gaps get erased by the parts of the bill that the leaderboard never shows. This post puts the two side by side on the benchmarks that matter for coding agents, then argues for a default based on total cost per solved task rather than a headline percentage.

Sep 4, 2026 AI Infrastructure

GPT-6 Astra vs GPT-5.6 Sol: Is the Upgrade Worth It (September 2026)

OpenAI released GPT-6 Astra on September 3, 2026, less than two months after GPT-5.6 Sol shipped on July 9. The launch framing is that Astra is the most intelligent and aligned model OpenAI has published, and the benchmark tables back a real capability step. But Sol did not get worse; it got cheaper. After the August 22 price cut, Sol sits near 4 dollars per million input tokens and 20 dollars per million output, while Astra lists around 10 and 50. That is roughly a 2.5x premium on both sides of the ledger for the newer model, and a 2.5x premium is a lot to pay for a capability gap that, on many workloads, you will never actually exercise. This post treats the choice as a procurement decision rather than a launch day upgrade reflex: what Astra buys you, what it costs, and the specific workloads where Sol remains the correct default even though it is now the older model.

Sep 4, 2026 AI Infrastructure

Kimi K3 vs GPT-6 Astra: Open Weights Against the Flagship on GPQA and Cost (2026)

OpenAI released GPT-6 Astra on September 3, 2026, and it currently leads the field on GPQA Diamond at 96.0 percent, the highest abstract reasoning score OpenAI has published. It costs 10 dollars per million input tokens and 50 per million output, with a Fast mode at roughly double that. The interesting comparison is not another flagship, it is Kimi K3, the open weights model from Moonshot AI that posts 93.5 percent on the same GPQA Diamond benchmark while charging 2.50 and 14 dollars per million tokens on its hosted API and shipping weights you can run yourself. This post puts the two side by side on the benchmark that actually compares cleanly, walks through why the Terminal-Bench numbers do not compare across versions, and works out the cost gap that decides which model belongs in which part of your stack.

Sep 4, 2026 AI Infrastructure

Qwen3.8 27B on Apple Silicon: MLX Throughput, Quantization, and Where It Fits (2026)

Alibaba's Qwen team shipped Qwen3.8 27B in August 2026: a dense 27.78 billion parameter multimodal model under Apache 2.0, with a native 262,144 token context, a vision encoder, and open weights on Hugging Face. On paper it is the most convincing dense, locally deployable model near 30 billion parameters yet, posting 90.3 on LiveCodeBench v6 and 61.7 on SWE-bench Pro. The reason it matters for Apple Silicon specifically is that a dense 27B model at 4-bit fits in the unified memory of a well specced Mac, and MLX now uses the M5 Neural Accelerators that llama.cpp does not. This post covers how to size the model for your machine, which quantization to run, the config to load it under MLX, and how to reason about throughput without trusting a number someone posted for a different chip.

Sep 4, 2026 Headless Commerce

Shopify SimGym: What AI Shopper Simulation Can and Cannot Test (2026)

Shopify SimGym, part of the Winter 2026 Edition, runs AI shoppers through your storefront to simulate buyer behavior before you push changes live. The AI shoppers browse products and collections, use navigation and search, add items to the cart, and report the friction they hit, and they are trained on the behavioral data behind billions of Shopify purchases. It sounds like a way to test your store without waiting for real traffic, and for themed stores it is a useful directional screen. The catch that matters for anyone reading this on the headless side is that SimGym operates on Shopify themes, and a headless storefront is not a theme. This post covers what SimGym actually tests, what it explicitly does not, and how headless teams should think about pre ship validation when the tool cannot see their front end.

Sep 4, 2026 Headless Commerce

Shopify Storefront API 2026-07: Breaking Changes and the Headless Migration (2026)

Shopify's Summer 2026 edition, codenamed Compass, shipped 65 updates, and the one that lands on every headless team is Storefront API version 2026-07 with breaking changes to the cart and product query structures. On a themed Shopify store these version bumps are invisible; the platform absorbs them. On a headless build, where your storefront talks to the Storefront API directly, a breaking change to the query shapes you depend on is a scheduled outage if you ignore it. Shopify runs a quarterly versioning cadence and supports each version for a fixed window, so 2026-07 is not optional, it is a migration with a deadline attached. This post covers what changed in the cart and product queries, why headless builds are uniquely exposed to Storefront API versioning in a way themed stores are not, and how to run the migration behind a version pin so you upgrade on your schedule rather than on an outage.