30 Jul 2026

strategy
inference
evaluation
on-device

Finding the Problems Where Capital Doesn't Decide

Summary

My founder friend said our Indic LLM story sounded two years old next to physical AI. The useful work was not chasing a fashionable comparable. It was killing wrong prescriptions until the only remaining space was hardware-constrained inference, evaluation that spends test once, and products that compound from owned usage.

My founder friend put it cleanly: why are you still building an Indian LLM when everyone is moving to physical AI and world models? The story, they said, sounded two years old.

I did not argue about the calendar. The sharper question was whether the diagnosis was about narrative packaging or about the operating plan underneath it. Most feedback of this kind collapses into that one question, and the answer is never "pick a fashionable comparable and rebuild toward it."

Separate the diagnosis from the prescription

The instinct, including mine, was to reach for a company that had already lived the plot twist. Scale AI is the name that tends to come up: start as labeling infrastructure, then reframe as something closer to enterprise AI applications. After Meta took a large stake in mid-2025, that story got more complicated, not less.

Reuters reported that Google, Scale's largest customer, planned to cut ties once Meta's stake became public, and that Microsoft, OpenAI, and xAI were also pulling back. Scale's interim CEO wrote that this was not a pivot or a winding-down, while also saying the company would double down on applications for governments and enterprises. Later reporting described a business under pressure to lean harder on that applications side after frontier-lab clients became harder to keep.

So when someone says "sound like Scale," I hear two stacked claims that need separating. One is that the narrative is stale relative to where capital and headlines are pointing. The other is that the operating plan is wrong. Copying another company's midpoint after its own customers and identity were forced to renegotiate is not a strategy. It is cosplay.

The question I kept returning to: are you telling me the narrative is stale, or that the operating plan is wrong? Answer that first. Everything else is a follow-on.

Aggregation is not defensible from the supply side

A plausible prescription, and one we scoped seriously, was to become the layer that routes and evaluates every model. One contract, many backends, continuous scoring. It dies quickly against what already exists on the supply side.

OpenRouter already exposes hundreds of models across dozens of providers behind one API. (Sacra has estimated the company's annualized revenue in the tens of millions of dollars as of early 2026; I treat that as an analyst estimate, not a filing.) LiteLLM is open source under the MIT license and free to self-host. AWS already ships Intelligent Prompt Routing inside Bedrock. Supply-side aggregation is a feature that hyperscalers and open-source projects can ship as table stakes.

Demand-side aggregation is a different animal. A platform that owns the users, the workflows, and the buying decision can put routing behind a product people already open. Naver is the clean example in my head: the models are ingredients; the demand is the business.

flowchart LR
  subgraph supply["Supply-side aggregation"]
    direction TB
    S1["Many models / providers"]
    S2["One API or gateway"]
    S3["Defensibility: thin<br/>(OpenRouter, LiteLLM, Bedrock routing)"]
    S1 --> S2 --> S3
  end

  subgraph demand["Demand-side aggregation"]
    direction TB
    D1["Owned users and workflows"]
    D2["Models chosen behind the product"]
    D3["Defensibility: the demand<br/>(e.g. Naver-style platforms)"]
    D1 --> D2 --> D3
  end

  supply -.->|we are not this| X["Not our wedge"]
  demand -.->|requires product gravity we do not buy| Y["Different company"]

We do not win by collecting every model. Someone else will do that cheaper, and already has.

A position built only on evaluation gets scooped

I next scoped a benchmark: how much do VLA (vision-language-action) models degrade when instructions leave English? The LIBERO manipulation suite looked like a natural host. Before we spent a research quarter building it, I searched whether the measurement already existed.

It did. Dong et al. translated LIBERO into ten languages and reported severe non-English degradation, with success rates dropping on the order of 30-50% in their setup (arXiv:2606.11906, ACL 2026). Separately, Beyond English documented multilingual gaps across VLA models on LIBERO and related environments. The instrument we were about to invent was already public.

Around the same time we were using GEPA (Genetic-Pareto prompt optimization: reflective evolution over prompts using execution traces rather than sparse RL rewards). The open-source GEPA repo is the reference implementation; we later open-sourced our own TypeScript harness, redrob-eval, for Indic LLM configs under Apache 2.0. Useful tooling. Not a moat.

The lesson I would write on the wall: a lab is judged on findings, not on its instruments. If the finding is already posted, your instrument is a homework assignment.

Elimination, stated plainly

I will write the chain without dressing it up.

flowchart TD
  A["LLMs commoditized<br/>by open weights"] --> B["Image and audio follow"]
  B --> C["Industry attention moves to<br/>world models + physical AI"]
  C --> D{"Where can a small team with<br/>limited budget still choose?"}
  D --> E["World models"]
  D --> F["Physical AI hardware"]
  D --> G["Constrained-hardware inference"]
  E --> E2["Compute wall"]
  F --> F2["Manufacturing wall"]
  E2 --> H["Cut"]
  F2 --> H
  G --> I["Surviving space"]

  style E2 fill:#f4c7c3,stroke:#a33,color:#222
  style F2 fill:#f4c7c3,stroke:#a33,color:#222
  style H fill:#f4c7c3,stroke:#a33,color:#222
  style I fill:#d4e8d4,stroke:#2a6,color:#222

Open weights compressed the time you can sell a private frontier text model. In India, Sarvam open-sourced 30B and 105B reasoning models under Apache 2.0 in March 2026. That does not end the market for serving Indic workloads well. It does end the fantasy that a small team out-pretrains the next open release on narrative alone.

World models need compute we do not have. Physical AI hardware needs manufacturing we do not have. Peers who tried to stay full-stack have been forced to admit the physics of capital: Krutrim shifted toward AI cloud services after pausing chip work and foundation-model ambitions. (Medianama reported the same realignment more sharply on the model and chip pause.)

What remains, for us, is the place where the ceiling is set by device memory, latency, battery, and deployment constraints rather than by who can buy the next training cluster. Constrained-hardware inference.

Find the problem where capital does not decide

A small wrench seated in one narrow notch of a massive gear: a visual metaphor for a fit that capital cannot force.

"Whoever has more data and more capital wins" is close to proven for transformer-scale competition. It fails in the places where physics sets the ceiling. No round size puts a frontier cloud model into 3GB of RAM. No hiring binge makes a 40ms phone budget suddenly host a 70B dense model.

That is the core insight I was looking for. Not "edge is trendy." Not "mobile AI is the next narrative." The useful cut is: find problems where additional capital cannot buy through the constraint, then build measurement and products that live inside that constraint honestly.

Open runtimes already treat this as a real environment. ExecuTorch and llama.cpp exist because people keep needing models to run where the GPU is not. As of late July 2026 those repos sit at roughly 4.8k and 122k GitHub stars respectively; the numbers will age, the need will not.

Do not fuse two arguments that only sometimes overlap

There is an Indian-language argument and a constrained-hardware argument. They look like one thesis when you squint: both are about serving people outside the English, cloud-default stack. They separate as soon as the path goes through robotics.

A camera has no language in it. A grasp success metric does not care which Indic script the operator typed. What transfers from Indic text work into physical or video work is not the language coverage itself. It is the instrumentation: how we measure quality under the actual constraint, how we refuse English-benchmark cosplay, how we keep cost relative rather than absolute.

Keeping those two arguments fused produces a strategy that feels coherent in a slide and fails on the first non-linguistic task. I unfused them on purpose.

Check what the model was trained to do before asking it to do something else

We looked hard at using a vision-language model (VLM) to judge video of a hands-on skill: does the candidate actually perform the steps, in order, correctly enough for a hiring decision?

That question tempts you to ask the model for causality and next-state prediction, the things world models are trained toward. Diagnostic benchmarks are blunt about the mismatch. TimeBlind shows frontier video LLMs still lean on static shortcuts when temporal structure is the only difference between pairs. PhysBench finds VLMs weak on physical-world understanding relative to their common-sense fluency. CLEVRER made the same shape of failure early for explanatory, predictive, and counterfactual questions.

World models are not the same object. Meta's V-JEPA 2 (paper) is trained for video understanding and predictive structure, including action-conditioned planning variants. A VLM aligned for image-language description is optimized for a different objective. Using one as if it were the other is a category error with expensive consequences.

Design rule that came out of this: ask only what happened. Never why it happened. Never what happens next. Observation, not prophecy.

On-device is not a security mechanism

flowchart TD
  R["Problem: hire for a hands-on skill<br/>with limited budget"] --> A["Track A: Scoring"]
  R --> B["Track B: Integrity"]
  R --> C["Track C: Compression"]

  A --> A1["On-device VLM judges what happened<br/>against a rubric"]
  B --> B1["Prove authenticity of capture<br/>(provenance, device signals)"]
  C --> C1["Fit the model into RAM / latency / battery"]

  C1 -.->|"does NOT satisfy"| B1

  style B fill:#fff3cd,stroke:#a80,color:#222
  style C fill:#e8e8e8,stroke:#666,color:#222

The counter-intuitive turn: pushing inference onto the candidate's own phone does not prevent fraud. It creates a new attack surface. A remote server can still be fed a deepfake; an on-device model can be fed a deepfake locally, or the device can be instrumented to lie about what the model saw. Compression (Track C) makes the model fit. It does not make the footage true.

The fix is proving authenticity, not detecting fakes after the fact. We are building toward provenance signing in the spirit of C2PA Content Credentials: capture a chain that says where this media came from and how it was edited.

Separately, we proposed signing gyroscope and accelerometer traces alongside the video, then checking IMU consistency against the motion the frames imply. We have not found prior art for this low-cost check; treat it as a novel proposal from this process, not as established industry practice. If someone has shipped it quietly, I would rather be corrected early than wrong loudly.

Budget gates change the risk structure more than any technical choice

The managerial change that mattered most was not a model pick. It was refusing a lump three-year budget in favor of gated tranches. Each gate kills a track, or forces a rewrite, before the full commitment is spent. That sounds boring next to world models. It is how a small team survives being wrong about Track A, B, or C without mistaking sunk cost for strategy.

Write down what survives failure, in currency, not adjectives

Standalone inference-optimization companies keep ending as features inside larger balance sheets. Deci was acquired by NVIDIA in 2024. Neural Magic was acquired by Red Hat the same year. OctoAI, formerly OctoML, was acquired by NVIDIA as well. Optimization alone is valuable. It is rarely the last independent company standing.

So I write assets down the way I want the board to hear them. A trained proprietary model is worth writing off at zero in three years; open weights will pass it. Measurement data, especially measurement under constraints other people refuse to collect, does not depreciate the same way. The harness, the labels, the failure cases, the authenticity traces: those are the inventory. Adjectives are not.

Closing

A dark building at dusk with one lit window: one question still open.

One thing is still unverified going into execution. Whether an on-device VLM's skill judgments agree with human graders well enough to trust. I will score that agreement with quadratic weighted kappa (QWK), the usual ordinal-agreement metric for graded judgments. The number has to clear a bar we set before we see the labels. Until it does, Tracks B and C are scaffolding around a question we have not answered.

That is the honest ending. The elimination was clean enough to act on. The remaining uncertainty is measurable. Capital cannot buy the answer; only the study can.