Sek Chai, CTO and cofounder of Latent AI, joins The Tech Trek to talk about what it actually takes to get AI running on the edge. We explore the real-world constraints of power, compute, and hardware diversity, why an agent-assisted workflow can accelerate MLOps, and how to choose models that are good enough to ship. Sek also breaks down lessons from selling into the federal market and explains why a clear guiding principle beats chasing every shiny opportunity.
Key Takeaways
Edge AI is a different game than the cloud. Power limits, hardware diversity, and deployment realities have to shape the design from day one.
The best model is the smallest one that delivers the capability and latency you need. Bigger isn’t always better.
An AI agent that understands your data, model, and hardware personas can move teams from idea to deployment much faster.
Whether you’re selling to federal or commercial buyers, lead with capability, then meet security and compliance needs.
A strong tenet should guide product direction and market focus more than raw market size.
Timestamped Highlights
00:30 Why edge optimization matters and what Latent AI does
01:09 The messy reality of heterogeneity and power constraints in edge deployments
02:54 Why most edge AI projects never ship and how an agent can change that
05:03 Mapping MLOps personas and tailoring the workflow for each
11:49 Selling to both federal and commercial buyers without losing focus
15:55 Building a company around a tenet rather than chasing every market
Quote of the Episode
“It’s not the model that you’re really chasing after. It’s that capability.”
Pro Tips
Define capability and constraints first—latency, frame rate, and power budget—then pick and optimize the model.
Collect and use telemetry from experiments and deployments to guide model and hardware choices.
If federal markets are in play, bake security and compliance into your early prototypes.
Call to Action
Enjoyed this episode? Follow The Tech Trek, rate us on Apple or Spotify, and share it with someone working on an edge AI project.
