Software Engineering with Agentic AI

Build software without drowning in prompts!

Sonny Bhatia

Thursday, September 3, 2026

The Problem

• Syntax is now a commodity
  • AI can write and analyze and explain programs
  • Software Architecture requires deliberate thought
• The “Black Box” Trap
  • Unstructured prompting (“vibe coding”) bypasses systems thinking and domain expertise
  • Students lack architectural understanding
• Brittle Outcomes
  • Technical debt; no maintainability
  • Trial-and-error prompting does not scale to complex, reliable software

How do we transition from unstructured prompting to disciplined, agentic systems engineering?

What is Success?

Measure of Performance

• Token maxxing!!
  • Throwing tokens at problems until they go away
• Brute forcing was rewarded by tech companies; employees spending below threshold were penalized
• Tokenmaxxing: great for AI vendors
  • Claude Code
  • OpenAI Codex

Measure of Effectiveness

• Resources spent
  • GPU
  • Electricity
  • Time
• Model availability - frontier vs open weights
• Scrum team velocity per dollar spent?
• Token cost per line of code?
• Understanding of your work…?
• Model maxxing: Efficient Model Routing

The Big Idea: Agentic AI

• Agent = specific role with a focused set of instructions
• Artifacts = agent output at each step
• Humans validate intermediate artifacts

Simple software engineering pipeline inspired by OpenSpec

Core Concepts

Harness

• Software that orchestrates agents
• Manages the control flow and context
• Exposes tools to language models
• Makes your AI agents agentic
• e.g. Claude Code, Codex, OpenCode, pi.dev

Tools

• Deterministic functions for agents (~API)
• Interact with local or remote systems
• Harnesses provide local tools
  • e.g. read_file, run_command, fetch_url
• MCP servers provide remote tools

AGENTS.md

• README file for agents
• Per-directory context; grab the closest
• Given to every agent, always!
• Keep it really short and focused

Skills

• Agents load skills as needed
  • Unlock new capabilities!
• Reusable instructions loaded into context
• Loads ~5k-20k tokens \(\rightarrow\) bloats context

Challenges with current systems

• Proprietary harnesses are black boxes
  • System prompts are often huge; user has no control
• Context grows quickly
  • Difficulty reasoning and following instructions
  • Performance degrades on smaller models
  • Larger context = higher costs; who profits?
• Agents often orchestrated manually
  • Limited control over spawning sub-agents
  • Difficult to scale processes and parallelize tasks
• Models, configuration, and pricing can change without notice
  • Surprises await…

Guiding Principles

1. Markdown everywhere
2. Own your agent roles / system prompts
3. Context window: keep it compact
4. Control flow: manage agent interactions and artifacts
5. Agents: small, simple, and focused
   • Each agent does one thing well
6. Security
7. Governance

Inspiration: 12 Factor Agents: Principles for building reliable LLM applications

Markdown: The Lingua Franca

• Beautiful, simple, and human-readable

  • # and ## for headings
  • - for lists
  • …everything else is nice to know

• Human-readable + machine-parsable = ✨

  • Standard: specifications, designs, code, and documentation all in same format
  • Highly token-efficient format for LLMs

• Markdown Syntax Cheatsheet

• Text-based Diagrams

  • Mermaid
  • Graphviz
  • …and many more

Own Your Agent Roles / System Prompts

• Agent prompts define roles, processes, and expectations
• Focused instructions; no data or product artifacts
• Modular: Keep prompts scoped to a single, focused objective
• Chain various agents to orchestrate a complex process
  • Agents can spawn sub-agents!
  • Calling agent checks sub-agent’s output and continues the thread as needed
• Treat prompts as code; use version control
• Instead of loading skills, develop simple agents!!

Context Window

• Context is precious; don’t dump extensive amounts of texts into it
• Models (and people) perform better on focused tasks with smaller contexts
  • Shorter context = faster responses, cheaper calls, and fewer hallucinations
• Start new conversations often!
  • Artifacts provide continuity
• Skills often bloat context windows; use them judiciously
  • Pulled on-demand by agents probablistically; cost a lot of tokens
• Use custom agents that will do one thing well
  • The “Need to Know” Basis: Feed agents only the specific files or functions they need
  • Sub-agents start with fresh context and a small set of instructions

Control Flow

• Ideally: deterministic processes manage agent interactions
• Realistically: a more powerful model manages the long-running goal-driven task
  • Tasks sub-agents sequentially or in parallel as needed
  • Frontier models excel at this
• Use intermediate artifacts
  • pause execution, allow human review, ensure reliable outcomes
• Open-ended prompting enables discussion around artifacts
  • Start new conversations often!
• LLM-as-a-judge: Use models to review and approve artifacts
• Implementation is driven by artifacts, not prompts

Agents: Small, Simple, and Focused

• Analyst: Develop a proposal. Produce specifications.
• Architect: Write the design / architecture document.
• Planner: Develop and refine the plan. Identify dependencies and opportunities to parallelize tasks.
• Tech Lead: Review and execute the plan. Delegate extensively.
  • Developer: Implement the plan.
  • Tester: Test the implementation. Write unit tests.
  • Reviewer: Ensure each task is complete to standard.

Agents task sub-agents as needed.

Security

• Sandboxing & Isolation
  • Run agent execution environments in ephemeral, restricted containers
• Local & Open-Weight Models
  • Keep classified data and research entirely in-house
• Deterministic Controls & Checkpoints
  • Manage control flow and require approvals at key steps

Risks: runaway costs, spillage, or destructive actions

• Sandboxing & Isolation

  • An agent generating and testing code operates in an isolated Docker container with zero access to the host file system or network

• Local Models - can be deployed on an isolated enclave

• Deterministic Controls & Checkpoints

  • Agent asks for permission before running git push, giving developer tight control over what gets added to the repository

Governance

Agent Privileges and Standardization

• Agent RBAC (Role-Based Access Control): Apply the Principle of Least Privilege.
  • e.g. Analyst agent cannot execute code; the Tester agent cannot update specs
• Policy as Prompt: Centralize agent roles in version control to enforce organizational coding standards, security checks, and compliance globally

Artifact-Driven Auditing

• Explicit artifacts (Specs, Designs, Architecture Document) guarantee traceability
• Auditing: Traces artifacts back to the LLM inference
• Pedagogy: Faculty can grade intermediate artifacts to assess a student’s systems thinking, not just the final compiled code

Harness Selection Criteria

• Agnostic: Must work with any LLM provider, locally hosted or frontier
• Observable: Clear trail of agent communication + intermediate artifacts
• Extensible: Allow custom agent roles
• Secure: Sandboxing and tool permissions
• Deterministic: precise control over generated artifacts and state transitions

Harness is the orchestration layer that enforces our guardrails, governance, and multi-agent workflows.

Why OpenCode?

• Harness of choice: good balance between features and configurability
• Overall loop remains non-deterministic - driven by a model capable of long-horizon reasoning
  • For complex tasks, expect this loop to run for hours at a time without human interaction
• Run in terminal or web browser; supports multiple projects and sessions

Pros

• Open source; no vendor lock-in
  • Use any provider, model, skills
• Highly customizable
  • Fully control your system prompts / agent roles
• Powerful permission model; containerize for added security
• Agents can spawn multiple sub-agents
  • n-level recursion and parallelization
• Multi-modal inputs: text, code, images!

Cons

• Non-deterministic loop
  • Controlling model decides task accomplishment, states, transitions
• Configuration: must edit opencode.json for certain settings
• Many features still in development
• Defining specific agent roles takes additional effort
  • Result of wanting to control your own agent roles
  • Can be automated with proper tooling

Agentic Software Engineering Workshop with OpenCode

Future Work

• Deterministic harness, e.g. OpenHands
  • Predictable control flow; agents orchestrated by a state machine
  • Precise control over intermediate artifacts: human-in-the-loop, agents-as-a-judge
• Visualize agents and artifacts with a unified dashboard
• Agent composition
  • Dynamically define agent roles and permissions
  • Organize agents into a hierarchy for any given task
  • Generalize beyond software engineering!

Resources

Open Source Harnesses

• OpenCode
• OpenHands
• Pi.dev
• Mistral Vibe

Open Weight Models

• Mistral Medium 3.5 - long-running tasks
• Muse Glimmer 30B - reasoning, planning
• Poolside - capable developer
• Google Gemma 4 - reviewer, tester

Use your local GPU

• Do this if you have a GPU with 24GB+ VRAM
• LM Studio - easiest setup
• Ollama - easy setup, models just work
• vLLM - complex, high performance

Hosted Models

• Free and paid models with privacy tradeoffs
• Mistral - ~50M tokens free per month
• OpenRouter - many free and paid models
• models.dev - model details
• Hugging Face - discover and download models