Netomi scaling agentic systems into the enterprise – lessons from OpenAI

TL;DR

Netomi announced a production‑grade agentic platform that combines GPT‑4.1 for low‑latency tool use with GPT‑5.2 for deep multi‑step planning, demonstrating sub‑3‑second response times, 98 % intent‑classification accuracy, and built‑in governance at enterprise scale.

Lesson 1 – Build for real‑world complexity, not idealized flows

Conclusion: Enterprise AI agents must handle ambiguous, multi‑system requests, so Netomi places OpenAI models at the core of a governed orchestration pipeline that can persist state, enforce tool use, and plan across steps.

  • Netomi’s Agentic OS routes requests through GPT‑4.1 for fast reasoning and tool‑calling, and through GPT‑5.2 when deeper planning is required.
  • The platform follows OpenAI‑recommended prompting patterns:
    • Persistence reminders keep GPT‑5.2’s reasoning across long workflows.
    • Explicit tool‑use expectations suppress hallucinations by forcing GPT‑4.1 to call authoritative APIs.
    • Structured planning leverages GPT‑5.2 to outline and execute multi‑step tasks.
    • Agent‑driven rich‑media decisions let GPT‑5.2 signal when a tool should return images, videos, or forms.
  • In airline use‑cases, a single customer query may involve fare‑rule checks, loyalty‑benefit calculations, ticket changes, and flight‑operations coordination, illustrating the need for situational awareness and a context‑led ensemble architecture.

“In airlines, context changes by the minute. AI has to reason about the scene the customer is in—not just execute a siloed task,” – Puneet Mehta, CEO, Netomi.

Lesson 2 – Parallelize everything to meet enterprise latency expectations

Conclusion: To earn user trust under bursty load, Netomi executes model inference and tool calls concurrently, keeping end‑to‑end latency below critical thresholds.

  • Traditional pipelines are sequential (classify → retrieve → validate → call tools → generate). Netomi’s concurrency framework overlaps these stages, exploiting GPT‑4.1’s fast time‑to‑first‑token and stable streaming tool calls.
  • GPT‑5.2 is invoked only for the portions of a workflow that need deep reasoning, preventing it from becoming a latency bottleneck.
  • During DraftKings’ peak events, the system sustained sub‑3‑second responses while handling >40,000 concurrent requests per second, maintaining 98 % intent classification accuracy across accounts, payments, knowledge lookups, and regulatory checks.

“AI is central and critical to how we support customers in the moments that matter most.” – Paul Liberman, Co‑Founder & President of Operations, DraftKings.

Lesson 3 – Make governance an intrinsic part of the runtime

Conclusion: Trustworthy enterprise AI requires governance baked into the execution layer, so Netomi’s runtime automatically validates, enforces policies, and falls back safely when uncertainty arises.

  • Schema validation checks every tool call against OpenAPI contracts before execution.
  • Policy enforcement applies topic filters, brand restrictions, and compliance rules inline during reasoning.
  • PII protection detects and masks sensitive data during preprocessing and response generation.
  • Deterministic fallback routes ambiguous intents or low‑confidence predictions to pre‑approved safe behaviors.
  • Runtime observability exposes token traces, reasoning steps, and tool‑chain logs for real‑time debugging and auditability.
  • In dental‑insurance deployments processing ~2 million provider requests annually, the governance layer prevented regulatory risk during high‑volume open‑enrollment periods.

“We built the system so that if the agent ever reaches uncertainty, it knows exactly how to back off safely.” – Puneet Mehta, CEO, Netomi.

Blueprint for enterprise‑ready agentic systems

Conclusion: Combining OpenAI’s GPT‑4.1 and GPT‑5.2 with a governed, parallel execution engine yields a production‑grade agentic stack that satisfies Fortune 500 requirements for speed, accuracy, and compliance.

  • Speed: Sub‑3‑second latency under extreme load.
  • Accuracy: 98 % intent classification at scale.
  • Governance: Built‑in schema validation, policy enforcement, PII protection, deterministic fallback, and full observability.
  • Scalability: Handles >40 k concurrent requests per second across domains such as airlines, gaming, and insurance.

These principles provide a repeatable roadmap for any organization seeking to move from prototype chatbots to reliable, enterprise‑grade AI agents.

Results at a glance

  • Sub‑3‑second response times during high‑traffic events.
  • 98 % intent classification accuracy at scale.
  • Support for traffic spikes exceeding 40 000 concurrent requests per second.
  • Governance embedded in the runtime with deterministic fallback and policy enforcement.
  • Successful deployments for Fortune 500 customers including United Airlines and DraftKings.

Sources