For most organizations, "AI analytics" has quietly become a synonym for "send your data to someone else's model." A user types a question, the question and a slice of the schema — sometimes the data itself — travel to a cloud endpoint, and an answer comes back. For a large share of the market that is an acceptable trade. For another large share, it is a non-starter.
If your environment cannot make outbound calls, no amount of model quality matters until the data-egress problem is solved first. This post is about how AI Insights 365 On-Prem Edition solves it: running the full AI feature set — natural-language-to-SQL, auto-report generation, chart and anomaly explanation — against a local LLM served by Ollama, with zero external network calls.
What "air-gapped" actually means
Air-gapped is often used loosely. In practice it means a network with no route to the public internet — physically isolated or controlled to the point that no production workload can reach an external host. There is no firewall exception to request, no allow-listed endpoint, no "just this one API." Anything that requires an outbound call simply does not run.
That constraint is the whole point. An air gap is a control you can reason about: if the data cannot leave, an entire category of exfiltration, third-party-processing, and residency risk disappears. The cost is that every capability you want must run inside the boundary — including, now, your AI.
It is worth separating two weaker postures from a true air gap:
- "On-prem, but calls a cloud model" — the application runs in your data center but inference happens in a vendor's cloud. Data still leaves.
- "Private cloud / dedicated tenancy" — better contractual isolation, but the network path off-prem still exists.
- True air-gapped — no outbound path at all. The model runs on your hardware.
Only the last one satisfies organizations whose controls assume the network boundary is real.
Why some organizations need zero external calls
This is not caution for its own sake. Several sectors operate under rules or threat models where sending data to an external model is either prohibited or unacceptable.
- Defense and government — classified and controlled-unclassified environments are frequently disconnected by design. An external API call is not a policy violation to be managed; it is impossible by construction.
- Healthcare — patient data carries strict handling obligations. Keeping protected information inside the boundary removes the need to extend trust, contracts, and audit scope to a model provider.
- Financial services — trading, risk, and customer-data systems often sit behind controls that forbid third-party processing of sensitive records.
- Critical infrastructure and industrial — operational networks are isolated from IT and the internet as a safety and security baseline.
The common thread is that these teams already paid for an air gap. They are not going to puncture it to get AI features. The question they ask is not "is the cloud model better?" but "can I get useful AI analytics without an external call at all?" The answer is yes.
There is also a quieter, organizational reason this matters. Every external dependency you add to a sensitive system has to be defended — in an architecture review, in an audit, in an incident post-mortem. A model that calls out to a vendor expands the set of questions a CISO has to answer: where does the data go, who can see it, how is it retained, what happens if the provider is breached. Keeping the model inside the boundary collapses all of those questions back to controls you already own and already audit. The simplest dependency to defend is the one that does not exist.
How local LLMs deliver the same features without leaving the network
The AI engine in AI Insights 365 is pluggable. The same features that run against a cloud model run against a local one — the platform talks to a provider abstraction, not to a specific vendor. In an air-gapped deployment, that provider is Ollama, serving a model on your own hardware. The model weights, the inference, and your data all stay inside the boundary.
Switching the engine is a configuration choice, not a different product:
Ai:Provider=Ollama
Ollama:Endpoint=http://localhost:11434
Ollama:Model=<your-model>
With that in place, the AI surface area works exactly as it does anywhere else.
Natural-language-to-SQL chat
A user asks a question in plain language over a connected datasource. The local model writes the SQL, the platform runs it, and the model explains the result. The schema and the question never leave the network — the model that reads them is sitting on a GPU in the same room. This is the feature most teams want first, because it turns a governed datasource into something a non-engineer can interrogate.
AI auto-report generation
Point the engine at a datasource, give it a short prompt, and it builds a dashboard — chart selection, layout, and narrative — without a human assembling it by hand. Report generation is the heaviest AI workload, but it is also the most latency-tolerant, which matters for hardware sizing (more on that below).
Chart and anomaly explanation
Any chart can be explained in two to four insight bullets. Anomaly explanation runs over z-score outliers and tells the reader, in words, what stands out and why it might matter. Both are short, interactive prompts that benefit from a fast local model.
Transform suggestions and reusable agents
The model can suggest transform steps while a user is shaping data, and teams can define reusable, named AI agents per workspace with their own system prompts. Those agents — and the prompts that define them — live entirely inside your environment.
The important architectural fact: none of these features behave differently because the model is local. The data path is shorter and stays inside your walls, but the capability is the same.
It also means the rest of the platform comes along unchanged. The 24 connectors that reach systems directly across your segmented networks, the 72+ chart types, scheduled reports, RBAC, AD/Entra SSO, encryption at rest, and backup/restore all operate the same way whether the model is in Azure or on a GPU down the hall. AI is one feature of a data platform, not a separate cloud service bolted on, and in an air-gapped deployment it inherits exactly the same controls as everything else.
The trade-offs, stated honestly
A local model is not free of cost; it relocates the cost from a per-call cloud bill to your hardware and operations. Being clear about this is the difference between a deployment that succeeds and one that disappoints its users.
Hardware
Local inference needs compute, and for interactive features it really wants a GPU. NL-to-SQL chat and chart explanations only feel good when the model generates quickly. CPU-only inference is fine for small models and light, batch-style work, but interactive use at any scale calls for GPU acceleration. VRAM is usually the binding constraint: the model and its context have to fit in GPU memory for good throughput.
Model size versus quality
Larger models generally write more reliable SQL and produce cleaner narrative explanations — and they cost more memory and run slower. There is a real knee in this curve, and finding it is the main tuning exercise in an air-gapped deployment.
| If you choose... | You get | You pay |
|---|---|---|
| A smaller local model | Fast, responsive interactions; modest VRAM | Weaker SQL on complex schemas |
| A mid-sized local model | Good balance for most workloads | Solid GPU required |
| A large local model | Best SQL and report quality | High VRAM, slower, costlier hardware |
A smaller model that fits comfortably in your GPU often beats a large model that spills out of VRAM and crawls. Right-sizing the model to the hardware — not maximizing the model — is the goal.
One useful technique: profile the work by feature. NL-to-SQL over a small set of well-modeled datasources is often within reach of a modest model, because the model is pattern-matching a question to a schema rather than reasoning from scratch. Auto-report narrative and free-form explanation tend to expose model weaknesses sooner. Test the hardest feature you actually intend to use, not the easiest, before you commit to a model size.
Concurrency
One person asking occasional questions is a very different load from a workspace of analysts running interactive chat all day. Size GPU headroom for peak interactive demand, and lean on the fact that report generation tolerates latency to let one capable GPU serve both interactive and batch work when volume is modest.
Practical guidance to adopt
A pragmatic rollout looks like this:
- Validate features first, then go local. It is reasonable to evaluate the AI features against a cloud model during a proof of concept to confirm they fit your workflows, then switch the provider to Ollama before production. The features are identical; only the backend changes, so nothing you validate is wasted.
- Pick a mid-sized model and test on real schemas. Generic benchmarks do not tell you whether a model writes correct SQL against your datasources. Run representative NL-to-SQL questions and auto-reports against your actual connectors and judge the output.
- Size hardware to the chosen model, with concurrency headroom. Fit the model in VRAM, then add margin for the number of simultaneous interactive users you expect at peak.
- Transfer models through approved channels. In a true air gap, bring model artifacts in via your data-diode or approved offline media process rather than pulling them over a network, then load them locally with Ollama.
- Set per-organization token budgets. The platform enforces a per-organization monthly token budget as a soft gate that resets monthly, with usage tracked per user and per organization. Use it to keep a single workspace from monopolizing local GPU throughput and to attribute consumption during capacity reviews.
- Verify end to end. Confirm the connection, run an NL-to-SQL query, request a chart explanation, generate a report, and check that usage is being metered. If the connection is healthy but output is weak, the model is usually too small for your schemas — step up a size.
The bigger picture
The market has spent two years equating AI capability with cloud scale. For a lot of teams that equation simply does not apply, and pretending otherwise pushes them toward shadow tools or no AI at all. The more useful framing is that AI analytics is a feature of your data platform — and like every other feature in a regulated environment, it should run inside your boundary, under your controls, on your hardware.
AI Insights 365 On-Prem treats it exactly that way. The same NL-to-SQL chat, auto-reports, and explanations that run against a cloud model run against a local one served by Ollama, with no external calls, governed by the same RBAC, encryption at rest, and token budgets as the rest of the platform. For organizations that already invested in an air gap, that is the point: useful AI, and the boundary stays intact.