Orckai
Self-Hosted AI Agent Platforms: What Actually Has to Run Inside Your Network
“Runs in Docker” and “operates inside your perimeter” are different claims, and only one of them satisfies a security review. Here is what a self-hosted AI agent platform really involves, what still crosses the boundary, and how to check before you sign.
Why the question comes up at all
Nobody self-hosts for fun. The reasons are almost always external: a regulator, a customer contract, a data-residency rule, or a security team that will not approve sending internal documents to a vendor's cloud. Occasionally it is cost at scale, and occasionally it is a strategic dislike of lock-in, but the compliance reason dominates.
That matters, because it changes what “good” looks like. If you are self-hosting to satisfy a control, the platform has to satisfy the control — not merely be installable on your hardware. Those are different bars, and plenty of tools clear the second while quietly failing the first.
The distinction worth holding on to: “runs in Docker” is not the same as “operates inside your perimeter.” A platform that installs locally but calls a vendor API for embeddings, licensing, telemetry, or search has moved the binary, not the trust boundary. Ask specifically which calls leave the network and what is in them.
What actually has to run inside
An AI agent platform is more than one process. Before comparing options, know what you are agreeing to operate. In Orckai's case, self-hosting means these components run on your infrastructure:
- Application services — an API gateway that handles auth routing, an authentication service with RBAC, a core service running agents and the workflow engine, and the web console
- PostgreSQL with pgvector — both your operational data and your document embeddings, so nothing is indexed off-site
- Redis — cache and the queue that drives asynchronous workflow execution with retries
- Object storage — uploaded documents and workflow artefacts, either on the filesystem or an S3-compatible service such as MinIO
- A reverse proxy for TLS termination, using your certificates
- Generated MCP containers — the connections to your databases, which stay on your network by construction
- Monitoring — Prometheus scraping metrics with Grafana dashboards, so you can see queue depth and request latency without a vendor portal
The deployment is Docker Compose, and the services bind to localhost with the proxy in front. That is a deliberately boring architecture, which is the correct kind of architecture for something your security team has to approve.
The one call that still leaves
Unless you run a model locally, inference is an outbound HTTPS call to your provider with your own key, under your own contract and their data-handling terms. Everything else — documents, embeddings, conversations, audit records, credentials — stays inside.
This is the point most self-hosting conversations skip, and it is the one your security review will not. Be precise about it: what is sent is the prompt and the retrieved context needed to answer, and the commercial terms are between you and the model provider. If even that is unacceptable, the remaining path is a locally hosted model, which Orckai supports through an OpenAI-compatible endpoint. Expect to trade some answer quality for it, and to own a GPU estate.
The checklist for evaluating any self-hosted platform
Run these questions past any candidate, including ours. The ones that cause visible discomfort are the ones worth pressing.
| Question | Why it matters |
|---|---|
| Which outbound calls does it make? | Embeddings, licence checks, telemetry, and update pings are the usual surprises. Get the list in writing, then verify it with egress logging. |
| Does it work fully offline? | Some platforms degrade quietly when they cannot reach home. Test it with egress blocked, not just with the firewall in mind. |
| Where do the embeddings live? | A local install that indexes your documents into a hosted vector service has moved the data anyway. |
| How are secrets handled? | Provider keys and database credentials should be encrypted at rest and never appear in prompts, logs, or workflow definitions. |
| Does SSO integrate with your IdP? | Self-hosting and a separate user directory is a bad combination that surfaces at the first leaver. |
| What is the upgrade path? | Pinned image versions, reversible database migrations, and a documented rollback. Ask what happens if an upgrade fails at 2am. |
| What is the backup unit? | You need to know exactly which volumes and which database constitute a restorable system, and to have tested a restore. |
| What does support cover? | On someone else’s infrastructure the vendor cannot see your logs. Establish who diagnoses what before you need to know. |
What self-hosting costs you
It is worth being unsentimental about the trade. Self-hosting buys control and pays for it in ownership.
- You own uptime. There is no status page to point at when something breaks on a Friday evening.
- You own upgrades. Security patches arrive on the vendor's schedule and are applied on yours, and the gap between those two is your exposure.
- You own capacity. Queue depth, database size, vector index memory, and disk growth all become someone's job.
- You own the diagnosis. Support can advise, but it cannot read logs it has no access to.
For an organisation with a platform team, this is routine — it is what they already do for every other internal system. For a five-person team with no infrastructure practice, the hosted option is usually the honest recommendation even when self-hosting is technically available.
A sensible deployment plan
- Start on a single host. Docker Compose on one adequately sized VM handles far more than most teams expect. Distribute later, on evidence.
- Put the database on managed infrastructure if you can — a managed PostgreSQL with pgvector removes the backup and failover work that causes the most pain.
- Terminate TLS at your existing proxy with your existing certificate management, rather than introducing a second scheme.
- Block egress except to the model provider, then run a real workload. What breaks tells you what the platform was quietly calling.
- Wire monitoring on day one. Queue depth and execution failure rate are the two numbers that predict trouble.
- Test a restore before go-live, not after the first incident.
- Pin versions and stage upgrades. A non-production instance sharing the configuration is cheap insurance.
A reasonable middle path: evaluate on the hosted platform, deploy the production instance yourself. The evaluation phase is where you need speed and iteration, and it usually runs on synthetic or already-public data. Once the use case is settled and real data is involved, move it inside. Orckai is the same platform in both places, so nothing is rebuilt.
Where Orckai fits
Orckai was built for deployment on customer infrastructure from the start, not retrofitted for it. The same workflow engine, agents, knowledge base, widget, and generated MCP connections run in both places, with the same multi-tenancy, RBAC, and audit log. Self-hosting is part of the Enterprise plan; the practical checklist above applies whichever platform you choose.
Frequently Asked Questions
What does self-hosting an AI agent platform actually include?
The application services, the database holding both operational data and document embeddings, a cache and job queue, object storage for uploaded files, a reverse proxy for TLS, any generated database connectors, and monitoring. For Orckai that is a Docker Compose deployment with services bound to localhost behind a proxy.
Does self-hosting mean no data leaves my network?
Not quite, unless you also host the model. Documents, embeddings, conversations, audit records, and credentials stay inside; inference is an outbound HTTPS call to your model provider using your own key and contract. If even that is unacceptable, a locally hosted model behind an OpenAI-compatible endpoint keeps everything internal, at some cost in answer quality and a GPU estate to run.
Can it run fully air-gapped?
With a locally hosted model, yes in principle — but treat it as a project rather than a configuration flag. Test it properly: block egress, run a real workload, and see what fails. That test is the single most informative thing you can do when evaluating any platform’s self-hosting claims.
What hardware do I need?
For the platform itself, far less than teams expect — a single adequately sized VM running Docker Compose handles a substantial workload, since the heavy computation happens at your model provider. If you host models locally that changes entirely, and GPU capacity becomes the dominant cost.
Is self-hosted the same product as the hosted version?
For Orckai, yes — the same workflow engine, agents, knowledge base, widget, MCP generation, multi-tenancy, RBAC, and audit log. Self-hosting is part of the Enterprise plan. Not every vendor works this way, so it is worth asking directly whether the on-premises build lags the cloud one.
Should we self-host if we do not have a platform team?
Usually not, even when it is available. Self-hosting transfers uptime, upgrades, capacity planning, and first-line diagnosis to you. If no one owns that, the hosted option is the more honest choice — unless a compliance requirement removes the decision, in which case budget for the operational work explicitly.