The Private AI Services Container: Vector Embeddings That Never Leave Your Network

Who this is for: anyone whose security team has ever said “you can build semantic search, but nothing sensitive leaves this network to hit a third-party embeddings API,” and left you stuck doing keyword search instead.

A healthcare client I worked with a while back wanted vector search over patient intake notes, the kind of thing that makes a support tool actually useful instead of just a search box that only matches exact words. The blocker wasn’t the database, it was that every practical embedding model at the time lived behind a public API, and sending PHI to a third-party endpoint to get a vector back was a non-starter for their compliance team. We shelved the vector search idea and shipped keyword search instead, which is exactly the kind of compromise nobody’s happy with. Oracle AI Database 26ai’s Private AI Services Container is the piece that would have unblocked that project: it generates embeddings and builds vector indexes without your data ever leaving your own network.

Why This Matters

Oracle AI Vector Search has been able to generate embeddings and build HNSW indexes for a while, but the practical options were to run the model inside the database itself (using database compute for what’s fundamentally a batch inference workload) or call out to a public provider like OpenAI, Cohere, or Google AI over the internet. Both of those have real costs: in-database inference competes with your database for CPU and memory, and calling a public API means your source text crosses your network boundary to get turned into a vector.

The Private AI Services Container is a third option: a lightweight, containerized web service you run yourself, in your own data center or on your own cloud compute, that does the embedding generation and index building work instead of the database. It offers two services. The vector embedding service generates text and image embeddings using Oracle ONNX Pipeline format models (the same models you could run inside the database, produced by OML4Py), and stores the resulting vectors directly in your Oracle AI Database. The vector index service offloads HNSW vector index creation to an NVIDIA GPU on the container’s host, which is dramatically faster than building large indexes on CPU. Internet access isn’t required for either, and per Oracle’s own documentation, requests to the container are stateless: user data isn’t stored, only processed transiently.

Diagram comparing embedding generation via a public provider, where text crosses the public internet to reach an external API, versus the Private AI Services Container, where embeddings and HNSW index builds happen entirely inside your own network — Same DBMS_VECTOR interface either way. The difference is entirely in where the URL points.

Setting it up

The container exposes a REST interface, and the database talks to it through the existing DBMS_VECTOR package, the same one used for embedding generation with any other provider. You start by creating a credential that stores how to authenticate to your container:

			
DECLARE
  jo json_object_t;
BEGIN
  jo := json_object_t();
  jo.put('access_token', '<your-container-access-token>');
  DBMS_VECTOR.CREATE_CREDENTIAL(
    credential_name => 'PRIVATEAI_CRED',
    params          => json(jo.to_string));
END;
/

		

Once the credential exists, generating an embedding is a single call to UTL_TO_EMBEDDING, pointed at your container’s URL instead of a public provider’s:

			
DECLARE
  input clob := 'Patient reports intermittent chest discomfort during exertion.';
  v vector;
BEGIN
  v := DBMS_VECTOR.UTL_TO_EMBEDDING(
    input,
    json('{"provider": "privateai",
           "url": "https://your-container-host:8443/v1/embeddings",
           "credential_name": "PRIVATEAI_CRED",
           "model": "all-MiniLM-L12-v2" }'));
  DBMS_OUTPUT.PUT_LINE(vector_serialize(v));
END;
/

		

Nothing about the call site looks different from using OpenAI or Cohere as the provider, the difference is entirely in where url points. That’s a deliberate design choice: your existing PL/SQL that generates embeddings doesn’t need restructuring, you’re swapping a provider and a credential, not rewriting application logic.

Offloading the index build too

The same container can also take over HNSW index creation, which is normally CPU and memory-intensive on large vector sets. You point CREATE VECTOR INDEX at the container’s REST endpoint and API key, and the graph gets built on the container’s GPU instead of your database server:

			
CREATE VECTOR INDEX notes_hnsw_idx ON patient_notes (embedding)
  ORGANIZATION INMEMORY NEIGHBOR GRAPH
  DISTANCE COSINE
  PARAMETERS (TYPE HNSW, NEIGHBORS 32, EFCONSTRUCTION 200)
  WITH TARGET ACCURACY 95;

		

The GPU-offload parameters for the container’s endpoint and key attach to this same DDL. The point isn’t the exact index parameters, which you’d tune for your own accuracy and latency targets, it’s that the expensive part of building a large HNSW graph no longer has to compete with your database’s own workload for CPU, and it finishes considerably faster on a GPU built for exactly this kind of parallel work.

Where This Requires Care

Running the container yourself means you own its lifecycle: patching, certificate management for TLS, and access control lists on the database side that permit outbound calls to the container’s host and port. Oracle’s documentation defines three logical roles around the container (Container Admin, Model Creator, Inference Client), and even if one person holds all three in a small deployment, it’s worth actually assigning them deliberately rather than letting everyone with database access also have container admin rights.

The model support is narrower than “any embedding model you like.” Only Oracle ONNX Pipeline formatted models, produced specifically by OML4Py 2.1.1, are supported for deployment in the container as of this release, covering text embeddings (ONNX_TXT) and image embeddings (ONNX_IMG). If your team has standardized on a model that isn’t packaged this way, converting it is a step you’ll need to plan for before this becomes a drop-in replacement for a public embedding API.

And “stateless, data not stored” describes the container’s own behavior, it doesn’t substitute for your organization’s actual data governance review. If compliance sign-off was the blocker before, get the container’s specific data-handling guarantees in front of the same reviewers rather than assuming “runs in our data center” is self-evidently sufficient for whatever regulation you’re working under.

Quick Reference

New in Oracle AI Database 26ai: the Private AI Services Container, a self-hosted, lightweight web service that offloads vector embedding generation and HNSW vector index creation outside the database.
Runs in your own data center or your own cloud compute, no internet access required; Oracle states requests are stateless and user data isn’t stored.
Embedding generation goes through the existing DBMS_VECTOR.UTL_TO_EMBEDDING / UTL_TO_EMBEDDINGS procedures, just pointed at your container’s URL and credential instead of a public provider.
Index creation offloads to an NVIDIA GPU on the container host via parameters on the standard CREATE VECTOR INDEX DDL.
Only Oracle ONNX Pipeline format models (produced by OML4Py 2.1.1) are supported, covering text (ONNX_TXT) and image (ONNX_IMG) embeddings.
You own the container’s operations: patching, TLS certificates, and database ACLs permitting outbound access to it, plus the Container Admin / Model Creator / Inference Client roles around it.

My Take

This is one of the more directly useful pieces of the 26ai AI story for anyone outside a green-field startup. Plenty of organizations have wanted vector search for a year or two now and have been stuck exactly where that healthcare project was: the database technology was ready, but the embedding step forced a choice between burning database compute or sending data somewhere your compliance team wouldn’t sign off on. A self-hosted container that speaks the same DBMS_VECTOR interface as any other provider removes that tradeoff without asking you to rewrite anything. I’d treat the ONNX model requirement as the first thing to check against whatever embedding model your team already has planned, since that’s the one piece of this that isn’t just configuration.