Sovereign AI inference in Switzerland: the architecture

Sovereign AI inference is a property of the whole data flow, not of the model. An Apertus model hosted in Zurich is not sovereign if the traces land at a US monitoring service, the vector database sits in Ireland, or support from a third country reads the logs. An architecture is sovereign only when prompt, context, weights, logs and metadata never leave the same jurisdiction.

A great deal has been written about digital sovereignty. ipt, for instance, runs a page titled “Sovereign AI platforms on Kubernetes” and has worked the subject with the ETH AI Center. What is missing is the layer beneath: which providers exist, what they cost, what operating them looks like.

We work it through on the Swiss market, because prices and legal position there are publicly checkable. The architecture underneath is the same whether the sovereign cloud sits in Zurich, Frankfurt or Paris.

What sovereign AI inference means technically

Sovereignty can be checked against five artefacts produced by every inference request: model weights, prompt, retrieved context, generated response, and the telemetry about it. Each has a physical location and a legally entitled accessor, and those two are not the same thing.

Swiss law turns that into a concrete test. Art. 16 revDSG permits disclosure of personal data abroad only where the Federal Council has assessed the destination state as adequate.

The list is in Annex 1 of the Data Protection Ordinance (SR 235.11). The USA has been on it since 15 September 2024, but only for organisations certified under the Swiss-US Data Privacy Framework.

The second test runs through the provider itself. 18 U.S.C. § 2713, introduced with the CLOUD Act, obliges US providers to hand over stored data “regardless of whether such communication, record, or other information is located within or outside of the United States”.

A data centre in Zurich changes nothing about that if the operating company is subject to US corporate law. Which architecture decisions follow is in A revDSG-compliant AI architecture.

Which Swiss cloud providers offer sovereign AI inference

Four providers realistically come into question for production GPU inference in Switzerland, plus self-operation. Only one of them publishes complete prices.

One detail that counts in a sovereignty review: Exoscale’s RTX Pro 6000 instances run in DE-FRA-1, HR-ZAG-1 and CH-DK-2, and the 3080ti instances exclusively in AT-VIE-2. A Swiss provider does not automatically guarantee a Swiss zone (retrieved 19 August 2026).

Apertus 1.5: what the Swiss model can do today

Apertus 1.5 appeared on 24 July 2026 and is a different model from the September 2025 version. The context window is 262,144 tokens, a fourfold increase. The models now accept images; audio is marked experimental. Licence: Apache 2.0.

For operations, tool integration matters more than multimodality. The model card for Apertus-v1.5-70B documents a dedicated tool-call parser and an optional thinking mode, both drivable directly in vLLM. That makes Apertus usable for agent architectures, not only for chat.

It was developed by EPFL, ETH Zurich and CSCS on the Alps supercomputer in Lugano. For those who do not want to run it themselves, Infomaniak lists swiss-ai/Apertus-v1.5-70B as an API model, currently in beta.

Running Apertus yourself: Hugging Face, Ollama or Apertus Mini

There are three routes to self-operation, with very different degrees of production readiness.

Hugging Face plus vLLM is the production path. swiss-ai/Apertus-v1.5-70B and the 8B variant sit there under Apache 2.0 and load without a conversion step.

Ollama is the fastest route to a prompt and the slowest to production. There is no official Apertus build from swiss-ai; the widespread GGUF packages come from community packagers and carry the 2509 release, i.e. Apertus 1.0. Anyone wanting 1.5 locally quantises it themselves. For a laptop test that is fine; for a service under concurrent load it is not.

Apertus Mini often solves the actual problem. The official collection contains v1.1 distillates at 0.5B, 1.5B and 4B, plus quantisations for MLX and vLLM. For classification, extraction and routing a 4B model is usually enough, and it moves the GPU cost by an order of magnitude.

What inference on your own infrastructure costs

A model calculation, derived from published price lists retrieved on 19 August 2026. It applies to Apertus-v1.5-70B and compares two purchasing routes at the same level of sovereignty.

The model card gives --tensor-parallel-size 4 as the reference configuration at the full 262,144 tokens of context. With 72 billion parameters in BF16 that is 144 GB of weights alone, plus the KV cache. Four cards of 96 GB each are therefore the minimum, not generosity.

Self-operation, Exoscale dedicated inference on RTX Pro 6000, published price list:

• CHF 2.15277778 per GPU-hour, billed by the second
• 4 GPUs running continuously: 4 × 2.15277778 × 720 h = CHF 6,200 per month (720 hours, as Exoscale itself converts to monthly prices)
• Model storage 134 GiB (144 GB of weights) at CHF 0.0000275 per GiB-hour: CHF 2.65 per month

API purchase, Infomaniak AI Services, published price list: CHF 0.70 per million input tokens, CHF 2.50 per million output tokens for the same model.

Assumed request profile: 3,000 tokens of input (system prompt plus RAG context), 500 tokens of output. That comes to CHF 0.00335 per request, or CHF 0.96 per million mixed tokens.

The break-even sits at around 1.85 million requests per month, i.e. roughly 62,000 a day. Below that the API is cheaper. That corresponds to 6.5 billion tokens a month, or around 2,500 tokens per second of sustained throughput.

At 2,500 tokens per second, though, the four cards run flat out around the clock, and no operation runs like that. Planning headroom for load spikes and a second zone for failures doubles the GPU cost and with it the break-even: realistically three to four million requests a month.

Conversely, per-second billing helps: someone with load only during office hours who scales to zero pays CHF 1,894 for 220 hours a month rather than CHF 6,200, and the break-even falls to around 566,000 requests. The required peak capacity stays the same.

Not included: the cluster underneath (Exoscale SKS Pro at CHF 0.054995 per hour, around CHF 40 a month, in detail in what Kubernetes really costs in Switzerland) and your own time.

Recompute the break-even for your profile. It moves a lot with prompt caching, a smaller model, or a different input-output ratio. If you know your request profile, we will work it through together, with your numbers rather than our assumptions.

GPU scheduling on Kubernetes: DRA rather than device plugins

Dynamic Resource Allocation replaces the counting approach of “give me a GPU” with a requirement expressed over properties. The core of DRA has been GA since Kubernetes v1.34, with the API group resource.k8s.io/v1; the concept page in the docs now lists DRA as Kubernetes v1.35 [stable].

The old device plugin only knew nvidia.com/gpu: 4. Which four cards the scheduler picked was of no concern to it. With a 70B model using tensor parallelism that is not a detail: four 24 GB cards are not the same as four 96 GB cards, and the pod only dies when the weights load.

With DRA you express the requirement as a CEL expression against the attributes the driver publishes:

# Four GPUs of at least 80 GiB each, exclusively for one vLLM pod.
# Requires the DRA driver for NVIDIA GPUs (DeviceClass gpu.nvidia.com).
apiVersion: resource.k8s.io/v1
kind: ResourceClaimTemplate
metadata:
  name: apertus-70b-gpus
  namespace: inference
spec:
  spec:
    devices:
      requests:
        - name: gpu
          exactly:
            deviceClassName: gpu.nvidia.com
            allocationMode: ExactCount
            count: 4
            selectors:
              - cel:
                  expression: |
                    device.capacity['gpu.nvidia.com'].memory.isGreaterThan(quantity('80Gi'))

The pod references the template instead of asking for a number:

spec:
  resourceClaims:
    - name: gpu
      resourceClaimTemplateName: apertus-70b-gpus
  containers:
    - name: vllm
      image: vllm/vllm-openai:v0.27.1
      resources:
        claims:
          - name: gpu

The practical difference: a wrongly sized node fails at scheduling, not after an eight-minute model download. The DRA driver for NVIDIA GPUs publishes productName, architecture and cudaComputeCapability, among others, for this purpose.

Inference serving: what belongs between model and application

vLLM is the model server, not the platform. It does continuous batching, PagedAttention and an OpenAI-compatible endpoint. What it does not do: decide which of eight replicas gets a request.

A deployment fragment for Apertus. The tool parser and chat template come from the model card; the context window is halved, because otherwise the KV cache fills the GPU memory:

containers:
  - name: vllm
    image: vllm/vllm-openai:v0.27.1
    args:
      - --model=swiss-ai/Apertus-v1.5-70B
      - --served-model-name=apertus-70b
      - --tensor-parallel-size=4
      - --max-model-len=131072
      - --gpu-memory-utilization=0.90
      - --chat-template-content-format=string
      - --enable-auto-tool-choice
      - --tool-call-parser=apertus
    ports:
      - containerPort: 8000
    readinessProbe:
      httpGet: { path: /health, port: 8000 }

Routing belongs a layer higher. The Gateway API Inference Extension (v1.6.0 of 17 August 2026) brings the InferencePool and an endpoint picker that evaluates queue depth and KV cache utilisation. A round-robin load balancer, by contrast, hits a replica whose cache does not hold the prefix and pays for the prefill twice.

apiVersion: inference.networking.k8s.io/v1
kind: InferencePool
metadata:
  name: apertus-70b
spec:
  selector:
    matchLabels: { app: apertus-70b }
  targetPorts:
    - number: 8000
  endpointPickerRef:
    name: apertus-70b-epp
    port: { number: 9002 }
    failureMode: FailOpen

llm-d (v0.9.0 of 17 August 2026) combines both into disaggregated serving, i.e. prefill and decode on separate pods. For most installations of this size that is a size too big.

Where sovereignty actually breaks

The model is rarely the problem. The breaking points sit in the periphery that nobody perceives as an AI component and therefore nobody checks.

Monitoring SaaS is the most common breaking point. An OpenTelemetry trace around an LLM call carries attributes such as gen_ai.input.messages and gen_ai.output.messages, i.e. the full content of the conversation. Pointing the collector at a US provider exports the prompt, not just the latencies. That is not a misconfiguration but the default behaviour of common instrumentation. The data protection consequences hanging off it are in A revDSG-compliant AI architecture.

The vector database stores the cleartext alongside. Every RAG implementation puts the source text chunk as payload next to the vector, otherwise it cannot place it in the prompt. A managed vector database with region Ireland is therefore a copy of your document corpus outside Switzerland.

Support access is in the contract, not in the architecture diagram. Whether a provider delivers second-level support from a third country, and whether that team has read access to production systems, is a question for procurement. It is rarely asked.

The CDN sees every request before it reaches the origin. Putting a US CDN in front of the inference endpoint means the prompt is in cleartext at a provider subject to CLOUD Act access.

Then the quieter ones: the embedding service, often a different provider from the LLM service, and the object storage for model weights, whose bucket region nobody checks any more. Auditing these points does more for sovereignty than the choice of model. The same discipline was required for the distributed platform we built for onocoy.

When sovereign inference is the wrong answer

For the majority of the use cases we see, running your own GPUs is the more expensive and slower solution. Below roughly a million requests a month you pay a premium for sovereignty with no technical return.

An API provider with a clean processor agreement, a documented data location and a commitment on model training generally covers the revDSG requirements for business data. Infomaniak, for example, processes in Geneva and Zurich. That is in production in days, not weeks.

Self-operation earns its keep when one of these applies: volume is durably above the break-even; the data falls under professional secrecy under Art. 321 of the Swiss Criminal Code; or you need a pinned model version because a validation depends on it.

What we advise against: the halfway solution with your own cluster but monitoring, vector database and CDN at a US provider. That costs the full operational effort and still does not deliver sovereignty.

The decision matrix

The rows are not equally weighted. Almost always the second decides it: which class of data flows through the prompt. The rest is a question of cost after that.

Frequently asked

What Swiss alternatives to ChatGPT are there?

Apertus is the open Swiss language model from EPFL, ETH Zurich and CSCS, available in version 1.5 since 24 July 2026. It can be obtained through Infomaniak AI Services, the Swisscom Swiss AI Platform, Phoenix Technologies, or from Hugging Face for self-operation. Alongside it, Swiss providers run endpoints for open models from Mistral, Qwen and Google.

What does a self-hosted LLM cost compared with the API?

By our model calculation of 19 August 2026, four RTX Pro 6000 at Exoscale running continuously cost CHF 6,200 a month. The same load through the Infomaniak API costs CHF 0.00335 per request at 3,000 input and 500 output tokens. The arithmetic break-even is 1.85 million requests a month; with headroom for peaks and failures, closer to three to four million.

What hardware do I need for my own LLM?

For Apertus-v1.5-70B the model card gives four GPUs at the full context window: 72 billion parameters in BF16 are 144 GB of weights alone, plus the KV cache. The 8B model runs comfortably on one 48 GB card. Quantisation lowers the requirement but costs quality and has to be measured for the specific use case.

Does Swiss hosting protect against the US CLOUD Act?

Only if the operator is not subject to US law. 18 U.S.C. § 2713 obliges US providers to hand over data regardless of where it is stored. What decides is therefore the provider’s corporate structure, not the address of the data centre. A data centre in Zurich operated by a Swiss subsidiary of a US group remains within reach.

---

Let’s discuss the inference architecture. Thirty minutes with one of our engineers on your concrete case: which model, what volume, where the data sits, and where the periphery leaves the jurisdiction. No sales pitch, no preparation needed.

A slot is available here. If you would rather write first, you can reach us through contact. What we work on is under AI and machine learning and Kubernetes and infrastructure.