otwld/ollama-helm

What it is

ollama‑helm is a Helm chart that makes it easy to run the Ollama LLM server on a Kubernetes cluster. It packages all the Kubernetes objects (Deployment/StatefulSet, Service, Ingress, optional Knative Service, PVCs, etc.) and exposes a set of configurable values so you can spin up Ollama with CPU‑only or GPU acceleration, pre‑pull models, expose it via Ingress or Gateway‑API, and even bootstrap custom model builds.

Who it’s for

• Teams that already use Kubernetes and want to host LLMs locally without managing raw manifests.
• Users who need GPU support (NVIDIA or AMD) and want the chart to handle the extra flags.
• Anyone who wants to integrate Ollama with other cloud‑native tools (Knative, autoscaling, external‑gateway, etc.).

Core features

Feature	How the chart supports it
CPU / GPU deployment	ollama.gpu.enabled toggles GPU mode; type selects nvidia or amd; number sets how many GPUs to request.
Model pre‑loading	ollama.models.pull lists models to ollama pull at container start.
Custom model builds	ollama.models.create lets you supply a Docker‑like template (e.g., FROM llama3.1 …) and ollama.models.run runs the built model after creation.
Ingress / Gateway‑API	Optional ingress.enabled or gateway.enabled sections generate an Ingress resource or a Gateway‑API Gateway/HttpRoute.
Knative support	When knative.enabled=true the chart creates a Knative Service instead of a classic Deployment, and a separate bootstrap Job for model download/build.
Autoscaling	Standard HPA fields (autoscaling.enabled, minReplicas, maxReplicas, targetCPUUtilizationPercentage).
Persistence	persistentVolume.enabled creates a PVC; you can also attach an existing claim via persistentVolume.existingClaim.
Fine‑grained pod control	Options for affinity, hostNetwork, hostIPC, hostPID, extra env vars, extra args, and arbitrary extra Kubernetes objects.

Quick start (CPU‑only)

helm repo add otwld https://helm.otwld.com/
helm repo update
helm install ollama otwld/ollama \
  --namespace ollama --create-namespace

The chart will deploy Ollama with a default configuration and expose it on a ClusterIP service.

Quick start (GPU + model pull)

Create a values.yaml:

ollama:
  gpu:
    enabled: true
    type: nvidia
    number: 1
  models:
    pull:
      - mistral
      - llama2

Then install:

helm install ollama otwld/ollama \
  --namespace ollama --create-namespace \
  -f values.yaml

The pod will request one NVIDIA GPU and download the two models at start‑up.

Updating

1. Check the upstream Ollama release notes for breaking changes.
2. Adjust your values.yaml if needed.
3. Run:

   helm repo update
   helm upgrade ollama otwld/ollama -n ollama -f values.yaml
   

   The chart will create a new bootstrap Job if you changed any model‑related values.

Removing

helm delete ollama -n ollama

All resources created by the chart (including PVCs, unless you set persistentVolume.retain) are deleted.

Where to go next

• Read the official Ollama docs (API, client libraries, LangChain integrations) linked in the README.
• Explore the values.yaml in the repo for the full list of tunable options.
• If you need TLS, enable gateway.listener.tls and provide certificate references.
• For serverless‑style scaling, enable knative.enabled and follow the Knative feature‑flag instructions.

---

All information above is taken directly from the repository’s README and chart values; no additional features are assumed.