eleiton/ollama-intel-arc

Make use of Intel Arc Series GPU to Run Ollama, StableDiffusion, Whisper and Open WebUI, for image generation, speech recognition and interaction with Large Language Models (LLM).

What it solves

This project provides a streamlined, Docker-based deployment for running various AI models on Intel Arc GPUs. It eliminates the complexity of manual setup by bundling together a user-friendly interface (Open WebUI) and specialized backends for Large Language Models (Ollama), image generation (Stable Diffusion via ComfyUI or SD.Next), and speech recognition (OpenAI Whisper).

How it works

The project uses Docker Compose to orchestrate several containers:

  • Ollama: Runs with either a native SYCL backend (compiled from source using Intel oneAPI) or a Vulkan backend to leverage Intel GPU acceleration for LLMs.
  • Open WebUI: Acts as the primary user interface for interacting with Ollama and triggering image generation.
  • Image Generation: Uses ComfyUI or SD.Next containers based on the Intel Extension for PyTorch to run Stable Diffusion.
  • Whisper: A dedicated container using the Intel Extension for PyTorch for automatic speech recognition (ASR) tasks.

Who it’s for

Users with Intel Arc GPUs (including integrated graphics like Meteor Lake) who want to run a local AI stack including text, image, and speech models without complex manual installation.

Highlights

  • Native Intel GPU Support: Uses SYCL and Vulkan backends for Ollama to avoid dependency on IPEX-LLM.
  • Multi-Modal Stack: Combines LLMs, Stable Diffusion, and Whisper in a single deployment.
  • Customizable Runtime: Allows tuning of context length, KV-cache type, and flash attention via a .env file.
  • Flexible Deployment: Offers both a high-performance SYCL build and a simpler Vulkan-based setup.

Related

  • Project
    jim60105/docker-whisperX

    A collection of optimized Docker images for WhisperX, providing pre-bundled ASR models for fast and efficient speech-to-text transcription with word-level timestamps.

  • Project
    eugr/spark-vllm-docker

    A Docker-based deployment framework for vLLM optimized for NVIDIA DGX Spark, providing tools for single- and multi-node cluster setup, model distribution, and hardware-specific performance tuning.

  • Project
    docker/model-runner

    A tool for managing and deploying AI models via Docker, streamlining the process of pulling and serving LLMs using backends like llama.cpp and vLLM.

  • Project
    n8n-io/self-hosted-ai-starter-kit

    A Docker Compose template that bundles n8n, Ollama, and Qdrant to quickly set up a self-hosted, local AI development environment for building AI agents and workflows.

  • Project
    sandrohanea/whisper.net

    Dotnet bindings for OpenAI Whisper via whisper.cpp, enabling local speech-to-text and translation in .NET applications with extensive hardware acceleration support.