IBM/aihwkit

IBM Analog Hardware Acceleration Kit

What it solves

IBM Analog Hardware Acceleration Kit (aihwkit) addresses the "von Neumann bottleneck"—the efficiency loss caused by moving data between separate memory and computation units. It enables the exploration and simulation of Analog In-Memory Computing (AIMC) hardware, where computation happens directly where data is stored, significantly increasing speed and energy efficiency for AI workloads.

How it works

The toolkit provides a high-performance C++ simulator (CUDA-capable) that models the physical characteristics of analog devices (like phase-change memory or RRAM) and crossbar configurations. It integrates directly with PyTorch, allowing users to replace standard neural network layers with analog versions (fully connected, convolution, LSTM) and use specialized analog optimizers (like AnalogSGD).

Who it’s for

AI researchers and hardware engineers who want to simulate analog AI hardware, perform hardware-aware training to make models more robust to physical noise and non-idealities, or explore material specifications for in-memory computing devices.

Highlights

  • PyTorch Integration: Seamlessly use analog neural network modules and training workflows within PyTorch.
  • Hardware-Aware Training: Includes noise and non-idealities in the forward pass to ensure models work on real analog hardware.
  • High-Performance Simulator: CUDA-capable C++ simulator for modeling device fluctuations, ADC/DAC discretization, and stochastic update pulse trains.
  • Model Conversion: Utility to automatically convert pre-trained standard models into equivalent analog models.
  • Broad Hardware Support: Supports NVIDIA GPUs via CUDA and AMD GPUs via ROCm/HIP.

Related

  • Project
    qualcomm/aimet

    AIMET is Qualcomm’s open‑source toolkit for quantizing and compressing PyTorch/ONNX models, enabling fast, low‑memory inference on edge devices with minimal accuracy loss.

  • Project
    pytorch/benchmark

    PyTorch Benchmarks is a collection of standardized, lightweight versions of popular deep‑learning models (BERT, ResNet, Stable Diffusion, etc.) that can be run to measure the performance of different PyTorch builds, CUDA versions, and back‑ends. It offers a uniform API, install scripts, and multiple runners (simple test, pytest‑benchmark, custom userbenchmark) plus utilities for low‑noise tuning on AWS g4dn.metal instances.

  • Dispatch
    Hugging Face Accelerate: Running Large Models with PyTorch

    Hugging Face Accelerate enables the execution of massive AI models on consumer hardware by leveraging PyTorch's meta device and sharded checkpoints to manage memory across GPUs, CPU RAM, and disk.

  • Project
    DashAISoftware/dashAI

    dashAI is a cross‑platform graphical toolbox for training, evaluating and deploying modern AI models (especially LLMs). It offers ready‑to‑run installers, a PyPI package, and Docker images, supports CPU‑only as well as CUDA/ROCm GPU acceleration, and includes experiment management, dataset handling, and database migrations.

  • Project
    SYSTRAN/faster-whisper

    faster-whisper is a Python library that re‑implements OpenAI’s Whisper speech‑to‑text model using the high‑performance CTranslate2 inference engine. It delivers up to 4× faster transcription with similar accuracy, supports CPU, GPU and 8‑bit quantisation, and offers features like word‑level timestamps, VAD filtering and batched inference. Install via pip, pick a model size (e.g., "large‑v3"), create a WhisperModel with the desired device/precision, and call `transcribe`. Ideal for live captioning, batch processing, or any project needing Whisper‑quality transcription with lower latency and memory use.