hollance/neural-engine

Project: hollance/neural-engine

Scope: A community‑maintained knowledge base about Apple’s Neural Engine (ANE) – the on‑device NPU used in recent iPhones and iPads.
Goal: Collect, organise, and share practical information that the author discovered by experimenting with Core ML models on Apple hardware.
Audience: iOS / macOS developers who want their Core ML models to run on the ANE, and anyone curious about the chip’s capabilities and limits.

The ANE can give orders‑of‑magnitude speed‑ups over CPU/GPU for supported models, but Apple provides no official documentation on how to make a model ANE‑compatible.
Developers often see their models fall back to the GPU or run slowly, leading to confusion and wasted effort. This repo aggregates the hard‑won lessons that would otherwise be scattered across forum posts and personal notes.

Section	What you’ll learn
Supported devices	Which iPhone/iPad models ship with an ANE.
Why care about the ANE?	Performance and power‑efficiency benefits of using the NPU.
Running on the ANE	Steps to get a Core ML model to execute on the ANE (model conversion, `MLModelConfiguration`, etc.).
Preventing ANE use	How to force a model onto CPU/GPU when the ANE is unsuitable.
Detecting ANE usage	Using `MLModelConfiguration` and OS logs to verify that the ANE is actually being used.
Programming the ANE directly	Current (as of the repo) possibilities – essentially none; the ANE is only reachable through Core ML.
ANE vs GPU	Architectural differences and why they are not interchangeable.
Precision (16‑bit?)	What numeric formats the ANE supports and how that affects model accuracy.
Unsupported Core ML layers	A curated list of operations that cause a fallback to CPU/GPU.
Using `os_log`	How to read system logs for warnings/errors about ANE execution.
Model surgery	Techniques for replacing unsupported layers (e.g., fusing, rewriting, or approximating them).
Internals	The author’s best‑guess description of the ANE’s internal dataflow based on reverse‑engineering.
Other weird issues	Edge‑case bugs and quirks observed in the wild.
Reverse‑engineering	Links to tools, scripts, and community efforts that try to probe the ANE’s behavior.

Read the docs – each markdown file in docs/ is self‑contained; start with why-care.md or running-on-ane.md.
Check your device – use docs/supported-devices.md to confirm the hardware you’re targeting has an ANE.
Validate – after deploying a Core ML model, follow the steps in is-model-using-ane.md to confirm it really ran on the NPU.
Contribute – if you discover a new unsupported layer or a workaround, open an issue or submit a pull request as the README encourages.

The information is empirical and may be outdated as Apple releases new OS versions or hardware revisions.
There is no official API for direct ANE programming; everything goes through Core ML.
Some sections are still “work‑in‑progress” and rely on community contributions.

This summary is based solely on the repository’s README; the repo itself contains the detailed markdown files referenced above.