jeremyipark/vision-demos
Fun real-world computer vision demos!
What this project is
vision-demos is a collection of small, real-world computer-vision demo programs. Each one takes a video clip and uses AI models to analyze what's happening in it — mostly human movement and sports. The demos are independent, each with its own folder and README.
The demos
| Demo | What it does | AI models used |
|---|---|---|
| dance_sync | Compares two dancers doing the same choreography and gives a similarity score. | vitpose-plus-large (pose detection) |
| chin_ups | Counts chin-up repetitions in a clip and times the up and down phases of each rep. | vitpose-plus-large |
| pull_ups | Counts pull-up reps, times each pull, and marks each rep as "full" or "partial" range of motion, compared against the athlete's own measured dead hang. | vitpose-plus-large |
| push_ups | Counts push-up reps from a side-on video, times each push, and checks each rep for full range of motion and a straight back. | vitpose-plus-large |
| rock_climbing | Identifies the holds on a bouldering wall, shows which holds the climber used and in what order, and compares different attempts on the same route. | sam3.1 (image segmentation) + vitpose-plus-large |
| rock_climbing_3d | Same idea as rock_climbing, but from a video recorded with depth (iPhone LiDAR). It reconstructs the wall, holds, and climber in 3D and measures the route in meters. | sam3.1 + vitpose-plus-large |
| running | Measures a runner's cadence (steps per minute), times every foot strike, and averages the knee angle at the moment of contact. | vitpose-plus-large |
| deadlift | Counts deadlift reps and classifies each rep's back as straight or rounded, with a probability for the call. | sam3.1 + vitpose-plus-large + gemma-4-26b-a4b-it (language model) |
| basketball | Determines whether each shot is a make or a miss by tracking the ball through the rim, and plots every shot on an overhead court diagram with the field-goal percentage. | sam3.1 + vitpose-plus-large |
How to run it
- Each demo folder has its own README with setup and instructions.
- There's also a single browser app that lets you try most demos without per-project setup:
- Create the environment:
conda env create -f environment.yml(orpip install -r requirements.txt) - Activate it and run
python app.py - Open
http://127.0.0.1:7860in your browser
- Create the environment:
- In the app you upload a clip, trim it, and analyze it.
- No API key is needed — there's an anonymous, rate-limited tier. You can paste a VLM Run key or set the
VLMRUN_API_KEYenvironment variable for higher limits. - Two demos are not in the browser app:
rock_climbing_3d(because it needs a LiDAR capture) andpush_ups. - ffmpeg is required.
License
Apache-2.0.
Notes
- The README includes demo videos and screenshots for most projects (YouTube links for dance_sync, pull_ups, basketball, deadlift, and rock_climbing_3d; static images for the others).
- The models are accessed through the VLM Run gateway, which is why an API key is optional but recommended for higher usage limits.
Related
- Project
- Project
- Project
- Project
- Project