jeremyipark/vision-demos

Fun real-world computer vision demos!

What this project is

vision-demos is a collection of small, real-world computer-vision demo programs. Each one takes a video clip and uses AI models to analyze what's happening in it — mostly human movement and sports. The demos are independent, each with its own folder and README.

The demos

Demo What it does AI models used
dance_sync Compares two dancers doing the same choreography and gives a similarity score. vitpose-plus-large (pose detection)
chin_ups Counts chin-up repetitions in a clip and times the up and down phases of each rep. vitpose-plus-large
pull_ups Counts pull-up reps, times each pull, and marks each rep as "full" or "partial" range of motion, compared against the athlete's own measured dead hang. vitpose-plus-large
push_ups Counts push-up reps from a side-on video, times each push, and checks each rep for full range of motion and a straight back. vitpose-plus-large
rock_climbing Identifies the holds on a bouldering wall, shows which holds the climber used and in what order, and compares different attempts on the same route. sam3.1 (image segmentation) + vitpose-plus-large
rock_climbing_3d Same idea as rock_climbing, but from a video recorded with depth (iPhone LiDAR). It reconstructs the wall, holds, and climber in 3D and measures the route in meters. sam3.1 + vitpose-plus-large
running Measures a runner's cadence (steps per minute), times every foot strike, and averages the knee angle at the moment of contact. vitpose-plus-large
deadlift Counts deadlift reps and classifies each rep's back as straight or rounded, with a probability for the call. sam3.1 + vitpose-plus-large + gemma-4-26b-a4b-it (language model)
basketball Determines whether each shot is a make or a miss by tracking the ball through the rim, and plots every shot on an overhead court diagram with the field-goal percentage. sam3.1 + vitpose-plus-large

How to run it

  • Each demo folder has its own README with setup and instructions.
  • There's also a single browser app that lets you try most demos without per-project setup:
    1. Create the environment: conda env create -f environment.yml (or pip install -r requirements.txt)
    2. Activate it and run python app.py
    3. Open http://127.0.0.1:7860 in your browser
  • In the app you upload a clip, trim it, and analyze it.
  • No API key is needed — there's an anonymous, rate-limited tier. You can paste a VLM Run key or set the VLMRUN_API_KEY environment variable for higher limits.
  • Two demos are not in the browser app: rock_climbing_3d (because it needs a LiDAR capture) and push_ups.
  • ffmpeg is required.

License

Apache-2.0.

Notes

  • The README includes demo videos and screenshots for most projects (YouTube links for dance_sync, pull_ups, basketball, deadlift, and rock_climbing_3d; static images for the others).
  • The models are accessed through the VLM Run gateway, which is why an API key is optional but recommended for higher usage limits.

Related

  • Project
  • Project
  • Project
  • Project
  • Project