ayushjain1144/trackeverything

Code repository for "TrackEverything: Long Horizon Dense Tracking via De-Duplicating 3D Scene Representations"

What it is

TrackEverything is a research project (paper + upcoming code) from Carnegie Mellon University and Meta on a hard computer-vision problem: dense tracking — following every point in a video across frames — over long time horizons (many seconds or minutes, not just a few frames).

The core idea in the title is "de-duplicating 3D scene representations": instead of tracking pixels directly in 2D (which fails when objects are occluded, blurry, or change appearance), the method builds a 3D representation of the scene and removes redundant/duplicate 3D points so that each tracked point is unique and stable over time. This lets tracks survive long spans where a point disappears and reappears.

Why it matters

Long-horizon dense tracking is a building block for robotics, AR/VR, video editing, and 3D reconstruction — anything that needs to know "where did this physical point go" over a long video. Prior methods drift or lose points over time; this work targets that limitation.

Status

  • Paper: available on arXiv (link in README)
  • Project page: trackeverything.github.io (with teaser demo)
  • Code: not yet released — README says "soon (maybe next week)"

Who's behind it

A mix of academic (CMU) and industry (Meta) researchers, several with strong track records in 3D vision and SLAM (e.g., Tanner Schmidt, Jakob Engel).

Caveat

The README is a bare paper announcement — it gives the title, authors, links, and citation, but no technical details of the method. The explanation above is inferred from the title and the field's context, not from a detailed write-up.

Related

  • Project
    a312863063/Video-Auto-Wipe

    A deep learning video inpainting project that automatically detects and erases fixed-pattern content like subtitles, static logos, and dynamic icons from videos. It uses a two-step training process (time-domain perception + end-to-end inpainting fine-tuning) with synthetic foreground data overlaid on real video backgrounds. The subtitle removal model is open-sourced, while logo removal models are demonstrated but not released.

  • Project
    QuanjianSong/FashionChameleon

    FashionChameleon is a real-time framework for interactive human-garment customization in video generation, enabling high-speed garment switching at 23.8 FPS.

  • Project
    zhouwei713/seedance-prompt

    A prompting framework and skill for AI video generators that replaces generic descriptors with detailed simulations of real-world recording equipment and human imperfections to eliminate the "AI look."

  • Project
    Tencent-Hunyuan/Prism

    Prism is a research codebase for training and running high‑resolution (up to 2 K) joint video‑audio diffusion models. It introduces dynamic block‑sparse attention that adapts to local motion and audio‑visual coupling, achieving ~2.5× faster training while improving generation quality. The repository includes environment setup, pretrained checkpoints, inference scripts (single‑GPU DeepSpeed or multi‑GPU FSDP), and a full training pipeline with data‑preparation tools.