ok-oldking/ok-script

Modern Python Computer Vision Automation Framework

What it solves

It provides a unified Python framework for automating interactions with Windows applications, Windows games, Android emulators, and ADB-connected devices. It eliminates the need to write separate automation scripts for different platforms by allowing a single set of task code to work across these environments.

How it works

The framework leverages computer vision to identify UI elements and trigger actions. It integrates several vision techniques, including OCR (Optical Character Recognition), OpenCV template matching, and YOLO-based detection. To manage visual assets, it uses the COCO format for template materials, which allows the system to automatically scale templates based on the screen resolution of the target device.

Who it’s for

It is designed for developers creating automation bots, specifically for those targeting desktop software and mobile games running on Windows or Android.

Highlights

  • Multi-platform support: Works across Windows clients, Android emulators, and ADB devices with a single codebase.
  • Flexible interface options: Can be run in headless mode, via a Qt-based desktop GUI, or through a FastAPI-based web interface.
  • Vision-integrated: Built-in support for OCR, template matching, and YOLO.
  • Resolution adaptive: Automatically scales templates to fit different screen resolutions.
  • Developer tools: Includes tools for screenshotting, input simulation, device control, and a visual debugging overlay.

Related

  • Project
    oculix-org/SikuliX1

    A computer vision-based automation tool that identifies and interacts with screen elements using image recognition to simulate mouse and keyboard input.

  • Project
    WallBreaker2/op

    A Windows automation plugin that integrates window management, screen capture, OCR, and YOLO detection into a unified API for scripting and desktop automation.

  • Project
    askui/python-sdk

    A Python SDK that enables AI agents to control desktop and mobile devices using vision-based automation instead of brittle UI selectors.

  • Project
    VIAME/VIAME

    VIAME is a BSD‑licensed, open‑source computer‑vision toolkit that provides a modular C++/Python pipeline framework, a large set of ready‑made detection/tracking/measurement algorithms, and both desktop and web GUIs. It can be installed via pre‑built binaries, Docker, or pip, and supports GPU acceleration through CUDA, PyTorch, ONNX, TensorRT, etc. Ideal for researchers and developers building video analytics pipelines.

  • Project
    go-vgo/robotgo

    A cross-platform Go library for desktop automation that enables programmatic control of the mouse, keyboard, and screen for auto-testing and AI computer-use agents.