Media
1 item
Description
Detect hand signs and control media or browser actions What is NyandSign? NyandSign is a Chrome extension that detects hand signs with your webcam and lets you control media playback and browser actions hands-free. It uses Google MediaPipe's on-device AI to recognize hand signs in real time. All processing happens locally in your browser, and your camera feed is never sent to any external server. Use hand signs to control media on sites like YouTube, Spotify, and Netflix, or switch to Browser Control mode for scrolling, back/forward navigation, tab switching, and zoom. Any regular webcam works, and supported electric glasses camera devices can also be used. With XREAL Eye, you can choose between the high-resolution RGB camera and the low-resolution 6DoF monochrome camera. KEY FEATURES 🖐️ 12 Recognizable Hand Signs Fist, Peace, Three, Four, OK, Aloha (shaka), Rock, Point Right, Point Left, Point Up, Thumbs Up, and Thumbs Down — each sign can be mapped separately for each operation mode. 🎬 Media Control Mode ▶⏸ Play / Pause 🔊🔉 Volume Up / Down 🔇 Mute ⏩⏪ Seek Forward / Backward (10 seconds) ⏭⏮ Next / Previous Track ⏫⏬ Playback Speed Up / Down 2× 2x Speed 1× Reset Playback Speed 🌐 Browser Control Mode ↕ Hold to Scroll — hold the assigned sign and move your hand up, down, left, or right to scroll ↟↡ Page Top / Page Bottom ↩↪ Back / Forward ⇥⇤ Next Tab / Previous Tab ⇡⇣⇠⇢ Cursor Up / Down / Left / Right ↻ Reload ＋− Zoom In / Zoom Out 100% Reset Zoom 🟡 Pointer Control Mode Display a virtual cursor inside the page, hold an assigned hand sign, and move your hand to move the cursor. Use a separate click sign to click links, buttons, and other page elements. This is implemented as an in-page virtual cursor within what a Chrome extension can control; it does not move the operating system mouse cursor itself. ⚡ Fully Customizable Switch between Media Control, Browser Control, and Pointer Control modes Customize hand sign mappings separately for each mode Adjustable gesture hold time to reduce accidental triggers Repeatable actions for continuous volume, seek, scroll, and zoom control Wake sign mode — require an activation sign before commands are accepted Wake sign options — Open Hand (either side), Open Hand (palm only), or always active Two-hand signs — toggle operation ON/OFF or switch operation modes Built-in tutorial — explains the basic flow and core operation modes 📌 PiP (Picture-in-Picture) Mode When fullscreen video makes the side panel intrusive, switch to a compact popup window. In PiP, Media Control mode uses a green border, Browser Control mode uses a blue border, and Pointer Control mode uses an amber border, so you can tell which mode is active at a glance. 🎯 Target Tab Control Send gesture actions to a specific tab. Pin a target tab to control background media or a specific page while browsing elsewhere. 👓 Electric Glasses Camera Support NyandSign supports XREAL Eye UVC0 (RGB) and UVC1 (6DoF monochrome) camera selection. Because Windows may show similar labels for both cameras, NyandSign detects the camera profile from actual resolution and capabilities instead of relying only on the display name. UVC1 uses its native 512x378 resolution and applies preview aspect correction, light smoothing for grainy monochrome video, contrast correction, and brightness compensation. When Chrome exposes camera controls, NyandSign also tries to apply exposure/brightness controls. 🔒 100% Local & Private All hand tracking runs on-device using WebAssembly. Your camera feed never leaves your browser. No cloud processing, no data collection, no accounts required. 🌐 Bilingual English and Japanese (日本語) interface support. BENEFITS OF INSTALLING NyandSign Hands-free convenience — Control media and pages while cooking, exercising, eating, or when your hands are occupied. Accessibility — Provides an alternative input method for users who have difficulty using a keyboard or mouse. Electric glasses camera support — Designed to work with supported XREAL / Nreal camera devices. With XREAL Eye, you can choose UVC0 (RGB) or UVC1 (6DoF monochrome) to balance recognition quality and device load. Distraction-free viewing — PiP mode lets you enjoy fullscreen content without a visible side panel while keeping hand sign control active. Flexible controls — Configure media and browser actions separately, and map your most-used actions to the signs that feel natural to you. No additional software — Everything runs as a lightweight Chrome extension. No desktop apps, drivers, or accounts needed. WHY THESE PERMISSIONS ARE NEEDED NyandSign requests the minimum permissions required to function. Here is exactly what each permission does and why it is necessary: activeTab Allows the extension to interact with the currently active tab when you click the extension icon. This is needed to inject the content script into the page you are viewing, so media actions and browser-page actions such as scrolling can reach the page. tabs Allows the extension to read tab titles and manage tab-related actions. This is used for the Target Tab feature and for actions such as Next Tab and Previous Tab. Tab information is only used locally in the side panel and is never transmitted anywhere. scripting Allows the extension to inject a small content script into web pages. This script is the bridge between NyandSign and the page: it receives gesture commands and applies them to media elements or page scrolling. Without this permission, gestures could be detected but could not control the page. storage Allows the extension to save your settings, including hand sign mappings, operation mode, volume preferences, and display options, in Chrome's built-in storage. Your configuration persists across browser restarts. If Chrome Sync is enabled, Chrome itself may sync this settings data across your devices. NyandSign does not send it to its own external servers. sidePanel Allows the extension to display its interface as a Chrome side panel. The side panel is the primary UI where you see the camera feed, hand tracking visualization, current sign, operation mode, settings, and tutorial. Host permission: all URLs The content script needs to run on any page because media and scrollable content can appear on any website — YouTube, Spotify, Netflix, Twitch, SoundCloud, news sites, documentation sites, and countless others. Restricting access to specific domains would break control on unlisted sites. The content script only handles the page interactions required by gesture commands and does not send camera data or personal information anywhere. TECHNICAL DETAILS Uses Google MediaPipe Hand Landmarker (on-device, WebAssembly) Runs at configurable FPS (default 15 fps) to balance accuracy and CPU usage Detects XREAL Eye UVC0 / UVC1 camera profiles from actual resolution and capabilities UVC1 uses native 512x378 capture, preview aspect synchronization, and monochrome inference preprocessing Distance-based gesture recognition with palm-size normalization, plus 3DoF hand-pose checks (face / yaw / pitch / roll) for wake sign detection Temporal majority voting (200ms window) + hysteresis for stable detection All processing happens locally — camera feed is never transmitted Manifest V3 compliant WHAT'S NEW v1.1.163 - Added a 2x Speed media action - Improved repeated seeking so holding the sign advances in larger steps over time - Improved recovery for repeatable actions such as volume, speed, and seek when the hand briefly leaves detection - Extended wake-active timing while a command is pending or a repeat action is running, making hold actions less likely to time out mid-gesture - Reduced cases where an Open Hand shape was misrecognized as OK - Made the PiP return-to-side-panel button clearer and shared the gesture action runtime between the side panel and PiP v1.1.153 - Reworked the wake Open Hand check around 3DoF-style hand pose values (face / yaw / pitch / roll), making camera-facing hands easier to reason about - Reduced accidental wake triggers from angled, flat, or incidental hands such as hands appearing while typing - Tuned wake Open Hand checks separately for palm-side and back-of-hand views - Prevented a held wake sign from waking again after the standby timeout until the hand is released once - Added clearer wake-sign guidance, such as facing the hand toward the camera, reducing pitch, or spreading the fingers - Added a two-hand Open Hand sign - Expanded pointer movement start/recovery from Point Up to include Point Left and Point Right - Reduced cases where pointing signs were misrecognized as Thumbs Up at certain angles - Kept UVC1 inference input at native resolution to improve stability on the low-resolution monochrome camera v1.1.132 - Added XREAL Eye UVC0 (RGB) / UVC1 (6DoF monochrome) camera selection - Detects XREAL Eye camera profiles from actual resolution and capabilities, even when Windows shows similar camera labels - Captures UVC1 at its native 512x378 resolution and fixes preview/skeleton aspect mismatch - Added grain smoothing, contrast correction, and brightness compensation for UVC1 monochrome inference input - Tries to apply UVC1 exposure/brightness camera controls when Chrome exposes them - Improved retry behavior and diagnostics when XREAL Eye camera capture fails - Improved camera profile handoff between the side panel and PiP, shared notification sound generation, and camera cleanup v1.1.118 - Promoted Pointer Control mode to a standard mode alongside Media Control and Browser Control - Added the Point Up hand sign, with pointer movement assigned to Point Up and click assigned to OK by default - Improved pointer movement responsiveness, reduced target-highlight load, and stabilized iframe target display and clicking - Added settings for inference resolution, idle FPS limiting, Hold to Scroll speed, and pointer movement speed - Reorganized settings into Operation, Display, and Recognition tabs, with reset confirmation and two-hand sign reset - Improved hand sign recognition rules and debug display v1.0.91 - Added experimental Pointer Control mode, which lets you move an in-page virtual cursor with hand signs - Added click support and edge scrolling while using Pointer Control mode - Improved the visual feedback while pointer movement is active - Improved cleanup so the virtual cursor and camera stream are less likely to remain after stopping the camera or closing the extension - Improved recovery when your hand briefly leaves the camera view during Hold to Scroll or pointer movement v1.0.81 - Improved stability during Hold to Scroll when hand sign recognition briefly fluctuates - Prevented another sign from triggering accidentally when ending Hold to Scroll or repeatable actions - Improved scroll target detection on Google sites and similar pages - Shows a blocked-page indicator when page actions cannot run because script injection is unavailable v1.0.75 - Added Browser Control mode - Added browser actions such as scroll, back/forward, tab switching, cursor movement, reload, and zoom - Improved compatibility for cursor movement actions - Added separate hand sign mappings for Media Control and Browser Control modes - Added two-hand sign actions for operation ON/OFF and mode switching - Updated the tutorial and PiP visual feedback for the new two-mode workflow v1.0.61 - Improved OK sign recognition, especially for angled hands and landmark estimation variance v1.0.58-v1.0.60 - Added an Open Hand (palm only) wake gesture option - Refined Open Hand detection to reduce false positives against the OK sign v1.0.57 - Replaced the fullscreen action with playback speed reset v1.0.56 - Added playback speed up / down actions v1.0.55 - Improved PiP mode with clearer wake-state feedback v1.0.54 - Added Rock, Three, and Four signs, expanding recognition to 11 hand signs v1.0.44-v1.0.53 - Added a first-run tutorial - Added target-tab pinning - Improved stability by consolidating shared settings and constants v1.0.38-v1.0.43 - Expanded English/Japanese UI coverage - Improved multilingual smart-glasses labeling - Refined default settings and reset behavior SUPPORT If you encounter any issues or have feature requests, please visit our GitHub repository.