01 — OBSERVE
More than one viewpoint.
Start with one camera. Expand to room and bed views after calibration. Each observation needs a timestamp, a common coordinate space, and a confidence estimate.
Research preview · 001
A second
perspective on
body tracking.
Your Pico trackers keep the motion going. Cameras help it find its footing. A framework for camera-assisted full body tracking through WiVRn NX.
Local-first design. Per-joint corrections. Pico fallback.
FUSION / CONCEPT VIEW● ILLUSTRATION
Camera correctionPico fallbackConcept · no live data
Built to investigate.This release provides a native WiVRn NX dashboard, optional camera capture with MediaPipe 2D overlays and hip-relative inferred 3D, plus a separate interactive fusion simulation. Calibration and live tracker injection remain future work.
01 / The approach
Trust is earned.
Joint by joint.
A camera losing one ankle should not take down your entire skeleton. The design keeps assistance selective and the original Pico estimate available.
02 — CORRECT
Help where it makes sense.
Evaluate joints individually. Reject stale or implausible observations and blend eligible corrections. Headset and controller tracking remain authoritative.
03 — FALL BACK
Keep Pico in the loop.
When visibility drops, assistance fades back toward Pico. Camera processing belongs outside the VR tracking loop so a stalled camera cannot stall the headset.
BASELINEPico body poses
OBSERVATIONSCalibrated cameras
FUSIONNX Fuse
PLANNED ADAPTERWiVRn NX → VR
02 / Real spaces
Your room isn't a studio.
Different views solve different visibility problems. Overlapping cameras can add depth; separate views can extend coverage. Both need calibration.
Space to move.
Begin with visible hips and feet in a room-facing view. Test drift correction, movement lag, and transitions before adding more complexity.
Space to unwind.
Overhead views and lying poses are research targets. Blankets, crossed legs, and hidden joints remain hard cases; unseen limbs need fallback.
03 / Build in the open
Proof before promises.
Small milestones, explicit limits, and repeatable comparisons. The native dashboard and camera observation path are available; calibrated live correction remains ahead.
00
Fusion playground
Assisted and camera-only simulations, tracker-neutral fusion logic, and checks for fallback behavior.
Framework01
Native dashboard
WiVRn NX dashboard with a Body Tracking page, local camera discovery, explicit capture, and simulation transport.
Available02
One camera, real poses
MediaPipe 2D landmarks and hip-relative inferred 3D are available; camera-to-VR calibration and quality gates remain.
Preview03
Tracker injection
Verify the body-pose path, add an optional adapter, and demonstrate one corrected joint with an immediate bypass.
Planned04
Multiple perspectives
Register additional cameras, test overlap and view handovers, then evaluate lying and occluded poses.
ResearchStart exploring
Meet the framework.
Run the local simulation, or open the native WiVRn NX dashboard for camera discovery and explicit capture controls. No headset or camera is required for the simulation.
Native dashboard & setup guide Source, simulation & research plan# Clone the research framework
git clone https://github.com/nerdrx/nx-fuse.git
cd nx-fuse
# Launch the local playground
python3 app.py# Follow the local URL shown at startup.
A few honest answers
Before you plug anything in.
Does this improve my tracking today?
Not yet. The native dashboard and camera path provide local observation and simulation tools. Real tracker injection and calibrated camera-to-VR correction must be implemented and validated before they can change tracking in a game.
What does one camera provide?
The optional MediaPipe path shows 2D landmarks and hip-relative inferred 3D coordinates. They are uncalibrated model coordinates, not measured camera depth or calibrated VR-space poses. Read the monocular research plan.
Where is the native dashboard?
The dashboard is developed in WiVRn NX. Its Body Tracking page discovers local cameras and keeps capture explicit; the simulation service remains separate. Read the native dashboard guide.
Will it work with other trackers?
The fusion core accepts named baseline poses without a tracker SDK. Pico through WiVRn NX is the first hardware target. Tracker injection and other adapters are planned, not verified integrations.
Can I use phone cameras or any number of webcams?
Those are possible future inputs, not confirmed supported devices. Practical camera count depends on capture support, USB or network bandwidth, synchronization, inference cost, and calibration quality. The first hardware milestone deliberately uses one camera.
What happens under a blanket?
A camera cannot reliably recover invisible limbs from an image alone. The intended behavior is to reduce camera influence and retain the Pico estimate, rather than confidently invent a pose.
Where does camera footage go?
The camera path processes frames locally and only exposes the bounded latest JPEG through the local service. It does not upload footage or inject poses into WiVRn.