This is a collection of context-aware AI kits, fully integrated with the Meta XR SDK and Unity.
Vision kits are built to support Meta Camera Access (PCA).

1. Spectacles Reachy Mini
2. AI Capabilities Workbench
3. Zero-Shot Object Detection Kit
4. Custom AI Model Training Kit (Roboflow)
5. Poker AI Assistant (Roboflow)
6. Acknowledgements & Credits
7. License
8. Contact

🎯 A Snap Spectacles application that enables robot control through AR hand tracking.

• Control Reachy Mini robot using hand gestures through Spectacles
• Real-time hand tracking to robot movement mapping
• AR visualization of robot controls

⚠️ Setup Notes

• Hardware Required: Requires Reachy Mini robot and Snap Spectacles
• Configuration: Robot and Spectacles must be connected

🎯 A Unity based workbench implementing 7 workflows and 10 providers, to bring contextual AI to your apps. Compatible with Meta Quest.

• Multi-AI Provider Support – Supports Nvidia, OpenAI, Google Gemini, Groq, Roboflow, Stability AI, and AWS
• Pre-configured scenes with easy API key management, enabling quick AI-powered XR prototypes

⚠️ Limitations

• Requires API Keys: Users must register with the AI providers and configure their own API keys for detection to work
• Requires Internet: Some of the workflows send calls to cloud API

🎯 A Unity plugin that enables real-world object detection in XR using Microsoft Florence-2 on Meta Quest.

• Instantly detect objects in your environment with zero-shot AI (no training required)
• Send image data from your Quest to the Florence API and receive rich detection & description results
• Fully integrated with Unity for easy setup and flexible use in XR workflows

⚠️ Limitations

• Not Real-Time: API response times mean detection is fast but not instantaneous
• Requires Internet: Wi-Fi needed to send images to the cloud API
• No Timestamps: Bounding boxes are not time-synced

🤖 A Unity plugin that brings you custom-trained object detection to XR — powered by Roboflow and optimized for Meta Quest.

• Upload and annotate your own image datasets (capturing directly on Meta Quest recommended)
• Run models locally on-device for faster inference and offline use
• Fully integrate detection results into your Unity XR app

⚠️ Setup Notes

• Setup Time: Requires effort to collect, annotate, and train datasets
• CUDA & Docker Setup Needed: See Roboflow’s repo

♣️ A Meta Quest application that uses computer vision and Poker Odds API to calculate poker hand strenght in real time.

• Detects poker cards and calculates odds and win probabilities
• Runs 100,000+ simulations for highly accurate predictions using the Poker Odds API
• Performs local inference directly on Meta Quest
• Minimalistic UI/UX optimized for wearables

⚠️ Setup Notes

• Wi-Fi Configuration: Ensure Meta Quest is on the same Wi-Fi as your server
• Server Setup: Local Roboflow inference server and Node.js Poker Odds server must be running
• Permissions & IP: Verify permissions on Meta Quest and correct IP address in RoboflowCaller
• CUDA & Docker Setup Needed: See Roboflow’s repo

• Join SensAI Hack and connect with a community of creators and innovators.
• Check out our SensAI Knowledge Hub for curated learning use cases and inspiration across AI, XR, and robotics.
• Huge thanks to Lucas Martinic for the Zero-Shot Object Detection Kit – definitely follow him for more awesome AI projects.
• Thanks to Nigel Hartman for the Custom AI Model Training Kit (Roboflow) – a great source of AI & XR insights.
• Big shoutout to Rik Turnbull for the AI Capabilities Workbench – his AI workflows and XR demos are super inspiring.
• And thanks to Johannes Tscharn for the Spectacles Reachy Mini – check out his work for more cool robotics projects.

Powered by SensAI Hackademy

📜 By downloading and using these kits, you agree to the License Terms.

✉️ Have questions, suggestions, or feedback? We’d love to hear from you! Reach out to us at [email protected]