H
visionOS Medical XR

Code Blue

A scalable XR tool to improve Code Blue readiness anywhere — built for Apple Vision Pro with multiplayer SharePlay, hand tracking, and real-time patient simulation

Timeline Mar 2025 — Apr 2025
Role XR Designer
Team Hannah Yunbing Han, Jasmine Yiming Sun, Nandita Venkat, Sahil Mayenkar
Tools SwiftUI, RealityKit, ARKit, visionOS 2.0, SharePlay, Figma

Code Blue SIM is a medical simulation app for Apple Vision Pro that trains healthcare professionals to execute life-saving cardiac arrest protocols. It combines hand-tracked interactions, real-time patient simulation, and the full AHA 2020 ACLS algorithm in an immersive spatial computing environment.

01

Problem

When a patient's heart stops, the clock starts ticking — but delayed action, unclear roles, and lack of training stand in the way. Traditional simulation labs cost $100k+ and lack dynamic patient feedback. Communication breakdowns are implicated in 70% of medical adverse events.

Code Blue — Problem Statement
02

Solution

  • Tutorial Mode — self-paced practice for CPR, intubation, IV/Fluids, and Team Leader with real-time feedback on compression depth and rate.
  • Simulation Mode — multi-user collaboration through SharePlay with hand tracking and role assignment across different physical locations.
  • Advanced Training Mode — realistic scenarios with a living patient simulation, dynamic vital signs, and progressive skill levels from Compression Mastery to Full ACLS.
Code Blue — Three Training Modes
Code Blue — Tutorial Mode
Code Blue — Simulation Mode
Code Blue — Advanced Training Mode
03

Physical Integration

A heart-shaped foam trainer anchors the XR scenario in 3D space, replicating the exertion of real CPR with tactile haptics. Image tracking places the scene onto any surface — preserving physical authenticity at a fraction of the cost of traditional manikins.

Code Blue — SharePlay Role Assignment
Code Blue — VR Simulation Scene
04

Technical Highlights

  • Hand Tracking — controller-free medical procedures via ARKit hand joint tracking
  • Living Patient — dynamic cardiac rhythms (VF, VT, PEA, Asystole) with real-time ECG, SpO2, and EtCO2
  • Evidence-Based Scoring — ACLS deviation tracker graded from S-rank (96% ROSC) to F-rank (patient death)
  • Multiplayer Sync — SharePlay with GroupSessionMessenger for cross-device state and spatial personas
Code Blue — Team
05

Impact

  • Flexible — Train alone or with instructors and peers.
  • Accessible — Practice anytime, anywhere — no sim lab required.
  • Scalable — Lower cost than traditional manikins or sim centers.
06

Acknowledgements

Designers: Hannah Yunbing Han, Jasmine Yiming Sun

Engineers: Nandita Venkat, Sahil Mayenkar

Senior Advisors: Steven Patterson, Christopher Robinson, MD, PhD

Mentors: Hunter Harris, Dat Nguyen, Jenny Wang, Daniel Pillis, Jenna Fizel

Program: MedVR Accelerator 2025

3D Assets: Charite University Hospital Operating Room, Adult Intubation Procedure Demonstration via Sketchfab

Back to Home Next Project