Apple’s latest approved patent “ Method and device for Health monitoring” provides an outline of some of the thinking when it comes to the use of Vision Pro and other Apple devices for health monitoring.
Apple’s health ambitions are hardly limited to its Apple Watch and iPhone. The Cupertino giant is taking a close look at some of the unique features of its mixed reality platform and engineering approaches in order to assist people affected by a medical impairment.
The new patent clearly calls this out in its summary:
“Various implementations disclosed herein include devices, systems, and methods for remedying a medical impairment of a user.”
The paper goes on to explain a method that includes detecting, based on data from at least one of the image sensor and biometric sensors, a medical impairment of a user of the head-mounted device from a plurality of potential medical impairments associated with a plurality of remedies.
Related Reading:
- Apple is expanding its offerings in the Mindfulness space with Vision Pro
- Apple’s Emotion Detection research shows how the company is leveraging AI
- The best Apple Watch alternatives to Body Battery and other recovery features
- See the amount of time you or your child spends in daylight with an Apple Watch and the Health app
The outline discusses selecting, from the plurality of remedies, a remedy of the medical impairment of the user. The method includes controlling the display to effect the remedy of the medical impairment of the user.
With a vast amount of biometric information from a scene camera and/or biometric sensors, an HMD( Head Mounted device aka Vision Pro) is able to detect whether a user of the HMD suffers from any of a predetermined list of medical impairments.
If a particular medical impairment is detected, the HMD can control the display to effect a remedy for the particular medical impairment.
Medication Reminders
One use case as described in the patent details how the Vision Pro can be used for tracking medication reminders.
At the first time, the HMD detects that the user has failed to take a scheduled dose of medication located within the medicine bottle. For example, using the scene camera, the HMD did not detect the dose of medication in the hand of the user prior to a time of the scheduled dose. The figure below shows how the device respond to the missed medication event.
Detect Intoxication and call Ride Share options
A second use case discusses how the Vision Pro device could detect if a user is intoxicated and provide the user with options to auto call ride sharing options.
Track Eye and Vision issues
The eye tracking camera on the Vision Pro can be used to detect if the user is squinting or closing one eye to avoid double vision.
The device adjusts a parameter of the display, such as a brightness, color, contrast, saturation, sharpness, or zoom level., in response to detecting that the user is vision impaired, the CGR environment is adjusted accordingly.
Detect Hydration and provide Guidance
A much cooler use case takes advantage of the technology to determine if the user is dehydrated. The HMD detects the user is dehydrated, e.g. using the scene camera to determine water intake, using an IMU to determine dizziness, or using a thermometer, skin sensor, or a breathing monitor. The HMD will then guide the user to a source for water.
Detect Hyperthermia or Hypothermia
According to the patent, the device detects that the user has hypothermia or hyperthermia using the image sensor to determine a color of the user’s skin or a change in the color of the user’s skin (e.g., bluish hue indicates hypothermia and/or a reddish hue indicates hyperthermia) and/or one or more biometric sensors, such as a heart rate monitor to determine a heart rate of the user or a change in the heart rate of the user (e.g., increased heart rate indicates hypothermia or hyperthermia). Coupled with a breathing monitor, the process can determine a breathing rate of the user or a change in the breathing rate of the user (e.g., rapid breathing indicates hyperthermia or mild hypothermia and/or slowed breathing indicates severe hypothermia).
Assist with Seizure Detection and provide assistance
The device detects that the user has fallen or is having a seizure using the image sensor to detect a dropping (and/or tilting) of the camera or a shaking of the user’s body or a portion thereof (e.g., the user’s hands) and/or one or more biometric sensors, such as an IMU to detect a fall or shaking or a heart rate monitor to determine a heart rate of the user or a change in the heart rate of the user (e.g., increased heart rate indicates a fall or seizure).
In Summary, We are still in the early innings when it comes to using mixed reality for health monitoring and wellness use cases. We predict that as the platform matures and scales, thus making it more adorable in the future, we will begin to see more use cases in the healthcare space.
- Facebook’s Health Technology Group and Reality Labs hiring Clinical Research personnel
- Facebook exploring biosensors for future AR/VR devices and wearables
It is just not Apple that is engineering ways to leverage mixed reality for the health vertical but other tech giants such as Meta have been also prototyping various health-related use cases including EEG sensors.
Please stay tuned as we provide additional coverage on this emerging space.
