Apple is planning to release a new mixed-reality headset this year. The new headset, Reality Pro is expected to have some new health and fitness features.
According to a new patent filing released today, Apple is exploring calorie counter features on its augmented reality platform.
Contents
Related reading:
- Apple’s food monitoring feature could be the right step toward detecting interstitial glucose
- Review: Withings Blood Pressure Monitor (BPM) Connect for Apple iPhone users
- Using Apple Watch and Siri to check your health and fitness stats and a lot more!
Apple’s calorie counting AR patent details
The food calorie counter in this instance takes advantage of the microphone and the camera on the AR device to evaluate how much food is being consumed by a user.
How will the AR Calorie counter work?
According to the patent, the microphone on the AR headset will listen for ‘chewing noise.’
When the headset detects the chewing action by a user, it will trigger its camera unit in response. The camera’s purpose is to capture image data of the food.
Sophisticated algorithms on the company’s dedicated operating system for AR/VR, RealityOS, will use machine learning to help with identifying the food product based on image data. 
Once the time associated with the chewing session is determined, the volume of food intake can be determined based on the interval of the chewing session and estimated calorie consumption can be determined based on the recognized food and the session duration.
The big idea is that when food consumption can be determined, the user may keep a more accurate log of calorie intake and realize health benefits.
Furthermore, the health application interface associated with calorie counting on the augmented reality platform will also be able to interface with other sensors.
Interfacing with Glucose Monitoring sensors
According to the patent language,
“health application 1240 is configured to obtain a glucose measurement. For example, health application 1240 may work in conjunction with a blood glucose monitor included in sensors 1210 to obtain a blood sample and measure blood sugar.
In one or more embodiments where health application 1240 is stored in memory 1160, health application may work in conjunction with a blood glucose monitor included in sensors 1130 to obtain and/or utilize a blood sample to measure blood glucose levels.
As another example, sensor 1130 may obtain optical, electrical, or other data utilized to measure blood glucose levels.”
This is suggesting that not only the platform will be able to detect food consumption activity and thereby calorie intake but also interface or integrate with a sensor//s to determine blood glucose levels.
This design on the patent suggests that this feature is more likely to be embedded in a smart glass-type AR/VR platform as opposed to a full VR headset.
Although the patent language does not get into any details of a headset one would assume that you are not holding your iPhone to your ears while eating your food! (unless future AirPods have the Edge AI capability to determine chewing noises).
Latest rumors suggest that Apple has allegedly shelved its idea for lightweight and higher-end augmented reality glasses, and is instead choosing to focus on the yet unannounced mixed-reality headset for the mass market.
Food Detection from image apps
The idea of detecting eating habits via camera use is not new. Fitbyte is a company that has looked into this use case in the past.
There are various other apps that have explored this in the past as well. One of these apps Loseit started showcasing this feature way back in 2017.
Point your iPhone camera onto your plate and your device immediately recognizes the bowl of spaghetti. This was showcased when Apple released iOS 11.
More recently, there have been other apps like Food Lens AR and AR DeepCalorieCam that provide food analysis and nutrition analysis based on food images. The AR component helps.
It will be interesting to see how Apple is planning to differentiate its offerings in this space.
