Apple’s Augmented Reality Calorie Counter points to new health apps on Apple AR/VR

AR for calculating carb and insulin
Image source: Delish

Apple is planning to release a new mixed-reality headset this year. The new headset, Reality Pro is expected to have some new health and fitness features.

According to a new patent filing released today, Apple is exploring calorie counter features on its augmented reality platform.

Related reading:

Apple’s calorie counting AR patent details

The food calorie counter in this instance takes advantage of the microphone and the camera on the AR device to evaluate how much food is being consumed by a user.Apple AR Calorie counter

How will the AR Calorie counter work?

According to the patent, the microphone on the AR headset will listen for ‘chewing noise.’

When the headset detects the chewing action by a user, it will trigger its camera unit in response. The camera’s purpose is to capture image data of the food.

Sophisticated algorithms on the company’s dedicated operating system for AR/VR, RealityOS, will use machine learning to help with identifying the food product based on image data. Apple AR Food calorie determination

Once the time associated with the chewing session is determined, the volume of food intake can be determined based on the interval of the chewing session and estimated calorie consumption can be determined based on the recognized food and the session duration.

The big idea is that when food consumption can be determined, the user may keep a more accurate log of calorie intake and realize health benefits.

Furthermore, the health application interface associated with calorie counting on the augmented reality platform will also be able to interface with other sensors.

Interfacing with Glucose Monitoring sensors

According to the patent language,

“health application 1240 is configured to obtain a glucose measurement. For example, health application 1240 may work in conjunction with a blood glucose monitor included in sensors 1210 to obtain a blood sample and measure blood sugar. 

In one or more embodiments where health application 1240 is stored in memory 1160, health application may work in conjunction with a blood glucose monitor included in sensors 1130 to obtain and/or utilize a blood sample to measure blood glucose levels.

 As another example, sensor 1130 may obtain optical, electrical, or other data utilized to measure blood glucose levels.”

This is suggesting that not only the platform will be able to detect food consumption activity and thereby calorie intake but also interface or integrate with a sensor//s to determine blood glucose levels.

This design on the patent suggests that this feature is more likely to be embedded in a smart glass-type AR/VR platform as opposed to a full VR headset. 

Although the patent language does not get into any details of a headset one would assume that you are not holding your iPhone to your ears while eating your food! (unless future AirPods have the Edge AI capability to determine chewing noises).

Fitbyte smart glass for food intake
Fitbyte smart glasses for food intake detection

Latest rumors suggest that Apple has allegedly shelved its idea for lightweight and higher-end augmented reality glasses, and is instead choosing to focus on the yet unannounced mixed-reality headset for the mass market.

Food Detection from image apps

The idea of detecting eating habits via camera use is not new. Fitbyte is a company that has looked into this use case in the past.

There are various other apps that have explored this in the past as well. One of these apps Loseit started showcasing this feature way back in 2017.

Point your iPhone camera onto your plate and your device immediately recognizes the bowl of spaghetti. This was showcased when Apple released iOS 11.

More recently, there have been other apps like Food Lens AR and AR DeepCalorieCam that provide food analysis and nutrition analysis based on food images. The AR component helps.

It will be interesting to see how Apple is planning to differentiate its offerings in this space.

Previous articleSleep tracking not working on Samsung Galaxy Watch? Lets fix it
Next articleSamsung’s Wearable robot to feature a ‘Boost’ Mode and an easy to understand Gait Index score
Sudz Niel Kar
I am a technologist with years of experience with Apple and wearOS products, have a BS in Computer Science and an MBA specializing in emerging tech, and owned the popular site AppleToolBox. In my day job, I advise Fortune 500 companies with their digital transformation strategies and also consult with numerous digital health startups in an advisory capacity. I'm VERY interested in exploring the digital health and fitness-tech evolution and keeping a close eye on patents, FDA approvals, strategic partnerships, and developments happening in the wearables and digital health sector. When I'm not writing or presenting, I run with my Apple Watch Ultra or Samsung Galaxy Watch and closely monitor my HRV and other recovery metrics.


Please enter your comment!
Please enter your name here

This site uses Akismet to reduce spam. Learn how your comment data is processed.