Companion Medical researching use of Augmented Reality and AI for Diabetes Management

AR and AI for Diabetes management

Companion Medical, now a part of Medtronic, makers of InPen, the only FDA cleared Bluetooth enabled smartpen, is exploring novel approaches to diabetes management.

The new research focuses on using augmented reality experience with the patient user’s medical device to simplify carbohydrate estimation and conveniently inform the patient users about food nutrition before they make a selection.


Patients with diabetic conditions are often required to self-administer doses of fluid medication. When administering a fluid medication, the appropriate dose amount is set and dispensed by the patient using a syringe, a pen, or a pump.

The new Health Management App

A health management app on a user’s smartphone operates as a complementary medical device to the medicine dispensing device (also referred to as the pen device).

The app adds value by providing various support tools, including a dose calculator and decision support modules to calculate and recommend a medicine dose (e.g., insulin).

And the app can also automatically create records.

For example, each bolus dispensed by the pen device can be automatically logged and communicated to the companion device. 

In its latest research filing, Companion is exploring to extend the service offering of the health management app by using augmented reality along with AI algorithms.

 A/R to enable better Decision making by users

As per the patent, the health management software application includes augmented reality and contextual dose calculation module to calculate or estimate the carb value (e.g., number of grams of carbohydrate) of certain food and/or mealtime insulin needed upon consumption of said food.

AR for calculating carb and insulin
Image source: Delish

For example, the app commands the camera of the smartphone. It may overlay new data calculated from aggregated data of the user’s meal options (based on location, past meal, and other information) and recommended insulin dose based on tracked glucose and insulin data. 

In some implementations, the app can display estimated grams of carbs or the equivalent dose of insulin needed for the particular user based on their clinical parameters, where very high-carb (or high-insulin-required) items may be highlighted, and/or the healthiest or lowest-carb (low-insulin-required) items may be highlighted to help guide selection. 

Using AI for text recognition and optimizing calculations

Artificial intelligence may be implemented in any number of aspects of the app for many reasons, including (but not limited to) optimizing insulin recommendations and/or data aggregation. 

A typical insulin bolus (dose) calculator works by evaluating a diabetic person’s (also referred to as “user” or “patient”) current blood glucose level (BG), the insulin in their body from previous doses (e.g., insulin on board or JOB), and the number of grams of carbohydrates (“carbs”) the user is or recently has been eating.

There are established equations for calculating insulin doses based on the number of carbohydrates being eaten, but often estimating the carbs within a particular food or meal is difficult and imprecise.

Also, traditional dose calculators and meal estimation tools aim to assess food that has already been selected for a meal. However, providing carb or insulin information before selecting a meal or choosing where to dine could help educate patients and influence their better health and BG control decisions. 

The A/R use case uses the smartphone’s camera to identify the food that the user is planning to consume and overlay new data that show insights for best metabolic health decision making and provide users with insulin dosage information.

GPS positioning can automatically determine the store/restaurant the user is visiting. The A/R app may prompt the user to confirm, for example, a popup window on the user interface. 

Text recognition would identify the various menu items, and carb estimates could be sourced from published nutritional information for the restaurant, from crowd-sourced databases of nutritional information, or based on a database of common food dishes with similar names. 

The app may identify food items that are high-carb or low carb and then highlight those items on the user interface.

As A/R use cases find mainstream applications, health tech companies such as Companion Medical are exploring new ways to provide comprehensive support for personalized disease management, from informed decision making to precision drug delivery.

Source: Patent #20200327976 

Previous articleApple’s future AirPods/earbuds could facilitate biometric measurements
Next articleHow to Use YouTube on Samsung Galaxy Watch Devices
Sudz Niel Kar
I am a technologist with years of experience with Apple and wearOS products, have a BS in Computer Science and an MBA specializing in emerging tech, and owned the popular site AppleToolBox. In my day job, I advise Fortune 500 companies with their digital transformation strategies and also consult with numerous digital health startups in an advisory capacity. I'm VERY interested in exploring the digital health and fitness-tech evolution and keeping a close eye on patents, FDA approvals, strategic partnerships, and developments happening in the wearables and digital health sector. When I'm not writing or presenting, I run with my Apple Watch Ultra or Samsung Galaxy Watch and closely monitor my HRV and other recovery metrics.


Please enter your comment!
Please enter your name here

This site uses Akismet to reduce spam. Learn how your comment data is processed.