Carbcounter: uma solução visual para contagem de carboidratos

Abstract

Type 1 Diabetes Mellitus (T1DM) is a chronic autoimmune disease characterized by a deficiency in insulin production, a hormone responsible for regulating blood glucose levels. Patients with T1DM need to administer insulin doses throughout the day, requiring them to monitor the amount of carbohydrates (CHO) consumed in meals to correctly calculate the doses. However, carbohydrate counting can be challenging for some individuals and very exhausting, requiring patient training to perform this task, and even then, estimation errors may occur, leading to cases of hypoglycemia and hyperglycemia. Thus, with the growth of the Computer Vision field, image-based solutions have emerged to assist patients in managing the disease, facilitating nutritional value calculations and reducing the complexity of this task. In this context, we propose in this Course Conclusion Project (TCC, in portuguese) the CarbCounter, an application for calorie and CHO counting using Large Multimodal Models (LMMs) to analyze meal images and describe the foods they contain along with their nutritional content. The results indicated good model performance, achieving median absolute errors of 7.2g of CHO and 63.4 kcal, using an automated approach with an average response time of 4s, thus providing a more efficient experience for patients to perform CHO counting. Therefore, the LMM-based approach has proven to be promising for estimating nutritional values from images, aiming to improve the quality of life not only for people with T1DM but also for anyone interested in monitoring their eating habits.