Estimating Nutritional Composition from Food Volume Via Deep Learning-Based Depth and Segmentation Models

Abstract

Nutrition plays a critical role in human health, with a balanced diet being essential for preventing non-communicable diseases, enhancing immune function, and improving quality of life. However, dietary imbalances contribute to significant global health issues, including obesity and malnutrition, which have far-reaching economic and health consequences. This research aims to address these challenges by developing a method for estimating the nutritional content of food items from a single 2D image. Our approach integrates a U-Net architecture with a ResNet18 encoder for depth prediction and employs FoodSAM for precise food segmentation. These components enable the calculation of food volume and mass, which are then used to estimate nutritional content based on the USDA database. Experimental results show that our model achieves a mean relative error (MRE) ranging from 11.18% to 50.35% for individual food items. Furthermore, our method maintains consistent mass predictions across various scenarios, including complex food combinations. This method demonstrates robustness in handling foods with diverse shapes and colors, providing a solid foundation for practical dietary tracking applications. By enabling nutritional monitoring, our approach has the potential to support public health initiatives and promote healthier lifestyles.