.png)
Multimodal Deep Learning and IoT Sensor Fusion for Real-Time Beef Freshness Detection
Abstract
Beef freshness quality is one of the important indicators in ensuring food safety and suitability. However, conventional methods such as manual visual inspection and laboratory testing cannot be widely applied in real-time and mass scale. To overcome these challenges, this study proposes a meat freshness detection system based on a multimodal approach that combines visual imagery and gas sensor data in a single IoT-based framework. This system is designed by utilizing the YOLOv11 architecture that has been optimized using the Adam optimizer. The dataset consisted of 540 original beef images, expanded into 1,296 images after augmentation. The model is trained on these augmented images and is able to achieve detection performance with a mAP@0.5 value of 99.4% and mAP@0.5:0.95 of 95.7%. As a further improvement, the cropped image features from the YOLOv11 model are processed through a combination of the ViT model and CNN to classify the level of meat freshness into three classes: Fresh, Medium, and Rotten with an accuracy of 99%. On the other hand, chemical data was obtained from the MQ136 and MQ137 gas sensors to detect H₂S and NH₃ levels which are indicators of meat spoilage. Data from visual and chemical data were then combined through a multimodal fusion method and classified using the Random Forest algorithm, producing a final prediction of Fit for Consumption, Need to Check, and Not Fit for Consumption. This multimodal model achieved a classification accuracy of 98% with a ROC-AUC score approaching 1.00 across all classes. While the proposed system achieved very high accuracy, further validation across diverse real-world environments is recommended to establish its generalizability.
Keywords
Full Text:
PDFReferences
S. N. Hadi and R. H. Chung, “Estimation of Demand for Beef Imports in Indonesia: An Autoregressive Distributed Lag (ARDL) Approach,” Agriculture, vol. 12, no. 8, pp. 1–12, 2022, doi: 10.3390/agriculture12081212.
S. A. Mehdizadeh, M. Noshad, M. Chaharlangi, and Y. Ampatzidis, “AI-driven non-destructive detection of meat freshness using a multi-indicator sensor array and smartphone technology,” Smart Agric. Technol., vol. 10, no. March, pp. 1–10, 2025, doi: https://doi.org/10.1016/j.atech.2025.100822.
M. Hashemi, M. Salayani, A. Afshari, H. S. Kafil, and S. M. A. Noori, “The Global Burden of Viral Food-borne Diseases: A Systematic Review,” Curr. Pharm. Biotechnol., vol. 24, no. 13, pp. 1657–1672, 2023, doi: https://doi.org/10.2174/1389201024666230221110313.
D. L. Prado, E. J. T. Dajang, and I. K. Machica, “FRESHNet : A CNN and YOLO - based Mobile App for Meat Freshness Assessment,” Food Chem., vol. 463, no. January, pp. 1-12, 2024, doi: 10.13140/RG.2.2.29355.21289.
S. Dalal, U. K. Lilhore, M. Radulescu, S. Simaiya, V. Jaglan, and A. Sharma, “A hybrid LBP-CNN with YOLO-v5-based fire and smoke detection model in various environmental conditions for environmental sustainability in smart city,” Environ. Sci. Pollut. Res., vol. 1, no. January, pp. 1-10, 2024, doi: 10.1007/s11356-024-32023-8.
Calvin, G. B. Putra, and E. Prakasa, “Classification of Chicken Meat Freshness using Convolutional Neural Network Algorithms,” 2020 Int. Conf. Innov. Intell. Informatics, Comput. Technol. 3ICT 2020, vol. 02, no. January, pp. 3–8, 2020, doi: 10.1109/3ICT51146.2020.9312018.
A. Arsalane, A. Klilou, and N. El Barbri, “Performance evaluation of machine learning algorithms for meat freshness assessment,” Int. J. Electr. Comput. Eng., vol. 14, no. 5, pp. 5858–5865, 2024, doi: 10.11591/ijece.v14i5.pp5858-5865.
R. Wahyuni, Herianto, Ikhtiyaruddin, and Y. Irawan, “IoT-Based Pulse Oximetry Design as Early Detection of Covid-19 Symptoms,” Int. J. Interact. Mob. Technol., vol. 17, no. 3, pp. 177–187, 2023, doi: 10.3991/ijim.v17i03.35859.
Y. Irawan, E. Sabna, A. F. Azim, R. Wahyuni, N. Belarbi, and M. M. Josephine, “Automatic Chili Plant Watering Based on Internet of Things (Iot),” J. Appl. Eng. Technol. Sci., vol. 3, no. 2, pp. 77–83, 2022, doi: 10.37385/jaets.v3i2.532.
Y. Irawan, A. W. Novrianto, and H. Sallam, “Cigarette Smoke Detection and Cleaner Based on Internet of Things (Iot) Using Arduino Microcontroller and Mq-2 Sensor,” J. Appl. Eng. Technol. Sci., vol. 2, no. 2, pp. 85–93, 2021, doi: 10.37385/jaets.v2i2.218.
Y. Irawan, R. Wahyuni, and H. Fonda, “Folding Clothes Tool Using Arduino Uno Microcontroller And Gear Servo,” J. Robot. Control, vol. 2, no. 3, pp. 170–174, 2021, doi: 10.18196/jrc.2373.
A. R. Abidin, Y. Irawan, and Y. Devis, “Smart Trash Bin for Management of Garbage Problem in Society”, JAETS, vol. 4, no. 1, pp. 202–208, Sep. 2022, doi: https://doi.org/10.37385/jaets.v4i1.1015.
A. N. Damdam, L. O. Ozay, C. K. Ozcan, A. Alzahrani, R. Helabi, and K. N. Salama, “IoT-Enabled Electronic Nose System for Beef Quality Monitoring and Spoilage Detection,” Foods, vol. 12, no. 11, pp. 1-10, 2023, doi: 10.3390/foods12112227.
Z. W. Bhuiyan, S. A. R. Haider, A. Haque, M. R. Uddin, and M. Hasan, “IoT Based Meat Freshness Classification Using Deep Learning,” IEEE Access, vol. 12, no. October, pp. 196047–196069, 2024, doi: 10.1109/ACCESS.2024.3520029.
Z. Xun, X. Wang, Hao. X, “Deep machine learning identified fish flesh using multispectral imaging,” Curr. Res. Food Sci., vol. 9, no. June, pp. 1-12, 2024, doi: 10.1016/j.crfs.2024.100784.
D. Setiawan, R. N. Putri, I. Fitri, A. N. Hidayanto, Y. Irawan, and N. Hohashi, “Improved Deep Learning Model for Prediction of Dermatitis in Infants,” J. Appl. Data Sci., vol. 6, no. 2, pp. 871–884, 2025, doi: 10.47738/jads.v6i2.542.
T. Anwar and H. Anwar, “Beef quality assessment using AutoML,” in 2021 Mohammad Ali Jinnah University International Conference on Computing (MAJICC), vol. 01, no. July, pp. 1–4, 2021, doi: 10.1109/MAJICC53071.2021.9526256.
J. M. Lee, I. H. Jung, and K. Hwang, “Classification of Beef by Using Artificial Intelligence,” J. Logist. Informatics Serv. Sci., vol. 9, no. 1, pp. 1–10, 2022, doi: 10.33168/liss.2022.0101.
E. N. Cahyo, E. Susanti, and R. Y. Ariyana, “Model Machine Learning Untuk Klasifikasi Kesegaran Daging Menggunakan Arsitektur Transfer Learning Xception,” J. Sist. dan Teknol. Inf., vol. 11, no. 2, p. 371, 2023, doi: 10.26418/justin.v11i2.57517.
A. Denih and I. Anggraeni, “Beef Freshness Detection Device Based on Gas and Color Sensors using the K- Nearest Neighbor Method,” Ind. Eng. Oper. Manag. Manila, vol. 7, no. March, pp. 2539–2545, 2023, doi: 10.46254/an13.20230690.
O. B. Yurdakos and O. Cihanbegendi, “System Design Based on Biological Olfaction for Meat Analysis Using E-Nose Sensors,” ACS Omega, vol. 9, no. 30, pp. 33183–33192, Jul. 2024, doi: 10.1021/acsomega.4c04791.
Y. Lin, J. Ma, D. W. Sun, J. H. Cheng, and C. Zhou, “Fast real-time monitoring of meat freshness based on fluorescent sensing array and deep learning: From development to deployment,” Food Chem., vol. 448, no. March, pp. 1-11, 2024, doi: 10.1016/j.foodchem.2024.139078.
J. Zhang, W. Jizhong, W. Wenya, “Olfactory imaging technology and detection platform for detecting pork meat freshness based on IoT,” Comput. Electron. Agric., vol. 215, no. December, pp. 1-10, 2023, doi: https://doi.org/10.1016/j.compag.2023.108384.
A. Yudhana, R. Umar, and S. Saputra, “Fish Freshness Identification Using Machine Learning: Performance Comparison of k-NN and Naïve Bayes Classifier,” J. Comput. Sci. Eng., vol. 16, no. 3, pp. 153–164, 2022, doi: 10.5626/JCSE.2022.16.3.153.
K. Kiswanto, H. Hadiyanto, and E. Sediyono, “Meat Texture Image Classification Using the Haar Wavelet Approach and a Gray-Level Co-Occurrence Matrix,” Appl. Syst. Innov., vol. 7, no. 3, pp. 1-10, 2024, doi: 10.3390/asi7030049.
I. H. Kozan and H. A. Akyurek, “Development Of A Mobile Application For Rapid Detection Of Meat Freshness Using Deep Learning,” Theory Pract. Meat Process., vol. 9, no. 3, pp. 249–257, 2024.
A. Febriani, R. Wahyuni, Y. Irawan, and R. Melyanti, “Improved Hybrid Machine and Deep Learning Model for Optimization of Smart Egg Incubator,” J. Appl. Data Sci., vol. 5, no. 3, pp. 1052–1068, 2024, DOI: https://doi.org/10.47738/jads.v5i3.304.
D.-W. Sun, H. Pu, and J. Yu, “Applications of hyperspectral imaging technology in the food industry,” Nat. Rev. Electr. Eng., vol. 1, no. 4, pp. 251–263, 2024, doi: 10.1038/s44287-024-00033-w.
A. Rabehi, H. Helal, D. Zappa, and E. Comini, “Advancements and Prospects of Electronic Nose in Various Applications: A Comprehensive Review,” Appl. Sci., vol. 14, no. 11, pp. 1-9, 2024, doi: 10.3390/app14114506.
S. Tang and W. Yan, “Utilizing RT-DETR Model for Fruit Calorie Estimation from Digital Images,” Information, vol. 15, no. 8, pp. 1-10, 2024, doi: 10.3390/info15080469.
E. Hassan and H. Ghadiri, “Advancing brain tumor classification: A robust framework using EfficientNetV2 transfer learning and statistical analysis,” Comput. Biol. Med., vol. 185, no. February, pp. 1-12, 2025, doi: https://doi.org/10.1016/j.compbiomed.2024.109542.
D. Rastogi, P. Johri, and V. Tiwari, “Augmentation based detection model for brain tumor using VGG 19,” Int. J. Comput. Digit. Syst., vol. 13, no. 1, pp. 1227–1237, 2023, doi: 10.12785/ijcds/1301100.
H. A. Akbaci and E. Bayraktar, “Trajectory refinement in SLAM: the impact of Adam, AdamW, and SGD with momentum,” in Proc.SPIE, vol. 13540, no. Feb, pp. 1-10, 2025, doi: 10.1117/12.3056417.
N. S.T. and J. V Gorabal, “Design and Development of Multimodal Biometric System Using Finger Veins and Iris by CNN Integrated with Hybrid SIO and Whale Optimization Techniques,” Int. J. Interact. Mob. Technol., vol. 18, no. 22, pp. 97–114, 2024, doi: 10.3991/ijim.v18i22.50865.
M. K. Anam, L. L. Van FC, H. Hamdani, R. Rahmaddeni, J. Junadhi, M. B. Firdaus, I. Syahputra, and Y. Irawan, “Sara Detection on Social Media Using Deep Learning Algorithm Development,” J. Appl. Eng. Technol. Sci., vol. 6, no. 1, pp. 225–237, Dec. 2024, doi: 10.37385/jaets.v6i1.5390.
S. Tomar, D. Dembla, and Y. Chaba, “Analysis and Enhancement of Prediction of Cardiovascular Disease Diagnosis using Machine Learning Models SVM, SGD, and XGBoost,” Int. J. Adv. Comput. Sci. Appl., vol. 15, no. 4, pp. 469–479, 2024, doi: 10.14569/IJACSA.2024.0150449.
Y. Thanet, L. Potsirin, N. Wongpanya, and P. Nuankaew, “Information Systems for Cultural Tourism Management Using Text Analytics and Data Mining Techniques,” Int. J. Interact. Mob. Technol., vol. 16, no. 09, pp. 146–163, 2022, doi: 10.3991/ijim.v16i09.30439.
H. Fonda, Y. Irawan, R. Melyanti, R. Wahyuni, and A. Muhaimin, “A Comprehensive Stacking Ensemble Approach for Stress Level Classification in Higher Education,” J. Appl. Data Sci., vol. 5, no. 4, pp. 1701–1714, 2024, DOI: https://doi.org/10.47738/jads.v5i4.388.
K. Okoye, J. T. Nganji, J. Escamilla, and S. Hosseini, “Machine learning model (RG-DMML) and ensemble algorithm for prediction of students’ retention and graduation in education,” Comput. Educ. Artif. Intell., vol. 6, no. September, pp. 1-10, 2024, doi: 10.1016/j.caeai.2024.100205.
P. Nasa-Ngium, W. S. Nuankaew, and P. Nuankaew, “Analyzing and Tracking Student Educational Program Interests on Social Media with Chatbots Platform and Text Analytics,” Int. J. Interact. Mob. Technol., vol. 17, no. 05, pp. 4–21, 2023, doi: 10.3991/ijim.v17i05.31593.
Herianto, B. Kurniawan, Z. H. Hartomi, Y. Irawan, and M. K. Anam, “Machine Learning Algorithm Optimization using Stacking Technique for Graduation Prediction,” J. Appl. Data Sci., vol. 5, no. 3, pp. 1272–1285, 2024, DOI: https://doi.org/10.47738/jads.v5i3.316
A. Lubis, Y. Irawan, Junadhi, and S. Defit, “Leveraging K-Nearest Neighbors with SMOTE and Boosting Techniques for Data Imbalance and Accuracy Improvement,” J. Appl. Data Sci., vol. 5, no. 4, pp. 1625–1638, 2024, doi: 10.47738/jads.v5i4.343.
K. A. Rashedi, M. T. Ismail, S. Al Wadi, A. Serroukh, T. S. Alshammari, and J. J. Jaber, “Multi-Layer Perceptron-Based Classification with Application to Outlier Detection in Saudi Arabia Stock Returns,” J. Risk Financ. Manag., vol. 17, no. 2, pp. 1-12, 2024, doi: 10.3390/jrfm17020069.
H. Chen, J. Cui, Y. Zhang, and Y. Zhang, “VIT and Bi-LSTM for Micro-Expressions Recognition,” in 2022 IEEE 5th International Conference on Information Systems and Computer Aided Education (ICISCAE), vol. 9927522, no. September, pp. 946–951, 2022. doi: 10.1109/ICISCAE55891.2022.9927522.
A. B. Nassif, I. Shahin, M. Bader, A. Ahmed, and N. Werghi, “ViT-LSTM synergy: a multi-feature approach for speaker identification and mask detection,” Neural Comput. Appl., vol. 36, no. 35, pp. 22569–22586, 2024, doi: 10.1007/s00521-024-10389-7.
N. Zhou, M. Xu, “ViT-UNet: A Vision Transformer Based UNet Model for Coastal Wetland Classification Based on High Spatial Resolution Imagery,” IEEE J. Sel. Top. Appl. Earth Obs. Remote Sens., vol. 17, no. October, pp. 19575–19587, 2024, doi: 10.1109/JSTARS.2024.3487250.
A. R. Borah, A. A. Hameed, H. P. Thethi, J. L. Prasanna, A. Sangeetha, and D. D. Gautam, “ViT and RNN for Temporal and Spatial Analysis in Video Sequences,” in 2025 International Conference on Intelligent Control, Computing and Communications (IC3), vol. 2025, no. February, pp. 651–656, 2025, doi: 10.1109/IC363308.2025.10957553.
M. J. Zobair, M. A. Rahman, M. S. Hossain, N. Khan, M. A. A. K. Akash, and M. H. I. Bijoy, “A Hybrid ViT-GRU Model for Breast Cancer Detection: Addressing Class Imbalance Challenges,” in 2025 International Conference on Electrical, Computer and Communication Engineering (ECCE), vol. 2025, no. February, pp. 1–7, 2025, doi: 10.1109/ECCE64574.2025.11013095.
DOI: https://doi.org/10.47738/jads.v6i4.977
Refbacks
- There are currently no refbacks.
Journal of Applied Data Sciences
| ISSN | : | 2723-6471 (Online) |
| Publisher | : | Bright Publisher |
| Website | : | http://bright-journal.org/JADS |
| : | taqwa@amikompurwokerto.ac.id (principal contact) | |
| support@bright-journal.org (technical issues) |
This work is licensed under a Creative Commons Attribution-ShareAlike 4.0