Visual Analysis of Agricultural Workers using Explainable Artificial Intelligence (XAI) on Class Activation Map (CAM) with Characteristic Point Data Output from OpenCV-based Analysis



Abstract Views
438

Downloads
377

Citations

Share

##plugins.themes.bootstrap3.article.sidebar##

Submitted Jan 5, 2023
Published Feb 8, 2023
10.24018/ejai.2023.2.1.16

Abstract viewed = 438 times

Table of Contents

##plugins.themes.bootstrap3.article.main##

In this study, we use explainable artificial intelligence (XAI) based on class activation map (CAM) techniques. Specifically, we use Grad-CAM, Grad-CAM++, and ScoreCAM to analyze outdoor physical agricultural (agri-) worker image datasets. In previous studies, we developed body-sensing systems to analyze human dynamics with the aim of enhancing agri-techniques, training methodologies, and worker development. These include distant, visual data-based sensing systems that capture image and movie datasets related to agri-worker motion and posture. For this study, we first obtained the aforementioned image datasets for researcher review. Then, we developed and executed Python programs with Open-Source Computer Vision (OpenCV) libraries and PyTorch to run XAI-oriented systems based on CAM techniques and obtained heat map-pictures of the visual explanations. Besides, we implement optical flow-based image analyses using our Visual C++ programs with OpenCV libraries, automatically set and chase the characteristic points related to the video datasets. Next, we analyze the dataset features and compare experienced and inexperienced subject groups. We investigate the output’s features, accuracies, and robustness to be able to make recommendations for real agri-workers, managers, product-developers, and researchers. Our findings indicate that the visualized output datasets are especially useful and may support further development of applied methods for these groups.

Keywords: agricultural worker visual data class activation map–based techniques (Grad-CAM, Grad-CAM , and ScoreCAM) explainable artificial intelligence OpenCV

References

  1. Linardatos P, Papastefanopoulos V, Kotsiantis S. Explainable ai: A review of machine learning interpretability methods. Entropy. 2020; 23(1), 18: 1–45.
     Google Scholar
  2. Hariharan S, Rejimol Robinson, RR, Prasad RR, Thomas C, Balakrishnan N. XAI for intrusion detection system: comparing explanations based on global and local scope. Journal of Computer Virology and Hacking Techniques. 2022; 1–23.
     Google Scholar
  3. Das A, Rad P. Opportunities and challenges in explainable artificial intelligence (xai): A survey. arXiv preprint arXiv:2006. 2020; 11371.
     Google Scholar
  4. Vollert S, Atzmueller M, Theissler A. Interpretable Machine Learning: A brief survey from the predictive maintenance perspective. Proceedings of 26th IEEE international conference on emerging technologies and factory automation (ETFA). 2021; 1–8.
     Google Scholar
  5. Islam SR, Eberle W, Ghafoor SK, Ahmed M. Explainable artificial intelligence approaches: A survey. arXiv preprint arXiv:2101.09429. 2021.
     Google Scholar
  6. Dindorf C, Konradi J, Wolf C, Taetz B, Bleser G, Huthwelker J, Fröhlich M. Classification, and automated interpretation of spinal posture data using a pathology-independent classifier and explainable artificial intelligence (Xai). Sensors. 2021; 21(18): 23–63.
     Google Scholar
  7. Galhotra S, Pradhan R, Salimi B. Explaining black-box algorithms using probabilistic contrastive counterfactuals. Proceedings of the 2021 International Conference on Management of Data. 2021; 577–590.
     Google Scholar
  8. Bücker M, Szepannek G, Gosiewska A, Biecek P. Transparency, auditability, and explain ability of machine learning models in credit scoring. Journal of the Operational Research Society. 2022; 73(1): 70–90.
     Google Scholar
  9. Goodwin NL, Nilsson SR, Choong JJ, Golden SA. Toward the explain ability, transparency, and universality of machine learning for behavioral classification in neuroscience. Current Opinion in Neurobiology. 2022; 73: 102544.
     Google Scholar
  10. Ferreira LA, Guimarães FG, Silva R. Applying genetic programming to improve interpretability in machine learning models. Proceedings of 2020 IEEE congress on evolutionary computation (CEC). 2020; 1–8.
     Google Scholar
  11. Li XH, Cao CC, ShiY, Bai W, Gao H, Qiu L, Chen L. A survey of data-driven and knowledge-aware explainable ai. IEEE Transactions on Knowledge and Data Engineering. 2020; 34(1): 29–49.
     Google Scholar
  12. Saeed W, Omlin C. Explainable ai (xai): A systematic meta-survey of current challenges and future opportunities. arXiv preprint arXiv:2111.06420. 2021.
     Google Scholar
  13. Duval A. Explainable artificial intelligence (XAI). MA4K9 Scholarly Report, Mathematics Institute, The University of Warwick. 2019; 1–53.
     Google Scholar
  14. Taylor JET, Taylor GW. Artificial cognition: How experimental psychology can help generate explainable artificial intelligence. Psychonomic Bulletin & Review. 2021; 28(2): 454–475.
     Google Scholar
  15. Lai V, Chen C, Liao QV, Smith-Renner A, Tan C. Towards a science of human-ai decision making: a survey of empirical studies. arXiv preprint arXiv:2112.11471. 2021.
     Google Scholar
  16. Patil PA, Jagyasi BG, Raval J, Warke N, Vaidya PP. Design, and development of wearable sensor textile for precision agriculture. Proceedings of IEEE 7th International Conference on Communication Systems and Networks (COMSNETS). 2015; 1–6.
     Google Scholar
  17. Kawakura S, Shibasaki R. Movement Analysis of Agricultural Workers’ Visual Data by Functions of OpenCV Focusing on Items Related to Human Dynamics. Journal of Advanced Agricultural Technologies. 2017; 4(3): 260–269.
     Google Scholar
  18. Kawakura S, Shibasaki R. Visual Data Analysis Methods Using OpenCV Programs to Evaluate Walking and Falling with a Japanese Walking Support System. International Journal of Bioscience, Biochemistry and Bioinformatics. 2017; 8(1): 20–29.
     Google Scholar
  19. Kawakura S, Hirafuji M, Shibasaki R. Development of Automatic Posture and Stumbling Judgement System using Deep Learning, Jetson Nano and Drone with Information-Sharing Function. Smart Innovation, Systems and Technologies series of Springer books, CRC press (Springer), 2022; Not published yes.
     Google Scholar
  20. Kawakura S, Hirafuji M, Ninomiya S, Shibasaki R. Analyses of Diverse Agricultural Worker Data with Explainable Artificial Intelligence: XAI based on SHAP, LIME, and LightGBM”, European Journal of Agriculture and Food Sciences. 2022; 4(6): 11–19.
     Google Scholar
  21. Visual explanations GradCAM / GradCAM++ / ScoreCAM | Kaggle: https://www.kaggle.com/code/itoeiji/visual-explanations-gradcam-gradcam-scorecam/notebook.
     Google Scholar