1. Proceedings
  2. I3M
  3. 2022
  4. EMSS
  5. 10.46354/i3m.2022.emss.029

Mobile crane signalman static hand signals classification framework using deep convolution neural network

  • a Asif Mansoor ,
  • b Shuai Liu, 
  • c Ghulam Muhammad Ali,
  • d Ahmed Bouferguene, 
  • e Mohamed Al-Hussein,
  • f Soda
  • a,b,c,e Department of Civil and Environmental Engineering, University of Alberta, 9105 116 street, Edmonton, T6H 2W2,
    Canada
  • d Campus Saint-Jean, University of Alberta, Edmonton, T6C 4G9, Canada
  • f Department of Computer Science, University of Turbat, Pakistan
Cite as
Mansoor A., Liu S., Ali G.M, Bouferguene A., Al-Hussein M.and Soda (2022).,Mobile crane signalman static hand signals classification framework using deep convolution neural network. Proceedings of the 34th European Modeling & Simulation Symposium (EMSS 2022). , 029 . DOI: https://doi.org/10.46354/i3m.2022.emss.029

Abstract

Cranes are the need of every construction site as the construction paradigm is shifting from traditional (on-site) methods toward an off-site (modularization) approach. The communication between the crane operator and the crane signalman plays a significant role to complete the construction project safely and efficiently. The communication between crane operator and signalman relies on hand signals and two-way radio communication systems. However, these means of communication are not reliable in modern construction as the construction sites are more congested and noisy. Any miscommunication may lead to a disastrous accident on the construction site. The recent advancement in information technology can assist to add more layers of communication in the crane Industry. This paper presents a framework that uses deep convolutional neural network (DCNN) architecture for static hand signal classification using the crane signalman hand signals dataset. The DCNN model was developed to classify 18 different crane signalman hand signals. The model was trained, validated, and tested using a dataset of 8133 images, and achieved average accuracies of 89.1% and 84.6% for the training dataset and the validation dataset, respectively. The precision, recall, and F1 score in the test dataset were recorded as 81.5%, 81.8%, and 81.7%, respectively. The model is further validated with real-time hand signals classification and an accuracy of 87.9% is achieved. This developed framework can be used as another layer of communication with the current state of practice to reduce the communication error between crane signalman and operator.

References

  1. Arabi, S., Haghighat, A., and Sharma, A. (2019). “Adeep learning basedsolution for constructionequipment detection: from development todeployment.”http://arxiv.org/abs/1904.09021.
  2. Cha, Y. J., Choi, W., and Büyüköztürk, O. (2017). “DeepLearning-Based Crack Damage Detection UsingConvolutional Neural Networks.”Computer-AidedCivil and Infrastructure Engineering, 32(5),361–378.
  3. Chatfield, K., Simonyan, K., Vedaldi, A., andZisserman, A. (2014). “Return of the devil in thedetails: Delving deep into convolutional nets.” BMVC 2014-Proceedings of the British MachineVisionConference 2014, 1–11
  4. Everett, B. J. G., and Slocum, A. H. (1993). “Device forImproving Crane.”Journal of ConstructionEngineering and Management,119(1), 23–39.
  5. Fang, Y., and Cho, Y. K. (2016). “A framework of liftvirtual prototyping (LVP) approach for crane safetyplanning.”ISARC 2016-33rd InternationalSymposium on Automation and Robotics inConstruction, 291–297.
  6. Gu, Y., Xu, S., Wang, Y., and Shi, L. (2019). “Anadvanced deep learning approach for safety helmetwearing detection.”Proceedings-2019 IEEEInternational Congress on Cybermatics: 12th IEEEInternational Conference on Internet of Things, 15thIEEE International Conference on Green Computingand Communications, 12th IEEE InternationalConference on Cyber, Physical and So, 669–674.
  7. Hu,J., Geo, X., Wu, H., and Gao, S. (2019). “Detectionof Workers Without the Helments in VideosBased on YOLO V3.”12th International Congress onImage and Signal Processing, BioMedicalEngineering and Informatics, 1553–1560.
  8. LeCun, Y., and Bengio, Y. (1995). “ConvolutionalNetworks for Images, Speech, andTime-Series.”Handb. brain Theory Neural Netw. 3361 (10).
  9. Li, H., Chan, G., and Skitmore, M. (2013). “Integratingreal time positioning systems to improve blind lifting and loading crane operations.”ConstructionManagement and Economics, 31(6), 596–605.
  10. Mansoor, A., Liu, S., Ali, G. M., Bouferguene, A., & Al-Hussein, M. (2020). Conceptual Framework forSafety Improvement in Mobile Cranes.InConstruction Research Congress 2020: ComputerApplications(pp. 964-971). Reston, VA: AmericanSociety of Civil Engineers.
  11. Nair, V., & Hinton, G. E. (2010). Rectified linear unitsimprove restricted boltzmann machines.
    In:Proceedings of the International Conference onMachine Learning (ICML) 807–814
  12. Occupational Safetyand Health Administration(OSHA), Cranes and Derricks in Construction(1926.1408),https://open.alberta.ca/dataset/757fed78-8793-40bb-a920-6f000853172b/resource/9296e033-fd12-40dc-ac86-21e5873d4161/download/4403880-part-6-cranes-hoists-and-lifting-devices.pdf (2009) (accessedDecember 19, 2021)
  13. Rosenfeld,Y. (1995). “Automation of existing cranes:from concept to prototype.”Automation inConstruction, 4(2), 125–138.
  14. Shapira, A., Rosenfeld, Y., and Mizrahi, I. (2008).“Vision system for tower cranes.”Journal ofConstruction Engineering and Management, 134(5),320–332.
  15. Srivastava, N., Hinton, G., Krizhevsky, A., Sutskever,I., & Salakhutdinov, R. (2014). Dropout: a simpleway to prevent neural networks fromoverfitting.The journal of machine learningresearch,15(1), 1929-1958
  16. Wang, L., Xie, L., Yang, P.,Deng, Q., Du, S., and Xu, L.(2020). “Hardhat-wearing detection based on alightweight convolutional neural network withmulti-scale features and a top-down module.”Sensors (Switzerland), 20(7), 3–7.
  17. Wu, J., Cai, N., Chen, W., Wang, H., and Wang, G.(2019). “Automatic detection of hardhats worn byconstruction personnel: A deep learning approachand benchmark dataset.”Automation inConstruction, Elsevier, 106.
  18. Yin, X., Chen, Y., Bouferguene, A., Zaman, H.,Al-Hussein, M., and Kurach, L. (2020). “A deeplearning-based framework for an automated defectdetection system for sewer pipes.”Automation inConstruction, Elsevier, 109(2019), 102967.
  19. Zavichi, A., and Behzadan, A. H. (2011). “A Real TimeDecision Support System for Enhanced CraneOperations in Construction and Manufacturing.”Computing in Civil Engineering, 194–201.
  20. Zekavat, P. R., Moon, S., and Bernold, L. E. (2014).“Holonic construction management: Unifiedframework for ICT-supported process control.”Journal of Management in Engineering, 31(1), 1–15.