Publications

Katsamenis I., Bimpas M., Protopapadakis E.,  Zafeiropoulos Ch., Kalogeras D.,  Doulamis A.,  Doulamis N., Montoliu C. M-P.,  Handanos Y., Schmidt F.,  Ott L.,  Cantero M.,  Lopez R. (2022).  Robotic Maintenance of Road Infrastructures: The HERON Project. Proceedings of the 15th International Conference on PErvasive Technologies Related to Assistive Environments, Corfu, Greece.

https://dl.acm.org/doi/10.1145/3529190.3534746

Abstract: Of all public assets, road infrastructure tops the list. Roads are crucial for economic development and growth, providing access to education, health, and employment. The maintenance, repair, and upgrade of roads are therefore vital to road users’ health and safety as well as to a well-functioning and prosperous modern economy. The EU-funded HERON project will develop an integrated automated system to adequately maintain road infrastructure. In turn, this will reduce accidents, lower maintenance costs, and increase road network capacity and efficiency. To coordinate maintenance works, the project will design an autonomous ground robotic vehicle that will be supported by autonomous drones. Sensors and scanners for 3D mapping will be used in addition to artificial intelligence toolkits to help coordinate road maintenance and upgrade workflows.

Zafeiropoulos Ch., Protopapadakis E., Chatzidaki A., Doulamis A., Vamvatsikos D., Zotos N., Bogdos G., Kostaridis A., Schmidt F., Ientile S., Sevilla I., Tilon S., Rallis I. (2022).  A holistic monitoring scheme for road infrastructures. Proceedings of the 15th International Conference on PErvasive Technologies Related to Assistive Environments, Corfu, Greece.

https://dl.acm.org/doi/abs/10.1145/3529190.3534745

Abstract: This monitoring system aims at increasing the resilience of the road infrastructures and ensuring reliable network availability under unfavourable conditions, such as extreme weather, landslides, and earthquakes. The main target is to combine downscaled climate change scenarios (applied to road infrastructures) with simulation tools (structural/geotechnical) and actual data (from existing and novel sensors), so as to provide the operators with an integrated tool able to support more effective management of their infrastructures at planning, maintenance and operation level. Towards this, the proposed framework aims to use high resolution modelling data for the determination and the assessment of the climatic risk of the selected, transport infrastructures and associated expected damages, use existing SHM data (from accelerometers, strain gauges etc.) with new types of sensor-generated data (computer vision) to feed the structural/geotechnical simulator, utilize tailored weather forecasts (combining seamlessly all available data sources) for specific hot-spots, providing early warnings with corresponding impact assessment in real time; develop improved multi-temporal, multi-sensor UAV, computer vision and machine learning-based damage diagnostic for diverse transport infrastructures; design and implement a Holistic Resilience Assessment Platform environment as an innovative planning tool that will permit a quantitative resilience assessment through an end-to-end simulation environment, running “what-if” impact/risk/resilience assessment scenarios. The effects of adaptation measures can be investigated by changing the hazard, exposure and vulnerability input parameters; design and implement a Common Operational Picture, including an enhanced visualisation interface and an Incident Management System. The integrated platform (and its sub-modules) will be validated in two real case studies in Spain and in Greece.

Katsamenis I., Karolou E.E., Davradou A., Protopapadakis E., Doulamis A., Doulamis N., Kalogeras D. (2022).  TraCon: A Novel Dataset for Real-Time Traffic Cones Detection Using Deep Learning. Proceedings of the 2nd International Conference (NiDS 2022), Athens, Greece.

https://doi.org/10.1007/978-3-031-17601-2_37

Abstract: Substantial progress has been made in the field of object detection in road scenes. However, it is mainly focused on vehicles and pedestrians. To this end, we investigate traffic cone detection, an object category crucial for road effects and maintenance. In this work, the YOLOv5 algorithm is employed, in order to find a solution for the efficient and fast detection of traffic cones. The YOLOv5 can achieve a high detection accuracy with the score of IoU up to 91.31%. The proposed method is been applied to an RGB roadwork image dataset, collected from various sources.

Katsamenis I., Davradou A., Karolou E.E., Protopapadakis E., Doulamis A., Doulamis N., Kalogeras D. (2022).  Evaluating YOLO Transferability Limitation for Road Infrastructures Monitoring. Proceedings of the 2nd International Conference (NiDS 2022), Athens, Greece.

https://doi.org/10.1007/978-3-031-17601-2_34

Abstract: Road infrastructure is positively associated with a country’s socio-economic growth and therefore road maintenance is of great importance for every country. One of the critical maintenance steps is road damage detection, which typically requires large amounts of time and high costs. In this work, the YOLOv5 two-stage detector is leveraged, in order to create an image-based solution for road defect detection and classification. The damages are classified into three main categories: cracks, potholes, and blurred markings. The YOLOv5 can achieve a relatively high detection accuracy with a score of Intersection over Union (IoU) up to 88.89% and classification accuracy with an F1 score up to 80.72%. The precision and recall scores are 84.26% and 78.38%, respectively.

Burnreiter L., Khattak S., Ott L., Siegwart R., Hutter M., Cadena C.. (2022).  Collaborative Robot Mapping using Spectral Graph Analysis. Proceedings of the 2022 International Conference on Robotics and Automation (ICRA), Philadelphia, USA.

https://ieeexplore.ieee.org/document/9812102

Abstract: In this paper, we deal with the problem of creating globally consistent pose graphs in a centralized multi-robot SLAM framework. For each robot to act autonomously, individual onboard pose estimates and maps are maintained, which are then communicated to a central server to build an optimized global map. However, inconsistencies between onboard and server estimates can occur due to onboard odometry drift or failure. Furthermore, robots do not benefit from the collaborative map if the server provides no feedback in a computationally tractable and bandwidth-efficient manner. Motivated by this challenge, this paper proposes a novel collaborative mapping framework to enable accurate global mapping among robots and server. In particular, structural differences between robot and server graphs are exploited at different spatial scales using graph spectral analysis to generate necessary constraints for the individual robot pose graphs. The proposed approach is thoroughly analyzed and validated using several real-world multi-robot field deployments where we show improvements of the onboard system up to 90%.

Foster J., Ott L., Nieto J., Lawrance N., Siegwart R. (2023).  Automatic Extension of a Symbolic Mobile Manipulation Skill Set. Robotics and Autonomous Systems.

https://doi.org/10.1016/j.robot.2023.104428

Abstract: Symbolic planning can provide an intuitive interface for non-expert users to operate autonomous robots by abstracting away much of the low-level programming. However, symbolic planners assume that the initially provided abstract domain and problem descriptions are closed and complete. This means that they are fundamentally unable to adapt to changes in the environment or tasks that are not captured by the initial description. We propose a method that allows an agent to automatically extend the abstract description of its skill set upon encountering such a situation. We introduce strategies for generalizing from previous experience, completing sequences of key actions and discovering preconditions to ensure computational efficiency. The resulting system is evaluated on a symbolic planning benchmark task and on object rearrangement tasks in simulation. Compared to a Monte Carlo Tree Search baseline, our strategies for efficient search have on average a 25% higher success rate at a 67% faster runtime.

Katsamenis I., Sakelliou A., Bakalos N., Protopapadakis E., Klaridopoulos C., Grangakis N., Bimpas M., Kalogeras D.  (2023).  Deep transformer networks for precise pothole segmentation tasks. Proceedings of the 16th International Conference on PErvasive Technologies Related to Assistive Environments.

https://doi.org/10.1145/3594806.3596560

Abstract: Potholes on the road surface are a significant safety hazard and can cause severe damage to vehicles. Identifying and repairing potholes is a challenging task that requires efficient and accurate methods. In recent years, deep learning models, such as U-Nets and transformers, have been used for image segmentation tasks with promising results. This paper proposes a transformer-based model and in particular the SegFormer framework, for pothole segmentation using high-resolution images captured from a road inspection vehicle. The proposed network outperformed the traditional U-Net model that demonstrates state-of-the-art performance in various segmentation tasks, achieving an average F1-score close to 80%. The results show that the proposed method can effectively identify and localize potholes, providing a useful auxiliary tool for road maintenance and safety.

Katsamenis I., Protopapadakis E., Bakalos N., Varvarigos A., Doulamis A., Doulamis N., Voulodimos A. (2023).  A Few-Shot Attention Recurrent Residual U-Net for Crack Segmentation. In: Bebis, G., et al. Advances in Visual Computing. ISVC 2023. Lecture Notes in Computer Science, vol 14361.

https://doi.org/10.1007/978-3-031-47969-4_16

Abstract: Recent studies indicate that deep learning plays a crucial role in the automated visual inspection of road infrastructures. However, current learning schemes are static, implying no dynamic adaptation to users’ feedback. To address this drawback, we present a few-shot learning paradigm for the automated segmentation of road cracks, which is based on a U-Net architecture with recurrent residual and attention modules (R2AU-Net). The retraining strategy dynamically fine-tunes the weights of the U-Net as a few new rectified samples are being fed into the classifier. Extensive experiments show that the proposed few-shot R2AU-Net framework outperforms other state-of-the-art networks in terms of Dice and IoU metrics, on a new dataset, named CrackMap, which is made publicly available at https://github.com/ikatsamenis/CrackMap.

Bakalos N., et al. (2024).  Robotics-enabled roadwork maintenance and upgrading. In: Robotics and Automation Solutions for Inspection and Maintenance in Critical Infrastructures. Now Publishers

https://doi.org/10.3929/ethz-b-000666060

Abstract: Closing, this chapter presents recent research efforts and results on a Robotics-enabled Roadwork Maintenance and Upgrading approach and tools. This includes the implementation of a road infrastructure blueprint developed including advanced engineering solutions for interconnecting and facilitating seamless transitions between different transportation modes in the event of severe disruptions affecting one mode of transportation.

Andreoli G., Schmidt F., Katsamenis I., Bakalos N., Oleynikova H., et al.. (2024) H2020 Project HERON. 10th conference Transport Research Arena (TRA 2024), Dublin, Ireland.

https://univ-eiffel.hal.science/hal-04575144

Abstract: SCOPE : HERON is among the projects funded by the EU under the H2020 program for road infrastructure maintenance. Within HERON’s scope is the implementation of a road infrastructure plan, CONTEXT : The Forever Open Road (FOR) concept has been developed, with its national declinations (R5G in France for example) by the association of European Road Research Laboratories (FEHRL). HERON answers the three concepts of FOR, namely the resilient, the automated and the adaptable road (www.foreveropenroad.eu/). As outlined by FOR, HERON integrates existing technical components, including off-the-shelf technologies and outcomes from previous or ongoing research projects. GOAL : The goal of HERON is to develop an integrated automated system capable of conducting various road maintenance and improvement tasks, such as crack sealing, pothole patching, asphalt rejuvenation, autonomous replacement of Removable Urban Pavement (RUP) elements, and road marking/painting. Thus, HERON is tasked with supporting both pre- and post-intervention phases, which involve automated and controlled visual inspections, as well as the placement and removal of traffic cones.

Katsamenis I., Andreoli G., Skamantzari M.,  Bakalos N., Schmidt F., et al.. (2024) UAV-based Localization of Removable Urban Pavement Elements Through Deep Object Detection Methods. Proceedings of the 17th International Conference on PErvasive Technologies Related to Assistive Environments.

https://doi.org/10.1145/3652037.3663934

Abstract: We introduce a deep learning framework leveraging YOLOv8 architecture to automate the localization of Removable Urban Pavements (RUPs) using UAV imagery. The core idea behind RUPs is to provide pavements that can be quickly opened and closed using lightweight on-site equipment. This approach aims to efficiently restore the street’s original appearance and functionalities within a short timeframe, typically just a few hours. Our study explores the feasibility of autonomously localizing RUP elements, paving the way for robotic-driven replacement with prefabricated, fully functional components. Moreover, the integration of UAV data enhances safety and accessibility to challenging areas. Experimental results underscore the efficacy of our approach in achieving precise localization and thereby enabling proactive maintenance efforts.