{"id":15,"date":"2016-11-01T15:40:25","date_gmt":"2016-11-01T15:40:25","guid":{"rendered":"http:\/\/www.heron-h2020.eu\/?page_id=15"},"modified":"2025-07-09T09:50:59","modified_gmt":"2025-07-09T09:50:59","slug":"services","status":"publish","type":"page","link":"https:\/\/www.heron-h2020.eu\/?page_id=15","title":{"rendered":"Project"},"content":{"rendered":"

Project Summary<\/h2><\/div>
<\/div><\/div>

HERON aims to develop an integrated automated system to perform maintenance and upgrading roadworks, such as sealing cracks, patching potholes, asphalt rejuvenation, autonomous replacement of CUD elements and painting markings, but also supporting the pre\/post-intervention phase including visual inspections and dispensing and removing traffic cones in an automated and controlled manner. The HERON system consists of:<\/p>\n

i) autonomous ground robotic vehicle that will be supported by autonomous drones to coordinate maintenance works and the pre-\/post-intervention phase;<\/p>\n

ii) various robotic equipment, including sensors and actuators (e.g., tools for cut ad fill, surface material placement and compaction, modular components installation, laser scanners for 3D mapping) placed on the main vehicle;<\/p>\n

iii) sensing interface installed both to the robotic platform and to the Road Infrastructures (RI) to allow improved monitoring (situational awareness) of the structural, functional and RI\u2019s and markings\u2019 conditions;<\/p>\n

iv) the control software that interconnects the sensing interface with the actuating robotic equipment;<\/p>\n

v) Augmented Reality (AR) visualization tools that enable the robotic system to see in detail surface defects and markings under survey;<\/p>\n

vi) Artificial Intelligence\/AI-based toolkits that will act as the middleware of a twofold role for: a) optimally coordinating the road maintenance\/upgrading workflows and b) intelligent processing of distributed data coming from the vehicle and the infrastructure sensors for safe operations and not disruption of other routine operations or traffic flows; and<\/p>\n

vii) integrate all data in an enhanced visualization user interface supporting decisions and<\/p>\n

viii) communication modules to allow for Vehicle-to-Infrastructure\/-Everything (V2I\/X) date exchange for predictive maintenance and increase users safety.<\/p>\n

HERON aims to reduce fatal accidents, maintenance costs, traffic disruptions, thus increasing the network capacity and efficiency.<\/p>\n<\/div>

<\/div><\/div><\/div>
<\/span><\/div><\/div><\/div><\/div><\/div>
<\/div>
<\/div><\/div>
<\/div><\/div><\/div><\/div><\/div><\/div><\/div>
<\/span><\/div><\/div><\/div>

Concept<\/h2><\/div>
<\/div><\/div>

The (semi-) automated HERON system relying on improved intelligent control of a multi-degree-of-freedom (MDOF) robotised vehicle, improved CV and AI\/ML techniques combined with proper sensors, decision-making algorithms and AR components to perform corrective and preventive maintenance and upgrading of roadworks is considered an advanced solution, which pushes routine roadwork activities quite beyond the state-of-the-art. At the same time, by using advanced data coming from various sources (including V2I and aerial drone surveillance) and well-established methods (from existing know-how from research and industrial projects), the automated system will be able to provide some non-routine (emergency) maintenance operations when required. Towards that direction, HERON targets the development and prototype validation of an innovative, automated intelligent robotic platform for performing the above tasks safely, promptly, reliably and modularly.<\/p>\n<\/div>

<\/div><\/div><\/div><\/div><\/div>
<\/div>
<\/div><\/div>
<\/div><\/div><\/div><\/div><\/div><\/div><\/div>

Objectives<\/h2><\/div>
<\/div><\/div>

HERON targets the following Scientific and Technical Objectives (STOs):<\/p>\n<\/div>

<\/i><\/i><\/span>STO-1: Develop an integrated robotic platform with increased navigation and positioning capabilities for maintenance and upgrading tasks and concurrent assessment of the RIs<\/span><\/a><\/h4><\/div>
\n

HERON improves Real-Time (RT) perception and cognition of the robotic vehicles, through Simultaneous Localisation and Mapping (SLAM) and novel Machine Learning (ML) techniques. SLAM will provide the concurrent mapping generation of the robotic vehicles’ space (complementary to GPS). The algorithms will be used for the navigation of the robotised vehicles allowing manoeuvres and avoiding collisions. The improved cognition will be based on recent advances in Computer Vision (CV) and ML for detecting objects, signaling, road markings and Point of Interests (Pols), interpreting events and actions, which occur in the area of intervention. Existing robotic vehicles will be used and enhances with all necessary sensors and cognition devices so to perform the maintenance and assessment of the RI.<\/p>\n<\/div><\/div><\/div>

<\/i><\/i><\/span>STO-2: Provide an optimized control framework for developing and refining robotic manipulation skills required to perform assessment and maintenance and upgrading interventions of RI<\/span><\/a><\/h4><\/div>
\n

This requires: a) specialized robotic equipment attached on the vehicle to take care of certain RI maintenance and upgrading works such as modular components installation, cut and filling, surface material placement and compaction, painting, spraying, and b) a control mechanism to transform the high-level processes into low-level robotic manipulation skills. These tools are in close relation with the robotic equipment programming algorithms and the sensing interface. Improved learning-based methods to acquire specialized low-level manipulation and interaction abilities will be developed (e.g., grasping, pulling, pushing, cutting, filling) through a combination of sim-to-real and user-provided demonstrations. These low-level abilities will be further refined into higher-order skills (e.g., pouring, levelling, etc.) by leveraging a symbolic representation framework. Using such representations provides access to powerful planning tools that can be used to generate high-level behaviours such as sealing cracks, patching potholes, painting markings, etc.<\/p>\n<\/div><\/div><\/div><\/div><\/div>

<\/div><\/div><\/div>
<\/span><\/div><\/div><\/div>

<\/i><\/i><\/span>STO-3: Integrate improved sensing and communication capabilities to the robotic platform in order to extract the required measurements in the identified areas of concern (e.g., poor or lack of markings, surface defects and potholes, dispensing and removing traffic cones, etc.) within an acceptable degree accuracy.<\/span><\/a><\/h4><\/div>
\n

In order to achieve a precise, automated maintenance and upgrading task, an accurate inspection of the PoI is needed, as well as wider knowledge of the surrounding area. This is done through the sensing interface installed on the robotic vehicle, the drones and on the RI. Surface 3D mapping and modelling5 are some of the salient parts of HERON sensors aiming at extracting precise details on 3D geometry of a road defect and thus stimulate the proper control mechanisms for its repair or the activation of pre-fabricated solutions (such as the Demountable Urban Roadway -CUD- elements) to modularize the planning and accelerate the maintenance and upgrading tasks. CV tools will be applied towards a traffic management for a safe flow and working conditions of the maintenance personnel (access to PANOPTIS outcomes2). Regarding the communication tools, HERON will support any module (e.g., 4G\/5G, WiMAX, BLE4, etc.) that will be integrated into the robotic vehicle (as in STO-5), allowing the robotised vehicle(s) to have different configurations for communicating to the each other, the field maintenance team and the control center.<\/p>\n<\/div><\/div><\/div>

<\/i><\/i><\/span>STO-4: Implement an AI toolkit enriched with image analysis modules to optimally coordinate the whole maintenance process and simultaneously process in a smart way the data from the sensing interface to take accurate and prompt decisions which can guarantee an unhindered execution of the traffic flow and a safe execution of routine maintenance works by the personnel.<\/span><\/a><\/h4><\/div>
\n

The AI tools will be developed on novel deep ML algorithms and receive as input information from all data sources, installed either on the robotic platform or in the RI or being inputted from external sources (proper V2I data exchange for predictive maintenance, Copernicus, etc.), fuse this information and extract high-level conceptual decisions on the maintenance process. In addition, image analysis and CV algorithms will be applied to allow a continuous traffic flow during the maintenance works and guarantee a safe execution of routine works on the RI by the expert-personnel. Convolutional Neural Networks (CNNs), Long Short-Term Memory Networks (LSTMs), and Generative Adversarial Networks (GANs) will be exploited as Deep Learning (DL) tools, which will be aligned with vision-based object detection algorithms, defect detection schemes and traffic flow analysis modules. Finally, HERON supports a distributed AI architecture including the widely publicized paradigm of Federated Learning6, which is the epitome of collaboration between edge nodes (Unmanned Ground Vehicles-UGVs) and cloud for meeting privacy and application performance requirements. HERON will focus on optimizing a cloud\/edge solution for a big class of robotic applications in RIs.<\/p>\n<\/div><\/div><\/div><\/div><\/div>

<\/div><\/div><\/div>

<\/i><\/i><\/span>STO-5: Design and develop proper communication architecture to support seamless and ubiquitous services among the various actors of the maintenance and upgrading operations (sensors, UGVs and drones).<\/span><\/a><\/h4><\/div>
\n

Robust communication will be developed for keeping the network organized even in case of a link\/node failure; the HERON network will be self-healing with reconfiguration around a broken path or alternative connection, ensuring continuous connectivity. Our proposed architecture aims to operate in a seamless and cooperative manner with existing human maintenance and technical crews of the roads, as well as integrating HERON to the existing maintenance\/upgrading mechanisms. It will act as the technological backbone to provide improved Data Fusion (DF), seamless interconnection and interoperability between the system layers minimising data ambiguity.<\/p>\n<\/div><\/div><\/div>

<\/i><\/i><\/span>STO-6: Implement an integrated Decision Support System (DSS) and an advanced Incident Management System (IMS) with interactive AR\/VR visualization tools.<\/span><\/a><\/h4><\/div>
\n

We will integrate all the information to be provided by the various tools, the sensor data, etc. as different layers in a unified enhanced visualisation User Interface (UI) generating the Common Operational Picture (COP), as a complementary support to the prevailing Transport Management Systems (TMS) and Closed-Circuit Television (CCTV). The IMS will provide the information of facilities, equipment, personnel, procedures and communications. Assisted by the DSS tool, IMS will manage seamlessly routine incidents as well as emergency situations in the RIs. The decisions and response procedures per reported incident will be turned to actions and resource proposals by building related incident process workflows to support the decision making of RI operators. The visualization will be enriched with AR components to allow for the user to have an in-situ supervision of the automated maintenance and upgrading operation process.<\/p>\n<\/div><\/div><\/div>

<\/i><\/i><\/span>STO-7: Implement on-site integration, scaled-demonstration of the services and the technological components and validation of the HERON platform in three case studies (Greece, France and Spain).<\/span><\/a><\/h4><\/div>
\n

The cost, features, benefits and reaction time of the proposed system and incurred procedures will be compared to conventional (pre-project) procedures and means of motorway operation.<\/p>\n<\/div><\/div><\/div><\/div><\/div>

<\/div><\/div><\/div><\/div><\/div>
<\/div>
<\/div><\/div>
<\/div><\/div><\/div><\/div><\/div><\/div><\/div>
<\/span><\/div>
<\/div>

Pilot 1<\/h2><\/div>

The pilot will be deployed in the A2 Motorway stretch maintained by the company (R2\u2013CM42
\nstretch, coming from Madrid, and finishing in the limit between the provinces of Guadalajara and Soria, Spain), and
\nin the traffic control centre of the stretch located near the village of Torija The motorway is owned by the Spanish
\nNational Road Authority and the section selected has a length of 77.5 km, The section has 4 lanes (2 per traffic
\ndirection) and crosses a region with Continental-Mediterranean climate, with long and severe winters, long, dry
\nand hot summers and high heavy traffic levels, so pavement is exposed to severe requirements and maintenance
\nis crucial to preserve the optimum pavement conditions required. A2 is one of the main motorways in Spain,
\nconnecting Madrid with Barcelona, it is part of the Trans-European Transport Network (TEN-T) and the CEF corridor.<\/p>\n<\/div><\/div><\/div>

<\/span><\/div>
<\/div>

Pilot 2<\/h2><\/div>

Transpolis is a proving ground of more than 80ha, which has been created by 5 entities among
\nwhich UGE and which has been opened officially in 2019. It is typically used to test autonomous vehicles in a secure
\nand controlled environment, also by assessing the V2I communication possibilities (several types of Road-Side-
\nUnits- RSUs- and communication means) are already installed on site. It also is composed of several kilometers of
\nroad and all reinforced concrete buildings. Many types of V2X and I2V (Infrastructure to Vehicle) communication
\nmeans are available, as well as camera monitoring, all of them will be used during the HERON activities.<\/p>\n<\/div><\/div><\/div>

<\/span><\/div>
<\/div>

Pilot 3<\/h2><\/div>

OLO has undertaken the traffic management and routine maintenance of the Elefsina-Korinthos-
\nPatra motorway (in the heart of the Greek highway networks), which has 202 km total length and includes more
\nthan 25 km tunnels and a large number of bridges, culverts and ancillary structures. It includes corrective and
\npreventive maintenance both of civil works equipment and Early Equipment Management (EEM) of open road and
\ntunnels. OLO will provide a part of the motorway, where extensive tests of the automated vehicle can take place,
\nissuing the necessary permits in cooperation with the relevant Authorities (Public Service and Traffic Police) and
\nensuring safety conditions both for the road users and the people working for the project. The area that will be
\nexamined during the pilot program is the ELKO section, which is a dual carriageway with three lanes (3.5m width
\nleft lane, 3.75m width middle and right lane) and emergency lane (varies from 2.5 to 3.5m) per direction with
\nconcrete New Jersey safety barriers in the central axis of the motorway. Some major technical features of ELKO
\nsection: \u2055 Total length: 64 km; \u2055 Interchanges (I\/C): 12; \u2055 Bridges: 16; \u2055 Tunnels: 5 (total length of 4,473 km).<\/p>\n<\/div><\/div><\/div><\/div><\/div>

<\/div>
<\/div><\/div>
<\/div><\/div><\/div><\/div><\/div><\/div><\/div>
Deliverables Progress<\/h5><\/div>
WP1<\/div><\/div><\/div>
WP2<\/div><\/div><\/div>
WP3<\/div><\/div><\/div>
WP4<\/div><\/div><\/div>
WP5<\/div><\/div><\/div>
WP6<\/div><\/div><\/div>
WP7<\/div><\/div><\/div>
WP8<\/div><\/div><\/div>
WP9<\/div><\/div><\/div><\/div>

Starting Date: 1st of June, 2021<\/p>\n<\/div><\/div><\/div><\/div><\/div><\/p>\n","protected":false},"excerpt":{"rendered":"","protected":false},"author":3,"featured_media":0,"parent":0,"menu_order":0,"comment_status":"closed","ping_status":"closed","template":"100-width.php","meta":{"footnotes":""},"class_list":["post-15","page","type-page","status-publish","hentry"],"_links":{"self":[{"href":"https:\/\/www.heron-h2020.eu\/index.php?rest_route=\/wp\/v2\/pages\/15","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.heron-h2020.eu\/index.php?rest_route=\/wp\/v2\/pages"}],"about":[{"href":"https:\/\/www.heron-h2020.eu\/index.php?rest_route=\/wp\/v2\/types\/page"}],"author":[{"embeddable":true,"href":"https:\/\/www.heron-h2020.eu\/index.php?rest_route=\/wp\/v2\/users\/3"}],"replies":[{"embeddable":true,"href":"https:\/\/www.heron-h2020.eu\/index.php?rest_route=%2Fwp%2Fv2%2Fcomments&post=15"}],"version-history":[{"count":95,"href":"https:\/\/www.heron-h2020.eu\/index.php?rest_route=\/wp\/v2\/pages\/15\/revisions"}],"predecessor-version":[{"id":1630,"href":"https:\/\/www.heron-h2020.eu\/index.php?rest_route=\/wp\/v2\/pages\/15\/revisions\/1630"}],"wp:attachment":[{"href":"https:\/\/www.heron-h2020.eu\/index.php?rest_route=%2Fwp%2Fv2%2Fmedia&parent=15"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}