Tunnel Inspection with Artificial Intelligence

1 Introduction

Every year 4700 km of new tunnels are built, resulting in an annual growth value of 7 %. Accordingly, the total amount of tunnels that must be inspected will increase in the future. Currently tunnel inspections are carried out mainly by
(visual) walkthrough inspections based on the available input data. This allows to focus on certain features and areas that need special attention. Experienced but sometimes not specially trained and educated staff carry out these walkthroughs, which may also include some detailed visual inspections. Often basic knowledge about the severity of certain damages and their development is missing.

During these walkthrough inspections notes and photographs are taken and data in general is collected. Afterwards, the data is integrated into a digital database (Fig.1) – partly it still remains available only on paper. It then serves as a basis for further inspection at regular intervals or as a trigger for additional inspections and measurements. In some cases, immediate maintenance or rehabilitation activities become necessary. Evaluation of the data for a holistic state assessment, damage development, maintenance etc. is done only to a limited extent and not in a digitalized manner, since it is very time-consuming and expensive.

After an inspection, the project owner needs answers to the following questions:

• What is the condition of my structure?

• What types of damages are present?

• Is there a threat to structural and operational safety?

• How much money must be budgeted for maintenance?

Periodical and systematic inspections should help to keep the overall costs low. They are necessary to identify the critical areas early and start with refurbishment work when the areas of damages are still small. Without regular inspections, the risk of a major incident increases, leading to more extensive refurbishment works.

2 Methodology of Tunnel Maintenance Work

2.1 State of the Art today

To maintain operational safety, serviceability and structural safety of a tunnel infrastructure as economically as possible, a mature value preservation concept must be implemented. The basis of the value preservation concept is mostly regular monitoring, which means regular surveying and inspections.

Preservation of tunnel infrastructures is divided into surveying and maintenance measures. The surveying part consists of inspections, visual and measurement control, function and material tests. That results in the condition assessment, which determines how the maintenance is carried out. Preventive, corrective or renewing actions can be taken, depending on the state of the tunnel. Maintenance management starts as soon as the tunnel construction is finished, with the first inspection to document the as-built state. The survey measures serve both as initial documentation of the condition and as continuous or periodic documentation of the condition development.

As of this date, inspections are scheduled every five years to record the state of the construction and to determne when maintenance work is needed. After an intervention, the new state is also inspected.

Inspections are fundamental for all focused management processes for structural value preservation, which must be based on detailed knowledge about the condition of the tunnel. During the surveying measures, the following technical data about the properties, characteristics and the condition of the tunnel must be collected:

• construction materials and their properties

• geometry and profile of the cavity

• installations for operation and clearance profile

• damages and deficiencies

• additional data: Construction method and tunnel age, geology and geologically induced forces, hydrogeology and chemical characteristics of the rock water, climatic condition, rolling stock and operation type

Inspection or surveying work is partly done on-site (Fig. 2). For safety reasons, these on-site works must be carried out during closure periods. This usually can be done only during night shifts and therefore is very expensive. It is crucial to minimize the time spent in the field as much as possible. This is facilitated by computer-aided tools for survey activities and data analysis. Carrying out a pre-inspection in the office using previously scanned images as the basis for the inspection software reduces the time needed in the field. Accordingly, the tunnel structure, materials, installations and damages that have been already registered need to be checked, corrected or added, once the inspection is lead through the tunnel. The data collected and analysed is the basis for the tunnel state assessment and future planning. Damage categories are for example deformations, cavities, cracks, water inflow, corrosion etc. or any combination of these features.

After a complete collection of the damages data and deficiencies, each zone is categorized regarding its condition into one of the following classes:

1) good

2) sufficient

3) insufficient

4) aggravating

5) alarming

After classification, the management approach has to be defined (preventive maintenance, corrective maintenance or renewal) with regard to the condition classes 2–5.

This process needs to be carried out for the whole tunnel respectively the whole tunnel network, in order to prioritise the tunnels in order of the seriousness of the damages and the construction costs. After that, the owner or operator can organise finance plans and intervention schedules. Finally, a maintenance intervention plan for the tunnel (or zone) is made according to the priorities defined for the whole network.

2.2 Advantages & Limits of Digitalization

The assessment is carried out by experienced engineers, but it is naturally a subjective evaluation. The influence of subjectivity must be reduced without relieving the engineer from the responsibility. Digitalization and artificial intelligence push the construction industry to work out an increasing number of standards, which make the evaluation of the different features easier, faster and more unified. This development is already an important factor to clarify the more obvious results and to accelerate the communication between the participants within the entire maintenance process.

Beside the fact that digitalization pushes the development of the standards, an important acceleration happens in each step of the inspection project. For example, scanning data can be transformed to working data of the maintenance project. Periodically executed scanning jobs will then be used for the digital tunnel twin.

On the other hand,the effects of different damages are quite complex. The same feature can have different importance according to:

• the geology,

• the building materials,

• the geometric position,

• the age of the damage, and the speed of change.

Even though the combination of most of these factors with the damage or deficiency types can be programmed into a software to support decisions, the human control factor cannot be excluded. For example, the possible complexity of the geology or a lack of information can cause individual cases.

Some decisions in the process of value preservation cannot be made artificially, even though many steps can be standardized, digitalized, supported, optimized and finally accelerated by artificial intelligence. The evaluation of data capturing, processing and analysis is a must for integrating tunnel maintenance workflows to BIM, which is going to be more and more common in underground construction.

An important saving aspect of the use of new technologies is that the time spent on-site is minimized and less staff-intensive. The data capture is going to be more detailed, which contributes a lot to the pre-inspections, making them easier and more exact. Inspections on-site will be also more efficient and exact. A bigger part of the documentation of the inspection can be directly generated from the inspection data itself. The whole data is going to be accessible in the cloud solution, so the owner of the tunnel can see the inspection results online and in real time.

3 Methodology of Data Capture and
Data Processing

3.1 Data Capture

Data capture in a tunnel is very demanding due to low light, harsh conditions, usually very limited time for measurement, safety regulations and high expectations with regard to accuracy. Data capture techniques have de­veloped from visual inspection on-site to more automated technologies. The most common data acquisition method is laser scanning. Light conditions do not influence the quality of these results, but on the other hand, the colour information is lost. The opposite characteristics apply for photogrammetry. It needs light, resulting in good colour images – but the whole process takes a lot of time.

The transport of measuring equipment in the tunnel varies:
from manually pushed trolleys to locomotives, remote-controlled small robots or cars. Lately drones are being tested for use in tunnels. Fig. 3 shows the data capturing with an Amberg MISS (Mobile Infrastructure Scanning System) in the Ceneri Base Tunnel in Switzerland.

3.2 Data Processing and Analytics

There are many ways to capture data. What matters in the end is data quality. When it comes to data processing, low quality data is useless because the inspected features are usually not visible and additional tunnel visits are necessary. With good quality data it is important to be able to effectively use and reuse them. The aim is to have an intelligent system, which can compare each picture and feature with a database where all the additional information is collected and organized. This tool makes it easy to analyse the data and reuse them in the future.

3.3 Motivation of the Development Project

The currently used software TunnelMap provides the functions to create a database of feature drawings. But the whole software has its limitations, mainly due to its age. It was developed roughly two decades ago when data synchronization, ideas about Building Information Modeling (BIM) and possibilities of smart construction were on a completely different level or not existing, yet.

Today TunnelMap can no longer keep up with modern inspection demands. It is time for an evolved web-based platform that will keep pace with today‘s technologies and market expectations.

4 How intelligent Inspection works

4.1 Workflow

These days paper drawings are no longer state of the art. The future of tunnel inspection is digital. This is why data is moved to the cloud to make it available anywhere, anytime, from all types of devices and for all authorized users. In this chapter the new platform called Amberg Inspection Cloud will be described. Mainly the new methodology and advantages for tunnel inspection will be discussed.

4.2 Workflow Optimization

The basic workflow remains identical as described in chapter 3. The main goal is to use automation to make the whole inspection process easier, less time-consuming and more cost-effective. The focus is on the data processing and data analysis. There are many ways to collect data efficiently in an automatic or semi-automatic way. However,
the inspection and damage drawing has always been lengthy manual work.

The workflow optimization can be done in two steps. The first step is helping the user with manual drawing by highlighting areas with high crack (damage) probability. This way the user does not have to inspect the whole tunnel in detail and can focus only on problematic areas. The second step is an automatic feature: detection and drawing. It means that inspected damages or utilities are found and drawn fully automatically. Users can then do a quick visual check. This second level of the workflow optimization is expected to save three-quarters of the inspection data processing time. In both above-mentioned steps, artificial intelligence (deep neural network) is used for damage highlighting and automatic drawing.

4.3 The new Platform –
Why Artificial Intelligence?

The evolution of artificial intelligence and machine learning already began in the middle of the last century. Recent findings in this sector and powerful hardware enabled the technique to transfer from an academic field to a commercial application. The hardware development was especially important for so called deep neural networks. These networks are based on the same principle as the human brain. Their power lies in millions of parameters that need to be adjusted during the network training. The learning process may take from few minutes up to several days.

Various studies were already conducted on the topic of artificial intelligence. The biggest efforts were probably done in the medical sector. But some investigations have also been carried out in road, rail and concrete damage detection already [Yokoyama et al. 2017, Cha et al. 2017, Faghih-Roohi et al. 2016, Yang et al. 2017, Eisenbach et al. 2017]. In these studies, many similarities with tunnel damages are mentioned, but up to this day, no records of any work on this topic in tunnel environment can be found.

Using the previous research dealing with damage detection and available tools (Keras, TensorFlow), LeNet and VGG-16 network were tested. These networks promised good results in reasonable computation time.

The main limitation in the first development stages was a limited amount of training data. Therefore, very deep networks were excluded from testing. Even VGG-16 ended up overfitting the training data. After this observation, the LeNet was fine-tuned for searching cracks or wet areas on tunnel walls and ceilings.

As an input for the networktraining, Amberg used grey scale images created from laser scanning data. The visualizations can be presented in more detail on the new web-based platform Amberg Inspection Cloud.

4.4 From the Tunnel to the Cloud

The Inspection Cloud is a web-based platform specialized on tunnel inspections. It is open for data from different sources and measurement methods that can provide good quality, high-resolution images.

When the tunnel design and the high-resolution images are uploaded to a cloud server, the pre-trained neural network can automatically detect damage probability. An experienced operator can then process and interpret the automatically generated data (Figs. 4–6). In case of a bigger project, several operators can work independently without tedious data transfer and synchronization.

When the analysis is finished, it can be accessed directly online in the Inspection Cloud. That includes comparison of the tunnel states from different years, visualization of damages along the tunnel and the standard DXF export.

In the future, automatic feature drawing will be included. Once Amberg Inspection Cloud is fully developed, the inspection engineer only checks the automatic drawings and makes minor changes, if needed. That will save a lot of time compared to traditional manual drawing.

4.5 Outlook

The final goal is to eliminate standard manual work from tunnel inspection as far as possible. At present, it does not seem advisable to skip manual input and the human factor completely. And in the future experts will still need to make the final decisions on some unclear cases. But the time-consuming manual marking of all the clear cases will be replaced by artificial intelligence.

Not only the inspection evaluation itself an issue. Data transfer and business engineering are time-consuming as well. The goal in this area is to be fully BIM compatible. Currently this development is restrained by the fact that a BIM standard for tunnels is not yet defined. The Inspection Cloud combines position information (3D), feature info (1D), time (1D) and an effective data analysis. The missing part of the chain is a BIM format that allows easy transfer to other programs.

5 Conclusion

Deep learning is not a new technology, but the possibility to use powerful processing engines changed the usage of these technologies dramatically. The transformation from pure academic approach into the construction business area started in recent years. The construction industry is nowadays also transforming many operations and applications with a neural networks approach. Learnings from all these applications and experiences in various fields of use help to stimulate further developments.

The challenge for the future is to build safe tunnels which are predictable in terms of damage evolvement and which optimize the costs for the entire life-cycle of a tunnel infrastructure. Thanks to the digitalization of the industry, more data from infrastructures can be collected. However, it will be crucial to further increase the quality of data analytics. Deep learning helps to automate this process. The use of deep learning for feature detection has shown that these technologies will change the way of how tunnel inspection will be done in the future (Fig. 7).

For the time being there is still work to be done and more reliable training data to be collected. First test projects have shown that time saving of up to 60 % is possible in comparison to a traditional inspection. With the current state of the software it is possible to save up to 30 %. Once the data processing for feature detection has improved, it will also influence the data capturing cycle. The data capture process is currently going through a big change as well. New sensors and platforms are developed, which allow an automated data capture of the entire tunnel infrastructure. Combining this data and building a digital copy of an infrastructure will be possible. Then it should be feasible to start to change periodical inspections to event driven inspections so the owner of an infrastructure can plan his inspections based on the actual conditions and changes in certain sectors of the tunnel.