The 3D BRIDGE app is a mobile software tool designed to facilitate bridge inspection processes by enabling inspectors to collect and record element level bridge inspection data using 3G/4G network-enabled tablet devices. The overall goal is to help Michigan Department of Transportation (MDOT) reduce time needed for field staff to collect bridge inspection data, and move forward to a more efficient bridge inspection process that is available to all appropriate levels of MDOT. The application automatically renders a 3D model of the desired bridge using data from the bridge management databases. The 3D model enables inspectors to more easily record and visualize the defects and element level data associated with the bridge at specific locations along the infrastructure elements. The app has reached an initial level of development ready for more field testing, and features are being enhanced and added to fit the needs of MDOT staff. In the project's second phase, the 3D BRIDGE application is being readied for deployment, with both MDOT and MTRI committed to helping create the future of bridge inspection technology that could be made available nationwide.
Summary
Current MDOT bridge inspection practices use paper forms followed by a manual data entry step to populate the bridge management system (BMS) database with information needed for bridge management and repair. Faced with an aging bridge inventory and increasing federal regulations for collecting element level data, MDOT wishes to increase the efficiency and reliability of collected data. To achieve this MDOT requested a 2D/3D application that can utilize mobile tablet technology to aid inspectors in the field.
To develop this application, Michigan Tech Research Institute (MTRI) staff first examined the state of practice across the nation to better understand currently available options. As of the 2014 survey, no application assisted with collection of element level data. Additionally, MTRI met with experienced inspectors Amy Trahey from Great Lakes Engineering, and Janiene DeVinney and Lindsey Renner of MDOT to better understand the needs of bridge inspectors, tailoring the application design to their input.
Because end-to-end 3D bridge models are not yet available for all bridges, MTRI developed a server application using Django (a python web framework), to generate bridge Extensible Markup Language (XML) files using data available in MDOT's BMS database. Each XML file contains information about the element level components of a bridge, including location and size, representing a generic bridge model that is sufficiently representative for inspection purposes. The server application includes a user tuning component to correct initial assumptions due to lack of information, such as placement of bearings per beam.
To produce the client application, MTRI selected the Unreal Engine 4 (UE4) game engine by Epic Games to provide cross platform rendering capability that includes Android, iOS, and Windows. The application itself is built using C++ interfaced with the UE4 engine, as well as UE4 Blueprints for high level functionality and Java for integration with native camera functionality on Android devices. The application receives a bridge XML from the server application and constructs an interactive 3D model. Using a set of intuitive navigational views the user can traverse the bridge and mark the surface with Element Level defect information, photos, and comments. Defect markers are proportionally sized based on the defect quantity and color coded to match condition states. The application also has a summary view for reviewing the aggregate defect information.
Current Data Collection Process
As stated before, the current bridge inspection practices at MDOT utilize paper forms followed by a manual data entry step to populate their database. Additionally, photographs documenting bridge deterioration are collected and stored separately from inspection data. MDOT inspectors must also carry reference manuals and past inspection reports to help verify the accuracy of the data they are collecting. The exact locations of bridge defects are not stored which creates an inconvenience as the data are difficult to visualize, especially through time. In addition, Federal regulations now require inspectors to collect AASHTO Element level data. Current processes do not allow for the efficient collection of this data. All of these issues create an inefficient bridge inspection process that increases the difficulty of the bridge inspector's job. The 3D BRIDGE app is designed to solve these problems. This tool will allow the inspector to collect all of the data they need in one paperless application while allowing the inspector to better visualize the reports, defects, and collection data.
Server and Database Implementation
With 3D models not yet readily available for all bridges, a model had to be easily created for use in the inspection tool. Given the large amounts of descriptive information within MDOT's bridge management database, MTRI decided to build a model from the database utilizing all of the relevant data. The data was retrieved, missing information was derived from the data collected, and then a “model” was created with all of the necessary data as an XML. The resulting XML was then sent to the frontend of the application to render the 3D model.
3D BRIDGE User Interface
When the application is opened, the user will be greeted with the 3D BRIDGE tool's home screen.
The bridge inspector can then load their desired bridge through the “Load Bridge” Menu based on a unique bridgekey.
Once the bridge has been loaded, an interactive 3D model will appear along a proportionally accurate representation of the selected bridge. Users may navigate along the bridge my dragging their finger from left to right or up and down. Users may also zoom in or out on specific areas by pinching in or out on the screen.
At any point along the bridge, inspectors may record defect data. Each individual defect can be annotated with a description, photos, and quantity. Fields will be pre-populated from database values from the most recent inspection. Context-sensitive descriptions are attached to each element type, just as in the Bridge Element Inspection Manual.
The inspector can then change the size, shape, and location of the defect by entering defect editor mode. Inspectors can change the location of the defect by either clicking and dragging, or simply clicking to a new location. Inspectors can change the size of defect by either pinching the defect or by using the slider bar below. Users may also change the rotation by using the slider bar below, or by placing two fingers on the defect and rotating accordingly. Finally, the inspector can change the aspect ratio of the defect to switch between circular and rectangular defects.
The compass and eye icons let the inspector switch between different views of the same defect.
The application also lets inspectors record real live photos and enter in descriptive comments about the defect.
Finally, bridge inspectors can view different summaries of the defect and element data. The Summary Review tab displays the data based on the condition state of the defect and what bridge element the defect is on.
Bridge inspectors may also view the summarized defect and AASHTO element level data through the Element Report tab.
Admin Site and User Tuning
Due to assumptions in MTRI's calculations for key variables used to render a readily usable 3D model within the app, a secure website was developed through Django for the inspector to verify and/or modify these assumptions to create a more accurate model. For the generic concrete overpass style bridge, the calculations are reasonably accurate. However, there are several outliers where key pieces of information about how the bridge is composed, such as bridges with differing numbers of beams per span or bridges where the bearing placement per pier is different than normal. These bridges would be modeled incorrectly and therefore the inspector could not record defect data accurately. The admin site allows the bridge inspector to fix any errors in the model to create a better replica of the bridge and allows them to make any necessary changes to the data as they see fit. The information is divided into eight categories: Assumptions, General Bridge Information, Deck, Superstructure, Substructure, Bearings, Bearing Placement, and Culvert. The most important information that the bridge inspector will need to review are the Assumptions and Bearing Placement sections. These are the two areas where data is not present in MDOT's database but is derived from calculations and assumptions. In the future, more fields and categories may be incorporated to make a more accurate model.
Future Plans
This application is being prepared for deployment in Michigan through Phase II of the project, which has the goal of creating an application that fits all of the MDOT bridge inspector's needs. Many features are in the processes of being added based on actual bridge inspector's feedback. Outreach for use in other states is ongoing. The 3D BRIDGE app is a key component towards the future goal of utilizing 3D models to monitor and review a bridge throughout its lifetime.
