| Author: | Yussif, Abdul-mugis |
| Title: | A novel framework for automatic sidewalk defect monitoring with object detection and instance segmentation |
| Advisors: | Zayed, Tarek (BRE) |
| Degree: | M.Sc. |
| Year: | 2023 |
| Subject: | Sidewalks -- Maintenance and repair Structural health monitoring Hong Kong Polytechnic University -- Dissertations |
| Department: | Department of Building and Real Estate |
| Pages: | xi, 103 pages : color illustrations |
| Language: | English |
| Abstract: | Carbon footprints and low public health resulting from the tremendous gaseous vehicle emissions and low human physical activities have become environmental, societal, and economic issues with high priorities during policy formulations. Promoting walking is one of the approaches proposed by physiologists and environmentalists to minimize the effects of these problems. Cities such as Hong Kong, with high population densities per square kilometers, are in the right position to effectuate walkable neighborhoods. Notably, Hong Kong is one of the cities with the best transportation systems in the world, supplemented by well-layout sidewalks and footpath systems. However, sidewalk usability is marred by defects, reducing the prospects of reaching optimal productivity. The defects also threaten all pedestrians, especially those with challenging mobility, wheelchair users, and baby strollers. It is highly recommended to practice good maintenance culture and ensure early intervention to repair defects and curtail further growth to safeguard infrastructures from becoming ruins. It also presents economic advantages by preventing them from escalating to complete breakdowns, requiring huge sums of resources to reconstruct everything from scratch. Sidewalk inspection is time-consuming and less frequent due to the manual practices employed. Sidewalk defect studies have not received adequate attention from transportation engineers and city management, as they have been treated as a minor subsidiary of the roadway, making it difficult to attract funds and researchers. Furthermore, sidewalk defects are typically smaller in size and possess properties seemingly inherent to the infrastructure, making it complex to apply object detection and instance segmentation techniques on many of them. Most defects are inspected visually, although user surveys, people-centric sensing, and LiDAR scanners methods also exist. It is important to realize that the visual method is faster, less costly, and more convenient. This research proposed an object detection and instance segmentation approach toward sidewalk defect monitoring aimed at providing the framework to automate the visual inspection method. Data were obtained from various sources, including Google Street View, to build models to detect sidewalk defects in images of different sizes, resolutions, and conditions. Image processing techniques such as rotation, brightness enhancements, scaling, and sharpness were applied randomly to ensure model scalability and generalization. Based on their design, two main types of sidewalks were considered: concrete and brick sidewalks. Then, separate detection models were built to recognize and segment four types of defects in each of them. A feature extraction approach of transfer learning was adopted, and the sidewalk defect detection models were built on top of the feature extraction backbone using the mask R-CNN architecture. The concrete model emerged as the best-performing model, with a maximum error rate of 0.03 and an average F1-score of 0.988. It achieved an average mAP of 0.99 for all the instances and more than 90% validation accuracy in all iterations. On the other hand, the brick model achieved a maximum accuracy of 0.875, an average F1-score of 0.920, an average overlap of 0.90, and an average mAP of 0.97. This research provides a framework for implementing automated remote sidewalk defect monitoring using visual methods. A Google Street View integrated model is developed for users to monitor the sidewalks using street view images automatically. The predicted results are saved in images and JSON file formats for easy accessibility to users. It will reduce the required human efforts, increase inspection frequency and enhance record-keeping. Nonetheless, future research adding 3D scanning components will increase the reliability of the models, such as depressions and crack estimation. |
| Rights: | All rights reserved |
| Access: | restricted access |
Files in This Item:
| File | Description | Size | Format | |
|---|---|---|---|---|
| 7079.pdf | For All Users (off-campus access for PolyU Staff & Students only) | 5.63 MB | Adobe PDF | View/Open |
Copyright Undertaking
As a bona fide Library user, I declare that:
- I will abide by the rules and legal ordinances governing copyright regarding the use of the Database.
- I will use the Database for the purpose of my research or private study only and not for circulation or further reproduction or any other purpose.
- I agree to indemnify and hold the University harmless from and against any loss, damage, cost, liability or expenses arising from copyright infringement or unauthorized usage.
By downloading any item(s) listed above, you acknowledge that you have read and understood the copyright undertaking as stated above, and agree to be bound by all of its terms.
Please use this identifier to cite or link to this item:
https://theses.lib.polyu.edu.hk/handle/200/12615