Artificial intelligence-aided railroad trespassing detection and data analytics: Methodology and a case study
Introduction
Based on statistics from the Federal Railroad Administration (FRA) of the United States (U.S.) Department of Transportation, the U.S. railroad system is comprised of approximately 830 railroads, 134,000 miles of track, and 210,000 railroad crossings (FRA, 2018a). Trespassing accidents along rights-of-way (ROW) and at highway-rail grade crossings constituted over 90% of rail-related deaths over the past ten years (FRA, 2018a). More specifically, there were 855 trespass-related fatalities in 2017, which demonstrated an increase of 18 percent from 2012 (FRA, 2018b). In addition to fatalities, these incidents resulted in other serious consequences, such as nonfatal injuries, train derailments, hazardous material spillage, train delays, and traffic congestion. From 2012 to 2016, trespassing accidents in the United States cost railroads and society approximately $43 billion (FRA, 2018b), a sum that did not cover indirect costs (e.g., emotional distress or productivity losses). The FRA (2016a) concluded that most trespassing deaths occurring each year are preventable if effective countermeasures were implemented.
Amongst the limited studies of railroad trespassing, most researchers encountered challenges due to limited data resources and uncertain data quality. Most publicly available trespassing data takes the form of casualty information or grade crossing accidents, and does not include near-miss events. However, the FRA (2018b) postulated that the number of trespassing occurrences each year far exceeds the number of fatalities and injuries and more data on trespassing events that do not result in casualties would be valuable to railroad safety researchers. In other words, while the accident reports submitted to the FRA by railroads have proven to be helpful to railroad researchers, most of the valuable data on trespassing is still missing. Trespassing events indicate certain behaviors that may lead to severe consequences if they occur repeatedly. Specifically, the near-miss events of trespassing, involving common causation against trespassing accidents, can contribute to developing the perceptions of trespassing risks with a sufficient number of events. Learning from these trespassing events is critical towards better education for the public on trespassing safety, law enforcement, and engineering solutions to prevent trespassing on railroad tracks. The increasing availability of video data in the rail industry makes the collection of trespassing data more feasible.
Deployment of camera systems continues to increase in the United States following the 2015 Fixing America’s Surface Transportation (FAST) Act, which mandated the installation of cameras throughout passenger rail lines to promote safety objectives (FRA, 2015). In addition, the Transportation Security Authority (TSA) provided funding for surveillance in transit and passenger rail areas (Elias et al., 2016). Cameras can be found throughout rail lines, yards, bridges, grade crossings and stations, which provide numerous video data sources for railroads. However, most camera systems are reviewed manually by railroad crew, train police, or local police which is labor-intensive and expensive. Limited resources and operator fatigue (Dee and Velastin, 2008) can potentially lead to missing trespassing events. Besides, the trespassing incident/accident risks along the right-of-ways and at grade crossings are challenging to monitor and to manage since they involved non-railroad personnel (e.g., pedestrian, vehicle drivers). To address these challenges and leverage the untapped potential of this big video data, this research develops a novel Artificial Intelligence (AI)-aided tool that is capable of localizing and identifying trespassing events in both archival video data and live streams with acceptable processing speed and accuracy. You Only Look Once (YOLO), an emerging object detection algorithm developed by Redmon et al. (2016), Redmon and Farhadi (2018), is utilized in the trespassing detection methodology to achieve high-accuracy trespassing detection with relatively low computation cost. With this practice-ready technology, over two months of video data from one grade crossing are processed and over 3,000 trespassing events are detected and analyzed in this study. These detected events, along with recorded trespassing video clips, can contribute to developing practical trespassing risk mitigation strategies and improving the safety of the train crew, rail passengers, and road users.
Section snippets
Trespassing on railroad property
Railroad trespassing is defined as an event when any unauthorized person or vehicle enters or remains on a railroad right-of-way, grade crossing, equipment, or facility (FRA, 2018b). Railroads own their rights-of-way and have a reasonable expectation of operating on their property without the presence or interference of unauthorized persons. Pedestrians and motorists are only permitted on railroad property where a roadway intersects with the railroad tracks at the same level or grade, provided
Overview of You Only Look Once (YOLO)
YOLO uses features learned by a single deep convolutional neural network to detect objects. As introduced in the previous section, most deep learning-based object detection algorithms, such as the R-CNN family, have a complex detection pipeline, in which bounding box generation, object classification, duplicate detection elimination, and bounding box refining and rescoring are executed sequentially. Instead, YOLO sees the entire image or video frame and implicitly encodes contextual information
Overview of selected grade crossing
To validate the functionality of the proposed AI-based trespassing detection technique, a grade crossing located in New Jersey is selected as a case study, although the developed methodology can also be applied to rights-of-way. The selected crossing experiences about 110 activations per day, with the majority being commuter trains. One train station with three parking lots, two to the west of the train tracks and one to the east, are adjacent to the grade crossing. Several restaurants,
Conclusions
This paper presents a state-of-the-art AI-aided methodology with high-accuracy fast-processing railroad trespassing detection capabilities for both highway-rail grade crossings and rights-of-way. The applications of YOLO and computer vision in trespassing detection have been validated in around 1,632 h of videos with reasonable accuracy. Around 3,000 trespassing violations are detected and recorded during the analyzed period. In the location-specific case study, the collected trespassing
Future work
Firstly, future work would focus on accuracy improvement by mitigating noise from sunlight on the surface of red signal and extreme weather conditions. For example, the automatic identification of grade crossings gate position can be used as a supplemental activation trigger. Moreover, analyses of passive, non-signalized grade crossings can also be explored in the future. Secondly, future work can investigate the possibility of integrating a proposed AI-based trespassing detection tool with
CRediT authorship contribution statement
Zhipeng Zhang: Methodology, Validation, Formal analysis, Investigation, Data curation, Writing – original draft. Asim Zaman: Validation, Investigation, Data curation. Jinxuan Xu: Conceptualization, Methodology, Supervision, Writing – original draft, Project administration. Xiang Liu: Investigation, Methodology, Project administration, Writing – review & editing.
Declaration of Competing Interest
The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.
Acknowledgements
The authors thank Mr. Francesco Bedini Jacobini from the FRA, Mr. Marco daSilva from the Volpe National Transportation Systems Center, and Mr. Todd Hirt from the NJDOT for their feedback and insight during this research. The first author was financially supported by the Rutgers, The State University of New Jersey in the preparation of this manuscript. The views and opinions expressed herein at these of the authors.
References (46)
- et al.
Highway-rail grade crossing safety challenges for shared operations of high-speed passenger and heavy freight rail in the US
Saf. Sci.
(2014) - et al.
A systematic review of the literature on safety measures to prevent railway suicides and trespassing accidents
Accid. Anal. Prev.
(2015) - et al.
An evaluation of four types of railway pedestrian crossing safety intervention
Accid. Anal. Prev.
(2003) - et al.
Near miss experiences of transport and recreational cyclists in New South Wales, Australia. Findings from a prospective cohort study
Accid. Anal. Prev.
(2017) - et al.
The effectiveness of road-safety crossing guards: Knowledge and behavioral intentions
Saf. Sci.
(2008) Analysis of fatal train-pedestrian collisions in metropolitan Chicago 2004–2012
Accid. Anal. Prev.
(2016)- et al.
Main characteristics of train-pedestrian fatalities on Finnish railroads
Accid. Anal. Prev.
(2012) - et al.
Near-miss narratives from the fire service: a Bayesian analysis
Accid. Anal. Prev.
(2014) - et al.
Accident versus near miss causation: a critical review of the literature, an empirical test in the UK railway domain, and their implications for other sectors
J. Hazard. Mater.
(2004) Valuing walkability: New evidence from computer vision methods
Transp. Res. Part A: Policy Pract.
(2019)
A real-time Chinese traffic sign detection algorithm based on modified YOLOv2
Algorithms
A comparative study of rail-pedestrian trespassing crash injury severity between highway-rail grade crossings and non-crossings
Accid. Anal. Prev.
Automated detection of grade-crossing-trespassing near misses based on computer vision analysis of surveillance video data
Saf. Sci.
A deep-learning-based computer vision solution for construction vehicle detection
Comput.-Aided Civ. Infrastruct. Eng.
Traffic congestion detection from camera images using deep convolution neural networks
Transp. Res. Rec.
Effect of Gate Skirts on Pedestrian Behavior at Highway-Rail Grade Crossings
How Close Are We to Solving the Problem of Automated Visual Surveillance?
Mach. Vision Application
Transportation security: issues for the 114th congress
Library of Congress, Congressional Research Service
Cited by (13)
AAP – Technology in rail safety – Applications of advanced technologies in rail safety
2024, Accident Analysis and PreventionModeling interactions between the Montreal subway and other urban transportation modes at the station level
2023, Tunnelling and Underground Space TechnologyReal-time dense traffic detection using lightweight backbone and improved path aggregation feature pyramid network
2023, Journal of Industrial Information IntegrationCitation Excerpt :For instance, the objects of a person and train are quite difficult to be detected using a traditional detector designed for street vehicles in the same image. Unfortunately, there are more frequent traffic delays and accident reported recently that are related with the railroad grade crossing area where mixed traffic is present [11, 12]. Additionally, it needs to be mentioned that the accurate detection of train and pedestrians are the basis of efficient traffic management and safety assurance when there is railroad involved in the urban areas.
Quantitative risk assessment of railway intrusions with text mining and fuzzy Rule-Based Bow-Tie model
2022, Advanced Engineering InformaticsCitation Excerpt :Specifically, the images of unauthorized objects can be obtained through stationary cameras, train-mounted cameras, or unmanned aerial vehicles [10]. Then, traditional computer vision approaches [11] and deep learning models [12,3] assist the video surveillance. LiDAR data is another useful type of information that provides precise depth information and complements camera data well [13].
Design of Indoor Navigation Scheme Based on Bluetooth Low Energy
2024, Lecture Notes in Electrical Engineering