Using video analytics and machine learning to deter trespassers

Many locations – not just rail tracks, but construction sites, hospitals, and critical infrastructure – face similar problems of people entering places they shouldn’t, and doing things that are dangerous. In all cases, video image analysis combined with machine learning (ML) can offer clever ways to solve these problems at scale, enabling interventions that save lives, protect property, and keep critical infrastructure operating efficiently.

Back in 2020, a Network Rail-backed funding competition on innovative ways reduce trespassing gave us an early chance to develop technologies to track unwanted behaviour, which has provided a foundation for the more advanced systems we are currently developing.

For the project itself we developed The Trespasser Incursion Deterrent System (TIDS).  Each TIDS unit contained a camera, 4G connectivity, and a ruggedised computer. These were fixed to key vantage points at two stations to monitor specific ‘trespass zones’.

Once we were happy we had a reasonably accurate first version, we ran the system six months on live camera footage, watching for false positives and negatives and updating the algorithm accordingly. For example initial runs identified heads in windows of trains when they stopped at the platform as trespassers (see image), so the algorithm was tweaked to exclude these situations.

“Machine learning doesn’t really see the people. It interprets vast data sets – where each image is read as millions of bits of data – to determine what combination of data points represents a trespasser, and what is something else,” says Paul Simms, Operations Director at Zircon.

“Whilst humans can quickly spot a person on a track, processing image data is really hard for machines. The skill of working with machine learning is navigating the trade-offs. Set the parameters too broad and it will pick up lots of false positives, like animals on tracks or people standing close to the edge. Set them too tight and you can guarantee every result is a trespasser, but you may miss edge cases such as people dressed in black at night, or when the camera in direct sunlight.”

“Anyone who has followed the development of self-driving cars will know the challenges that camera systems face in trying to distinguish between situations that need an immediate response, and people going about their business.”

Engaging station managers helped us optimise detection zones, tune the model, and set alert thresholds to meet their real-world needs. In this case, station operators told us that they would rather tolerate a few false positives than miss a real trespasser, so we designed it accordingly.

“There is no ‘right answer’ in machine learning,” he adds. “The skill is in tuning it to get the best possible result for the desired use case, which means understanding how it will be used”.

Even when tuned to be over-cautious in identifying anything that looked like a trespasser, only 7% of the alerts were false positives. The trespass system has now been approved by Network Rail for purchase by any station on the network.

"This was just a single project looking at one problem,” says Simms. “But this trial offered a glimpse of just what is possible using video and ML to track people using CCTV. Detection is just the tip of the iceberg.”

In other projects, Zircon is developing ML video systems that identify not just people, but specific behaviours, such as aggressive drunks, people looking to harm themselves, or suspicious luggage.

This technology can seamlessly track individuals across multiple CCTV cameras by recognising their unique characteristics. The system can automatically tag an 'offender' or anomaly based on predefined behaviours and follow their movements through large stations and across hundreds of camera feeds. This provides operators – often tasked with monitoring a bank of many screens displaying hundreds of feeds – with targeted alerts tailored to specific behaviours that might otherwise go unnoticed by the human eye. Additionally, it enables operators to access a continuous stream of footage showing the individual’s movements. This functionality supports rapid and precise emergency responses, simplifies the investigation of crimes, and aids in analysing crowd control measures.

“Creating these complex solutions needs careful design and fine tuning of models,” concludes Simms. “That goes beyond just choosing the best technology. It means engaging users to understand the nature of the problem in the real world, so you can set the data and model parameters accordingly, and develop solution that deliver what the user needs, not just something that hits performance metrics.”

“With machine learning, it is not always about being the most accurate, but about solving the problem in a way that works for the user”.

Looking for a team to support your next Rail software venture?  Zircon is there to help you ensure project success, contact the team today.