Analyzing Crowd Behavior from 400 Feet Above the Ground

MIDDLEBURY, Vt. – To improve public safety at rallies, demonstrations, and other large events, a Middlebury College assistant professor of computer science is conducting research this summer in an attempt to detect abnormal crowd behavior from a drone.

Jason M. Grant, who works in the field of computer vision, created an algorithm that can discern abnormal crowd motion for his doctoral dissertation in computer science at the University of Notre Dame. Now he is trying to become one of the first computer scientists to employ the technology in real time with a drone.

“Advances in crowd-analysis research have improved greatly in recent years to ensure the safety of individuals at large public events,” said Grant. “My research is aimed at preventing incidents from occurring through real-time detection of abnormal crowd behavior.” Grant references such tragic events as stampedes at the Jamaraai Bridge in Mecca or the 2010 crowd disaster at the Love Parade concert in Germany. His work in the detection of abnormal crowd behavior is aimed at giving event organizers and safety personnel a tool to reduce the severity of incidents that can occur whenever large crowds of people gather in a space.

This summer Grant is working in tandem with Henry Mound ’20.5, a computer science major from Redding, Conn., to perfect crowd-behavior analysis from up to 400 feet in the air – the highest that a drone can be flown legally. For their research, Grant and Mound are flying a professional DJI Matrice 600 Pro hexacopter controlled by three Global Navigation Satellite System (GNSS) receivers with a video camera and computer mounted onboard.

Grant’s methodology is based upon detecting deviations in the dominant paths of motion within a scene. “We implement a method for dominant-motion extraction based upon optical flow clustering, and we develop a matching algorithm to compare dominant motion paths at various points in a video feed,” he explained.

Optical flow is a tool used by computer scientists to perceive the apparent pattern of motion in a video, which is a key component in abnormal crowd behavior detection

Grant also employs a second method using a “particle advection model” for local motion that predicts the location of a particle based on its previous motion. “By measuring the distance between the predicted trajectory and the observed trajectory, we are able to determine the presence of abnormal behavior,” he said.

Both methods complement each other and can be deployed in real time. Grant and Mound are hoping to test both drone-borne technologies this summer or early fall by conducting a controlled experiment using upwards of 100 volunteers in a large open space on campus, such as the football field.

Mound is also assisting Grant in preparing the syllabus for a new course tentatively called Robotics and Drones that will be offered in the spring of 2019 by the Department of Computer Science. The undergraduate research assistant is “learning the basics of computer vision and writing programs to control the smaller drones that will be used by the students in the course. Many of the experiments he is conducting this summer will form the basis for projects and homework assignments in that new class,” Grant explained.

Mound experimented with the Tello quadcopter made by DJI in June and will try out Robolink’s CoDrone quadcopter in July. After the summer, Grant and Mound will look at the strengths of both units before determining which model will be coded and deployed by students in the Robotics and Drones course next spring.

“It’s been a very exciting summer so far,” Mound said while working with a Tello on the sixth floor of McCardell Bicentennial Hall. “One of the reasons I came to Middlebury was to have an experience like this working for 10 weeks on research with a computer scientist like Professor Grant.”

Grant, who earned his bachelor’s degree at the University of Maryland Baltimore County and both an MS and PhD in computer science and engineering at Notre Dame, was originally interested in facial recognition but, after about three years, he became fascinated with “the motion and behavior of crowds, rather than that of individual people.” His dissertation completed earlier this year was titled, “Analysis of crowd behavior based on optical flow clustering: detection, classification, and clustering,” and he hopes to see it published soon.

The recipient of numerous awards and honors while at Notre Dame, Grant has received funding for his research and conference travel from Middlebury College, the Institute of Electrical and Electronics Engineers, the National Society of Blacks in Computing, and the Center for Minorities and People with Disabilities in Information Technology.

– With reporting and on-the-ground photography by Robert Keren