Finding dory in the crowd: Detecting social interactions using multi-modal mobile sensing

Paper type: 
Workshop paper
K Katevas, K Hänsel, R Clegg, I Leontiadis, H Haddadi, L Tokarchuk
SenSys-ML 2019 - Proceedings of the 1st Workshop on Machine Learning on Edge in Sensor Systems
Year: 
2013
Abstract: 
Remembering our day-To-day social interactions is challenging even if you aren't a blue memory challenged fish. The ability to automatically detect and remember these types of interactions is not only beneficial for individuals interested in their behavior in crowded situations, but also of interest to those who analyze crowd behavior. Currently, detecting social interactions is often performed using ethnographic studies, computer vision techniques and manual annotation-based data analysis. However, mobile phones offer easier means for data collection that is easy to analyze and can preserve the user's privacy. In this work, we present a system for detecting stationary social interactions inside crowds, leveraging multi-modal mobile sensing data such as Bluetooth Smart (BLE), accelerometer and gyroscope. To inform the development of such system we conducted a study with 24 participants where we asked them to socialize with each other for 45 minutes. We built a machine learning system based on gradient-boosted trees that predicts both 1:1 and group interactions with a 30.2\% performance increase compared to a proximity-based approach. By utilizing a community detection-based method, we further detected the various group formation that exist within the crowd. Using mobile phone sensors already carried by the majority of people in a crowd makes our approach particularly well suited to real-life analysis of crowd behavior and influence strategies.
bibtex: 
@article{katevas2019findingsensing, author = {Katevas, K and Hänsel, K and Clegg, R and Leontiadis, I and Haddadi, H and Tokarchuk, L}, journal = {Proceedings of the 1st Workshop on Machine Learning on Edge in Sensor Systems}, month = {Nov}, pages = {37--42}, title = {Finding dory in the crowd: Detecting social interactions using multi-modal mobile sensing}, year = {2019} }
doi: 
10.1145/3362743.3362959
Subject area: