Building Distributed Temporal Graphs From Event Streams

Paper type: 
Workshop paper
Benjamin A Steer, Alessandro Di Stefano, Richard G Clegg and Felix Cuadrado
Second Workshop on Advances in Mining Large-Scale Time Dependent Graphs (TD-LSG)
Temporal graphs capture the development of relationships within data throughout time. This model would fit naturally within a streaming architecture, where new events can be inserted directly into the graph upon arrival from a data source, being compared to related entities or historical state. However, the vast majority of graph processing systems only consider traditional graph analysis on static data, with some outliers supporting either temporal analysis on similarly static data or traditional analysis on graphs updated via event streams. In this work we define a temporal graph model which can be updated via event streams and discuss the challenges of distribution and graph management. To solve these challenges, we introduce Raphtory, a distributed temporal graph management system which maintains the full graph history in-memory, leveraging this to insert streamed events directly into the graph model without batching and with minimal synchronisation.
This paper describes the Raphtory system which is used to analysis large-scale time-varying graph systems. It can ingest streaming graph information and store the complete graph history. It enables queries to be made over the graphs at different points in that graph's history.