Big data-driven optimization in transportation and communication networks.

Summary The evolution of metropolitan structures has created various types of urban networks. Among which two types of networks are of great importance for our daily life: transportation networks corresponding to human mobility in physical space and communication networks supporting human interactions in digital space. The rapid expansion in scope and scale of these two networks raises fundamental research questions about how to optimize these networks. Some of the primary objectives include on-demand resource provisioning, anomaly detection, energy efficiency and quality of service. Despite differences in design and implementation technologies, transportation networks and communication networks share common fundamental structures, and exhibit similar dynamic spatio-temporal characteristics. As a result, there are common challenges in optimizing these two networks: traffic profiling, mobility prediction, traffic aggregation, node clustering and resource allocation. To achieve the optimization goals and research challenges, various analytical models, optimization algorithms, and simulation systems have been proposed and widely studied across several disciplines. These analytical models are often validated by simulation and could lead to suboptimal results in deployment. With the emergence of the Internet, a massive volume of urban network data can be collected. Recent advances in Big Data analysis techniques have provided researchers with great potential to understand this data. Motivated by this trend, the objective of this thesis is to explore a new paradigm of data-driven network optimization. We address the above scientific challenges by applying data-driven methods for network optimization. We propose two data-driven algorithms for network traffic clustering and user mobility prediction, and apply these algorithms to optimization in transportation and communication networks. First, by analyzing the large-scale traffic datasets of the two networks, we propose a graph-based clustering algorithm to better understand the traffic similarities and traffic variations between different areas and times. Based on this, we apply the traffic clustering algorithm to the following two network optimization applications: 1. A dynamic traffic clustering for on-demand planning of bike-sharing networks. In this application, we dynamically cluster bike stations with similar traffic patterns to obtain more stable and predictable grouped (clustered) traffic demands, so that overcrowded stations in the network can be predicted and dynamic demand-based network planning can be performed. Evaluation results using real data from New York City and Washington, D.C. show that our solution accurately predicts overloaded clusters [.]