Learning to Simulate on Sparse Trajectory Data
Moving from RL-based traffic signal control, recently we are heading towards simulation which is a more basic and fundamental problem, in the sense that RL methods rely heavily on the simulator.
Actually building a simulator is not only useful for RL methods, but also for the whole decision-making process. With a simulator, one can just heuristically propose some decision strategies, run it on the simulator and get the quantitative results. Recently we’ve launched a challenge on containing the pandemic at this year’s KDD by running different mobility intervention strategies on our simulator.
However, building a good simulator is a challenging task. First, simulation is more of a compound process – it has to model both observed and unobserved factors – this brings the main challenge in building digital twins. How to model the process from the observations we have? One of our preliminary paper is trying to answer this question with GAIL.
Different from the traditional game setting, in many real-world applications like health care and transportation, we do not have observations at all times. For example, we might only have certain treatment data of a patient only when he/she visits the hospital. Another example is, we might only observe the behavior of a vehicle once it passes a roadside camera. Facing these sparse data, how can we model the internal mechanism of the agents (patients and vehicles)?
If you raise the question of how to deal with sparse data to researchers in trajectory data mining, a straight forward method would be: Interpolation! Interpolation is a commonly adopted way to deal with sparse data in trajectory data mining. Linear interpolation, for example, is often used to interpolate the missing points between two observed trajectory points. But in real-world cases, considering the interactions between vehicles, the vehicle is unlikely to drive at a uniform speed during that unobserved time period, hence the interpolated trajectories may be different from the true trajectories.
As a matter of fact, such interpolation methods often assume the static relationship between the interpolated points and given points, which doe not always hold in real-world cases. The vehicles driving at the beginning and end of a road segment is different from how they drive in the middle of a road, considering the effects of traffic signals. Moreover, vehicles driving in the middle of the road, are also influenced by how many vehicles around them at that time – And this kind of information is usually missing in the data.
How do we know such missing information? The answer is through simulation. The simulation of all vehicles would compensate for the missing information by providing detailed driving behaviors. Though such driving behavior might be far from the true behavior at the beginning, if we can somehow know the difference of current driving behavior and observed driving behavior, we can use some optimization methods to optimize the behaviors by iterating through the expectation of current driving behavior and maximization the likelihood of driving behavior being true behavior, similar to EM algorithm which guarantees to converge.
Therefore, in this paper, we propose to model the driving behavior with interpolation in one model, i.e., Imitation with interpolation (ImInGAIL). For the detailed design, please refer to our paper:
@inproceedings{imingail,
author = {Wei, Hua and Chen, chacha and Liu, Chang and Zheng, Guanjie and Li, Zhenhui},
title = {Learning to Simulate on Sparse Trajectory Data},
booktitle = {Joint European Conference on Machine Learning and Knowledge Discovery in Databases},
series = {ECML-PKDD '20},
year = {2020},
organization={Springer}
}