In this work, we use control improvisation to synthesize voluntary lane-change policy that meets human preferences under given traffic environments. We first train Markov models to describe traffic patterns and the motion of vehicles responding to such patterns using traffic data. The trained parameters are calibrated using control improvisation to ensure the traffic scenario assumptions are satisfied. Based on the traffic pattern, vehicle response models, and Bayesian switching rules, the lane-change environment for an automated vehicle is modeled as a Markov decision process.