Many complex engineering systems involve interactions among a large number of agents with coupled dynamics and decisions due to their shared environment and resources. Such systems are often operated using a hierarchical architecture, where a coordinator determines some macroscopic control signal to steer the population to achieve a desired group objective while respecting local preferences and constraints for individual agents. Examples include electricity demand response programs, ground and air transportation systems, data center power management, robotic networks, among others.