| Title | Heterogeneity Aware Workload Management in Distributed Sustainable Datacenters |
| Publication Type | Journal Article |
| Year of Publication | 2019 |
| Authors | Cheng, D., Zhou, X., Ding, Z., Wang, Y., Ji, M. |
| Journal | IEEE Transactions on Parallel and Distributed Systems |
| Volume | 30 |
| Pagination | 375–387 |
| ISSN | 1558-2183 |
| Keywords | batch job migration algorithm, brown energy, cloud computing, clouds, composability, computer centres, constrained optimization problem, datacenter power consumption, distributed datacenters, distributed self-sustainable datacenters, distributed sustainable datacenters, dynamic power availability, flexible batch job management approach, flexible batch job manager, green computing, green energy, green power availability, green power supply, Green products, heterogeneity, heterogeneity aware workload management, heterogeneity-oblivious approach, heterogeneous workloads, holistic heterogeneity-aware cloud workload management approach, Human Behavior, Internet service operators, Internet-scale Computing Security, Internet-scale services, job migration, large-scale data analytics, Metrics, nonlinear programming, Optimization, policy governance, power aware computing, Power supplies, pubcrawl, QoS requirements, quality of service, real-world weather conditions, Resiliency, scheduling, sCloud, Sustainable datacenter, sustainable development, system goodput, System performance, Task Analysis, transactional workload placement, workload placement, workload traces |
| Abstract | The tremendous growth of cloud computing and large-scale data analytics highlight the importance of reducing datacenter power consumption and environmental impact of brown energy. While many Internet service operators have at least partially powered their datacenters by green energy, it is challenging to effectively utilize green energy due to the intermittency of renewable sources, such as solar or wind. We find that the geographical diversity of internet-scale services can be carefully scheduled to improve the efficiency of applying green energy in datacenters. In this paper, we propose a holistic heterogeneity-aware cloud workload management approach, sCloud, that aims to maximize the system goodput in distributed self-sustainable datacenters. sCloud adaptively places the transactional workload to distributed datacenters, allocates the available resource to heterogeneous workloads in each datacenter, and migrates batch jobs across datacenters, while taking into account the green power availability and QoS requirements. We formulate the transactional workload placement as a constrained optimization problem that can be solved by nonlinear programming. Then, we propose a batch job migration algorithm to further improve the system goodput when the green power supply varies widely at different locations. Finally, we extend sCloud by integrating a flexible batch job manager to dynamically control the job execution progress without violating the deadlines. We have implemented sCloud in a university cloud testbed with real-world weather conditions and workload traces. Experimental results demonstrate sCloud can achieve near-to-optimal system performance while being resilient to dynamic power availability. sCloud with the flexible batch job management approach outperforms a heterogeneity-oblivious approach by 37 percent in improving system goodput and 33 percent in reducing QoS violations. |
| DOI | 10.1109/TPDS.2018.2865927 |
| Citation Key | cheng_heterogeneity_2019 |
Heterogeneity Aware Workload Management in Distributed Sustainable Datacenters