Straggler Resilient Serverless Computing Based on Polar Codes
Title | Straggler Resilient Serverless Computing Based on Polar Codes |
Publication Type | Conference Paper |
Year of Publication | 2019 |
Authors | Bartan, Burak, Pilanci, Mert |
Conference Name | 2019 57th Annual Allerton Conference on Communication, Control, and Computing (Allerton) |
Date Published | Sept. 2019 |
Publisher | IEEE |
ISBN Number | 978-1-7281-3151-1 |
Keywords | channel coding, cloud computing, coded computation scheme, coded computation technique, coding methods, coding theory, Complexity theory, computational burden, Computational modeling, computationally effective manner, computationally expensive tasks, Computing Theory, Decoding, distributed computation, emerging cloud based computation model, encoding, gradient methods, hybrid computing framework, linear algebra, matrix multiplication, optimisation, polar codes, pubcrawl, Random access memory, resilience, Resiliency, sequential decoding, serverless computing mechanism, serverless platforms, Servers, straggler resilient serverless computing, straggler-resilience, Task Analysis |
Abstract | We propose a serverless computing mechanism for distributed computation based on polar codes. Serverless computing is an emerging cloud based computation model that lets users run their functions on the cloud without provisioning or managing servers. Our proposed approach is a hybrid computing framework that carries out computationally expensive tasks such as linear algebraic operations involving large-scale data using serverless computing and does the rest of the processing locally. We address the limitations and reliability issues of serverless platforms such as straggling workers using coding theory, drawing ideas from recent literature on coded computation. The proposed mechanism uses polar codes to ensure straggler-resilience in a computationally effective manner. We provide extensive evidence showing polar codes outperform other coding methods. We have designed a sequential decoder specifically for polar codes in erasure channels with full-precision input and outputs. In addition, we have extended the proposed method to the matrix multiplication case where both matrices being multiplied are coded. The proposed coded computation scheme is implemented for AWS Lambda. Experiment results are presented where the performance of the proposed coded computation technique is tested in optimization via gradient descent. Finally, we introduce the idea of partial polarization which reduces the computational burden of encoding and decoding at the expense of straggler-resilience. |
URL | https://ieeexplore.ieee.org/document/8919767/ |
DOI | 10.1109/ALLERTON.2019.8919767 |
Citation Key | bartan_straggler_2019 |
- Resiliency
- hybrid computing framework
- linear algebra
- matrix multiplication
- optimisation
- polar codes
- pubcrawl
- Random access memory
- resilience
- gradient methods
- sequential decoding
- serverless computing mechanism
- serverless platforms
- Servers
- straggler resilient serverless computing
- straggler-resilience
- Task Analysis
- channel coding
- encoding
- emerging cloud based computation model
- distributed computation
- Decoding
- Computing Theory
- computationally expensive tasks
- computationally effective manner
- Computational modeling
- computational burden
- Complexity theory
- coding theory
- coding methods
- coded computation technique
- coded computation scheme
- Cloud Computing