Visible to the public Deepcode: Feedback Codes via Deep Learning

TitleDeepcode: Feedback Codes via Deep Learning
Publication TypeJournal Article
Year of Publication2020
AuthorsKim, Hyeji, Jiang, Yihan, Kannan, Sreeram, Oh, Sewoong, Viswanath, Pramod
JournalIEEE Journal on Selected Areas in Information Theory
Volume1
Pagination194—206
Date PublishedApril 2020
ISSN2641-8770
KeywordsAWGN channels, channel coding, composability, Computing Theory, Decoding, Deep Learning, feedback communication, machine learning, Neural networks, Noise measurement, pubcrawl, Recurrent neural networks, reliability theory, Schalkwijk–Kailath scheme
AbstractThe design of codes for communicating reliably over a statistically well defined channel is an important endeavor involving deep mathematical research and wide-ranging practical applications. In this work, we present the first family of codes obtained via deep learning, which significantly outperforms state-of-the-art codes designed over several decades of research. The communication channel under consideration is the Gaussian noise channel with feedback, whose study was initiated by Shannon; feedback is known theoretically to improve reliability of communication, but no practical codes that do so have ever been successfully constructed. We break this logjam by integrating information theoretic insights harmoniously with recurrent-neural-network based encoders and decoders to create novel codes that outperform known codes by 3 orders of magnitude in reliability and achieve a 3dB gain in terms of SNR. We also demonstrate several desirable properties of the codes: (a) generalization to larger block lengths, (b) composability with known codes, and (c) adaptation to practical constraints. This result also has broader ramifications for coding theory: even when the channel has a clear mathematical model, deep learning methodologies, when combined with channel-specific information-theoretic insights, can potentially beat state-of-the-art codes constructed over decades of mathematical research.
URLhttps://ieeexplore.ieee.org/document/9062338
DOI10.1109/JSAIT.2020.2986752
Citation Keykim_deepcode_2020