| Title | Learning, Development, and Emergence of Compositionality in Natural Language Processing |
| Publication Type | Conference Paper |
| Year of Publication | 2021 |
| Authors | Maruyama, Yoshihiro |
| Conference Name | 2021 IEEE International Conference on Development and Learning (ICDL) |
| Keywords | Animals, Bit error rate, Cognition, compositionality, Conferences, contextuality, information processing, Linguistics, natural language processing, philosophy of language, pubcrawl, statistical distributional model of language, symbolic compositional model of language |
| Abstract | There are two paradigms in language processing, as characterised by symbolic compositional and statistical distributional modelling, which may be regarded as based upon the principles of compositionality (or symbolic recursion) and of contextuality (or the distributional hypothesis), respectively. Starting with philosophy of language as in Frege and Wittgenstein, we elucidate the nature of language and language processing from interdisciplinary perspectives across different fields of science. At the same time, we shed new light on conceptual issues in language processing on the basis of recent advances in Transformer-based models such as BERT and GPT-3. We link linguistic cognition with mathematical cognition through these discussions, explicating symbol grounding/emergence problems shared by both of them. We also discuss whether animal cognition can develop recursive compositional information processing. |
| DOI | 10.1109/ICDL49984.2021.9515636 |
| Citation Key | maruyama_learning_2021 |
Learning, Development, and Emergence of Compositionality in Natural Language Processing