On the Geometry of Rectifier Convolutional Neural Networks

Title	On the Geometry of Rectifier Convolutional Neural Networks
Publication Type	Conference Paper
Year of Publication	2019
Authors	Gamba, Matteo, Azizpour, Hossein, Carlsson, Stefan, Björkman, Mårten
Conference Name	2019 IEEE/CVF International Conference on Computer Vision Workshop (ICCVW)
Date Published	Oct. 2019
Publisher	IEEE
ISBN Number	978-1-7281-5023-9
Keywords	Complexity theory, compositionality, Computer vision, Computing Theory and Compositionality, convolution, Convolutional codes, convolutional layers, convolutional networks, convolutional neural nets, convolutional neural networks, Deep Learning, Geometry, gradient descent, Human Behavior, human factors, inductive bias, Kernel, learning (artificial intelligence), natural data, preactivation space, preimage, pubcrawl, rectifier convolutional neural networks, Tensile stress, trained rectifier networks, understanding
Abstract	While recent studies have shed light on the expressivity, complexity and compositionality of convolutional networks, the real inductive bias of the family of functions reachable by gradient descent on natural data is still unknown. By exploiting symmetries in the preactivation space of convolutional layers, we present preliminary empirical evidence of regularities in the preimage of trained rectifier networks, in terms of arrangements of polytopes, and relate it to the nonlinear transformations applied by the network to its input.
URL	https://ieeexplore.ieee.org/document/9022156
DOI	10.1109/ICCVW.2019.00106
Citation Key	gamba_geometry_2019