Objectifs globaux, verrous scientifiques/techniques

Because of the abundance of data, machine learning techniques have now to deal with increasingly large and complex inputs (video, sounds, speech, text, clicks, recommendation, etc). As an example, the ImageNet dataset involves millions of examples and thousands of classes and it is now one of the major challenges for object recognition in images in the computer vision community. In parallel, computing hardware is now moving fast towards a new dimension with GPUs and these recent progresses in high performance computing change the nature of machine learning research. Full exploitation of the available data and computing power implies finding new efficient ways to parallelize learning algorithms over many threads. Because of its recent breakthroughs, this includes novel efficient approaches to designing and training deep networks that leverage inexpensive GPU computing power.

Deep in France collaborative research program aims at expanding the frontier of deep learning and associated massively parallel computing in machine learning. Our vision is to develop theory and new deep learning algorithms to facilitate widespread use of deep learning to yet under-explored domain applications such as embedded perception, video/audio scene understanding, text-based retrieval, sports analytics and medicine/healthcare.

The bottlenecks regarding deep learning addressed by Deep in France are both theoretical and methodological, as well as technical related to the optimization procedure and implementation. Deep learning suffers from the lack of a clear and adapted theoretical framework. Also, no clear architectural and design principles exists to determine the nature of the units, the number and the width of the layers and the connectivity of the network. The optimization process also raises many questions. Is stochastic gradient the best we can do? How to pre-train and interpret the units? What is the importance of unsupervised learning as quoted in the recent Nature paper on deep learning. Deep learning is very computationally intensive because of the number and the size of available data, because of the parallel nature of stochastic gradient, and deep architecture. This fact raises a lot of issues regarding the nature of CPU/GPU needed, associated architecture and software tool to facilitate the training phase.

Our project also aims at bringing together complementary machine learning, computer vision and machine listening research groups working on deep learning with GPU’s2 in order to provide the community with the knowledge, the visibility and the tools that brings France among the key players in deep learning. The long-term vision of Deep in France is to open new frontiers and foster research towards algorithms capable of discovering sense in an automatic manner, a stepping stone before the more ambitious far-end goal of machine reasoning.