SystemsThatLearn@CSAIL member Element AI will be providing technical insights into the research interests and projects being conducted across the company. The Element AI Research team will be presenting a few short technical talks followed by a poster reception highlighting their current research areas and projects. Research will be presented on topics including, explainability, learning theory, domain adaptation, adversarial learning, few shot learning and much more. This event will engage both MIT faculty and students on common research interests and collaborative opportunities.

October 23rd 4-7PM: 4th floor R&D Commons, MIT CSAIL

Food and refreshments will be provided!

About Element AI:

Element AI is an AI solutions and products provider that advances cutting-edge AI research and turns it into scalable products that make businesses safer, stronger, and more agile. Co-founded in 2016 by Jean François Gagné and leading AI researcher Yoshua Bengio in the deep learning hub of Montreal, the company is focused on pioneering an AI-First world by turning the world’s most important AI research into transformative business applications. The company has a deep commitment to advanced research, with a renowned faculty fellow network, the largest privately-owned Canadian artificial intelligence R&D lab, as well as a growing network of leading research partners.

TECH TALK- 4-5PM, R&D Commons MIT CSAIL

Chief Science Officer and Co-Founder, Dr. Nicolas Chapados will provide an overview on the research directions being investigated within the company and how research is conducted across the fundamental, applied and product teams at Element AI. Research scientist, Karolina Dziugaite from Element AI’s fundamental lab will speak about her recent work focused on constructing generalization bounds to help understand existing learning algorithms and propose new directions. Applied research scientist, Perouz Taslakian from Element AI’s AI Core team will present work and demos on the team’s recent progress on representation learning and vision.

4:00- 4:20 PM Research at Element AI - A snapshot of research activities across the fundamental, applied and product teams

Nicolas Chapados - Chief Science Officer and Co-Founder, Element AI

Bio: Nicolas Chapados holds an engineering degree from McGill University and a PhD in Computer Science from University of Montreal, Canada. While still writing his thesis and jointly with his advisor Yoshua Bengio, he co-founded ApSTAT Technologies in 2001, a machine learning technology transfer firm, to apply cutting-edge academic research ideas to areas such as insurance risk evaluation, supply chain planning, business forecasting, national defence, and hedge fund management. From this work, he also co-founded spin-off companies: Imagia (www.imagia.com), to detect and quantify cancer early with AI analysis of medical images, Element AI (www.elementai.com), to help organizations plan and implement their AI transformation, and Chapados Couture Capital (www.chapados-couture.com), a quantitative asset manager. He holds the Chartered Financial Analyst (CFA) designation.

Abstract: Nicolas will describe the research being conducted at Element AI by the various research themes and provide insight into the company’s research thematics and priorities.

4:20- 4:40 PM Revisiting Generalization for Deep Learning

Karolina Dziugaite - Research Scientist, Element AI

Bio: Karolina is an all-but-defense Ph.D. student in machine learning from the University of Cambridge under the supervision of Zoubin Ghahramani. Before that, she studied Mathematics at Warwick University and read Part III in Mathematics at the University of Cambridge, receiving a Masters in Advanced Studies in Applied Mathematics. During her Ph.D., Karolina was a long-term participant in the Foundations of Machine Learning program at the Simons Institute for the Theory of Computing at the University of Berkeley. There she started working in learning theory, studying generalization in deep learning, which was the focus of her Ph.D. dissertation. At EAI, Karolina continues to tackle important problems in deep learning, relying on theory as a guide. Her research spans deep learning related generalization theory, optimization, and algorithms

Abstract: Karolina will present recent work constructing generalization bounds in order to understand existing learning algorithms and propose new ones. Generalization bounds relate empirical performance to expected performance. Their tightness varies widely, and depends on the complexity of the learning task and the amount of data available, but also on how much information the bounds take into consideration. Her work is particularly concerned with data-dependent bounds that are quantitatively nonvacuous. After formalizing the notion of “flat minima” using PAC-Bayes generalization bounds, she obtains nonvacuous generalization bounds for stochastic neural network classifiers built by randomly perturbing solutions obtained by stochastic gradient descent (SGD) on MNIST.

4:40- 5:00 PM Applied Representation Learning Through AI Core

Perouz Taslakian - Applied Research Scientist, Element AI

Bio: Perouz received her Ph.D. in discrete and computational geometry from McGill University under the supervision of Luc Devroye and Godfried Toussaint. She spent three years as a postdoc in Brussels, where she mainly worked on problems in discrete and combinatorial geometry. She joined Element AI as an applied research scientist and is currently doing applied research in representation learning within AI Core.

Abstract: In this short presentation, Perouz will give an overview of how applied research is conducted at Element AI. Her talk will give an insight into how the teams within AI Core draw impact from state-of-the-art research. In particular, she will describe the work she does in the representation learning team and how this work is integrated with other components of AI Core.

POSTER SESSION- 5-7PM, R&D Commons MIT CSAIL

Poster Title: Unsupervised Domain Adaptation with Similarity Learning

Pedro O. Pinheiro - Research Scientist, Element AI

Bio: Pedro has a PhD in Computer Vision and Machine Learning at École Polytechnique Fédérale de Lausanne (EPFL), in Switzerland. Prior to EAI, his research focused mostly on large-scale image segmentation problems using deep learning techniques. This included problems in fully and weakly supervised semantic segmentation, object proposals and instance segmentation. His research focuses on computer vision, machine learning and their intersection. Currently, he is interested in learning how to leverage knowledge from one task to another. This include applications to problems related to domain adaptation, few-shot learning, imitation learning, 3d vision.

Abstract: The objective of unsupervised domain adaptation is to leverage features from a labeled source domain and learn a classifier for an unlabeled target domain, with a similar but different data distribution. Most deep learning approaches to domain adaptation consist of two steps: (i) learn features that preserve a low risk on labeled samples (source domain) and (ii) make the features from both domains to be as indistinguishable as possible, so that a classifier trained on the source can also be applied on the target domain. In this paper, we propose a different way to do the classification, using similarity learning. The proposed method learns a pairwise similarity function in which classification can be performed by computing similarity between prototype representations of each category. The domain-invariant features and the categorical prototype representations are learned jointly and in an end-to-end fashion. At inference time, images from the target domain are compared to the prototypes and the label associated with the one that best matches the image is outputted. The approach is simple, scalable and effective. We show that our model achieves state-of-the-art performance in different unsupervised domain adaptation scenarios.

Poster Title: Functionality-preserving Adversarial Learning

Ousmane Dia - Applied Research Scientist, Element AI

Bio: Ousmane Amadou Dia defended his Ph.D. in Computer Science at the University of South Carolina Columbia. His research focused on Security. Ousmane did his undergraduate studies in Applied Mathematics and completed his Master of Science at the “Universite Catholique de Louvain” in Belgium where he was involved in the European SIMILAR network of Excellence Project focusing on multimodal interfaces for Mammograms Computer Assisted Diagnosis. Ousmane Amadou DIA was awarded the Fulbright Scholarship for his Ph.D. Ousmane also held different positions in the field of Machine Learning culminating with a research scientist position at Amazon and a now Research Engineer at Element AI.

Abstract: Multiple approaches of adversarially attacking deep neural networks exist. Most of these techniques deal with images and either involve computing the gradients of a loss with respect to the images, or inject noise generally sampled from a normal distribution to the true cases. While for images the adversarial examples are generated to be identical to the true cases, the precise locations of some details in a true image may still not be preserved. However, exact locations of fine details are not important for perceptual recognition. In Malware detection or Text understanding, this is not however the case. In fact, being able to generate new examples that preserve the malignant properties of the true malware is useful as these new cases could be used to augment intrusion detection signature databases. For text, generating sound adversarial examples means the modified text needs to preserve the meaning of the original text. We posit the reason existing techniques fail to preserve such properties is because the noise they inject in the true cases is not necessarily sampled within the manifold of the true cases or that the gradients they exploit is not selected in the right neighborhood. In this study, we explore a novel approach to generate functionality preserving adversarial examples starting with images, then text and malware.

Poster Title: Deep Prior

Alexandre Lacoste - Research Scientist, Element AI

Bio: Alexandre has his PhD in theoretical machine learning from Laval University. Previously, he worked for approximately 3 years at Google in the Research Group as a software engineer where he focused on building end-to-end question answering systems using Recurrent Neural Networks (RNNs) with attention. At EAI, Alexandre is focused on Bayesian Deep Learning and Multi-Task Transfer Learning.

Abstract: Using variational Bayes neural networks, we develop an algorithm capable of accumulating knowledge into a prior from multiple different tasks. The result is a rich and meaningful prior capable of few-shot learning on new tasks. The posterior can go beyond the mean field approximation and yields good uncertainty on the performed experiments. Analysis on toy tasks shows that it can learn from significantly different tasks while finding similarities among them. Experiments of Mini-Imagenet yields the new state of the art with 74.5% accuracy on 5 shot learning. Finally, we provide experiments showing that other existing methods can fail to perform well in different benchmarks.

Poster Title: Interpretable genotype-to-phenotype classifiers with performance guarantees

Alexandre Drouin - Research Scientist, Element AI

Bio: Alexandre recently completed his doctoral studies in machine learning at Université Laval under the supervision of François Laviolette. Prior to joining EAI, he worked on machine learning algorithms for computational biology. Specifically, he studied learning from extremely high dimensional data with very few examples and applied this work to the genomic prediction of antibiotic resistance. He also studied kernel methods for peptide-protein binding affinity prediction, a problem with implications in drug design and personalized medicine. Currently, his research investigates how information can be shared between related tasks to achieve good generalization with small sample sizes (e.g., domain adaptation, meta-learning, transfer learning). He is also interested in model interpretability and explainability, which are bound to favor applications of machine learning to fields, such as health care and insurance.

Abstract: Learning to predict the biological state (phenotype) of an individual from its genome is a problem of high importance for personalized medicine. With the increasing availability of genomic data, it is now possible to conduct studies that compare large cohorts of individuals in order to learn models that are predictive of phenotypes. However, the large dimensionality of genomic data poses great challenges for learning algorithms in terms of scalability and generalization. In this presentation, we will discuss two scalable learning algorithms that admit strong generalization guarantees based on sample compression theory. We will rely on an application to the prediction of antibiotic resistance in bacteria to demonstrate the ability of these algorithms to generalize well in this setting. Furthermore, we will demonstrate the interpretability of the resulting models, which provide insight into the biological underpinnings of such phenotypes.

Poster Title: TADAM: Task dependent adaptive metric for improved few-shot learning

Boris Oreshkin - Research Scientist, Element AI

Bio: Boris obtained a Ph.D. in Electrical Engineering from McGill University. He completed a postdoc in Medical Imaging at McGill University and a postdoc in Operations Research at University of Montreal. His academic research focused on the applications of machine learning and advanced statistical modeling to distributed signal processing, medical imaging and call center modeling and simulation. Boris has extensive industrial experience working as a machine learning algorithm developer. His current research interests include few-shot learning, weakly supervised representation learning, multi-modality generative modeling and inference as well as advanced ML models for time series and forecasting.

Abstract: Few-shot learning has become essential for producing models that generalize from few examples. In this work, we identify that metric scaling and metric task conditioning are important to improve the performance of few-shot algorithms. Our analysis reveals that simple metric scaling completely changes the nature of few-shot algorithm parameter updates. Metric scaling provides improvements up to 14% in accuracy for certain metrics on the mini-Imagenet 5-way 5-shot classification task. We further propose a simple and effective way of conditioning a learner on the task sample set, resulting in learning a task-dependent metric space. Moreover, we propose and empirically test a practical end-to-end optimization procedure based on auxiliary task co-training to learn a task-dependent metric space. The resulting few-shot learning model based on the task-dependent scaled metric achieves state of the art on mini-Imagenet. We confirm these results on another few-shot dataset that we introduce in this paper based on CIFAR100.

Poster Title: Pix2Scene: Learning Implicit 3D Representations from Images

David Vazquez - Research Scientist, Element AI

Bio: Dr. David Vázquez is a Research Scientist at Element AI. Previously he was a postdoctoral researcher at Computer Vision Center of Barcelona (CVC) and Montreal Institute of Learning Algorithms (MILA) and Assistant Professor in the Department of Computer Science at the Autonomous University of Barcelona (UAB). He received his Ph.D. in Computer Vision (2013), M.Sc. in CV and AI (2009) and B.Sc. in Computer Science (2008) from the UAB. Previously he received the B.Sc. in Software Engineering from the UDC (2006). He has done internships at Daimler AG, UAM and URJC. He is an expert in machine perception for autonomous vehicles. His research interests include deep learning, computer vision, robotics, 3D vision and autonomous driving.

Abstract: Modelling 3D scenes from 2D images is a long-standing problem in computer vision with implications in, e.g., simulation and robotics. We propose pix2scene, a deep generative-based approach that implicitly models the geometric properties of a scene from images. Our method learns the depth and orientation of scene points visible in images. Our model can then predict the structure of a scene from various, previously unseen view points. It relies on a bi-directional adversarial learning mechanism to generate scene representations from a latent code, inferring the 3D representation of the underlying scene geometry. We showcase a novel differentiable renderer to train the 3D model in an end-to-end fashion, using only images. We demonstrate the generative ability of our model qualitatively on both a custom dataset and on ShapeNet. Finally, we evaluate the effectiveness of the learned 3D scene representation in supporting a 3D spatial reasoning.

Poster Title: Revisiting Generalization for Deep Learning

Karolina Dziugaite - Research Scientist, Element AI

Poster Title: W2GAN: Recovering an Optimal Transport Map with a GAN

Amjad Almahairi - Research Scientist, Element AI

Bio: Amjad recently completed his Ph.D. at MILA, Université de Montreal under the supervision of Prof. Aaron Courville. His research interests lie in the area of machine learning, and specifically deep learning, with a focus on unsupervised learning and generative models. Previously, he was a research engineer at Nuance, where we he worked on NLU for personal assistant applications. At EAI, Amjad’s research focuses on representation learning and generative models for images and natural language.

Abstract: Understanding and improving Generative Adversarial Networks (GAN) using notions from Optimal Transport (OT) theory has been a successful area of study, originally established by the introduction of the Wasserstein GAN (WGAN). An increasing number of GANs incorporate OT for improving their discriminators, but that is so far the sole way for the two domains to cross-fertilize. We consolidate the bridge between GANs and OT with one model: W2GAN, where the discriminator approximates the second Wasserstein distance. This model exhibits a twofold connection: the discriminator implicitly computes an optimal map and the generator follows the optimal map during training. Perhaps surprisingly, we also provide empirical evidence that other GANs are also approximately following the optimal map.

Poster Title: Human-AI Interaction at Element AI

Masha Krol - Human-AI Interaction Team Lead, Element AI

Bio: As the HAII Team Lead at Element AI, Masha is applying over 9 years of product design expertise to the creation of collaborative human-AI interactions. Having previously led UX teams for innovation projects at IBM (what later became IBM Watson Analytics) and Kinaxis gives Masha a wealth of experience in design approaches to solving complex enterprise-level problems. Masha holds a BCS‘09 from Carleton University.

Abstract: Masha will share the mission and vision of the Human-AI Interaction Team at Element AI, describe the early-stage projects that the team is currently exploring, and the path of the team’s successful experiments to productization.

Element AI Tech Talk and Research Showcase