Second HSE-Yandex autumn school on generative models

24 Nov 2020, 09:45 → 27 Nov 2020, 21:00 Europe/Moscow

R602 (NRU HSE, Moscow)

Alexey Naumov (HSE), Dmitriy Vetrov (HSE), Andrey Ustyuzhanin (Yandex School of Data Analysis (RU)), Natalia Talaikova

HSE University and Yandex invite you to the joint second autumn school on generative models aimed at young undergraduate/graduate students planning to develop research career. This intense four-day workshop will consist of 3 interdisciplinary mini-courses, list of invited talks and round table discussion with the leading researchers. Key topics of this school are:

• Introduction to Deep Learning
• Generative Models for fun and profit
• Introduction to bandit problems

Due to pandemic, the school is going to run in the mixed format: in addition to a zoom meeting room, there is a physical room booked that would allow participants to follow the event from HSE Pokrovsky campus.

The school is organised by three HSE laboratories:

• LAMBDA (Laboratory of Methods for Big Data Analysis)
• DeepBayes (Centre of Deep Learning and Bayesian Methods)
• HDI Lab (International laboratory of stochastic algorithms and high-dimensional inference)

Yandex is an industry partner of the school.

Mini-courses:

• Introduction to Machine Learning by Dmitry Vetrov, Ekaterina Lobacheva (HSE University) and Nadia Chirkova (HSE University)
• Generative models by Denis Derkach (HSE University)
• Introduction to few-shot learning by Quentin Paris (HSE University)

Invited talks:

• Alexey Artemov (Skoltech)
• Evgeny Burnaev (Skoltech)
• Anton Konushin (MSU, HSE, YSDA)
• Stanislav Morozov (Yandex)
• Maxim Panov (Skoltech)
• Evgenii Stepanov (PDMI RAS, HSE)
• Alexandra Suvorikova (WIAS, HSE)
• Nikita Zhivotovsky (Google, HSE)

Live discussion with leading researchers:

On the 27th afternoon, we plan to have a live panel discussion with leading scientists on the future of AI and research career.

School materials:

All materials of the school is available via this page timetable. It contains slides of the lectures and talks. All seminar materials are available via github repository: https://github.com/HSE-LAMBDA/GenModels-2020

This school is supported by the RSF grant N19-71-30020 "Applications of probabilistic artificial neural generative models to development of digital twin technology for Non-linear stochastic systems".

2020 HSE University and Yandex.pdf

Tuesday 24 November

Organization: Welcome

1-opening.pdf

1-opening.pptx

DeepBayes lab: Introduction to machine learning

Convener: Ekaterina Lobacheva (HSE University)

DL_slides - Chirkova N.pdf

intro_to_ml.pdf

11:00 Coffee Break

DeepBayes lab: Introduction to machine learning. Practice

Convener: Ms Nadia Chirkova (HSE)

DL_slides - Chirkova N.pdf

intro_to_ml.pdf

12:30 Lunch

DeepBayes lab: Introduction to neural networks

Convener: Ms Nadia Chirkova (HSE)

DL_slides - Chirkova N.pdf

intro_to_ml.pdf

15:00 Tea Break

DeepBayes lab: Introduction to neural networks. Practice

Convener: Ms Nadia Chirkova (HSE)

DL_slides - Chirkova N.pdf

intro_to_ml.pdf

1 Unusual Effects in Modern Deep Neural Networks

Speaker: Prof. Dmitriy Vetrov (HSE)

Wednesday 25 November

LAMBDA lab: Introduction, the metrics for the generative models

Convener: Denis Derkach (National Research University Higher School of Economics (RU))

11:00 Coffee Break

2 Generative adversarial networks

Speaker: Mr Artem Maevskiy (HSE University)

GANs - Maevsky.pdf

Organization: Group Photo

1-opening.pdf

1-opening.pptx

12:40 Lunch

LAMBDA lab: Generative Adversarial Networks. Practice

Convener: Artem Ryzhikov (National Research University Higher School of Economics (RU))

3 Generative adversarial networks

Speaker: Mr Artem Maevskiy (HSE University)

15:00 Tea Break

LAMBDA lab: Introduction to the normalizing flows. Lecture

Convener: Artem Ryzhikov (National Research University Higher School of Economics (RU))

16:30 Tea Break

LAMBDA lab: Normalizing flows, practice

Convener: Artem Ryzhikov (National Research University Higher School of Economics (RU))

Thursday 26 November

HDI lab: An introduction to bandit problems

Convener: Quentin Paris (Higher School of Economics)

11:00 Coffee Break

HDI lab: An introduction to bandit problems

Convener: Quentin Paris (Higher School of Economics)

12:30 Lunch

HDI lab: An introduction to bandit problems

Convener: Mr Sergey Samsonov (HSE)

15:00 Tea Break

HDI lab: An introduction to bandit problems

Convener: Mr Sergey Samsonov (HSE)

4 Robust mean estimation

Speaker: Nikita Zhivotovskiy (Google, Higher School of Economics)

Friday 27 November

5 Synthetic data and generative models in road sign recognition and other computer vision problems

Speaker: Anton Konushin (Higher School of Economics)

KonushinAS - traffic signs.pdf

6 Latent-Space Laplacian Pyramids for Adversarial Representation Learning with 3D Point Clouds

Constructing high-quality generative models for 3D shapes is a fundamental task in computer vision with diverse applications in geometry processing, engineering, and design. Despite the recent progress in deep generative modelling, synthesis of finely detailed 3D surfaces, such as high-resolution point clouds, from scratch has not been achieved with existing approaches. In this work, we propose to employ the latent-space Laplacian pyramid representation within a hierarchical generative model for 3D point clouds. We combine the recently proposed latent-space GAN and Laplacian GAN architectures to form a multi-scale model capable of generating 3D point clouds at increasing levels of detail. Our evaluation demonstrates that our model outperforms the existing generative models for 3D point clouds.

Speaker: Alexey Artemov (Yandex)

Artemov_Kravchenko_LSLP_GAN_Point_Clouds.pdf

Artemov_Kravchenko_LSLP_GAN_Point_Clouds.pptx

11:20 Coffee Break

7 Deep Generative Models for Knowledge Transfer

Speaker: Prof. Evgeny Burnaev (Skoltech)

Burnaev_Manifold_Knowledge_Transfer_v2.pdf

8 Optimal transport and its application in bioinformatics

Speaker: Alexandra Suvorikova (Higher School of Economics)

OT_ML.pdf

13:00 Lunch

9 How to reveal hidden geometric structures in data

Speaker: Prof. Evgeny Stepanov (Higher School of Economics)

EvoSpaceTalkHuawei1020.pdf

10 Variational Inference: Mean Field, Normalizing Flows and beyond

In this talk we are going to discuss Variational approach to Bayesian Inference. We will start by reviewing standard mean-field approximations to the posterior as well as more powerful methods such as normalizing flows. Then we will discuss a new computationally efficient method to combine Variational Inference (VI) with Markov Chain Monte Carlo (MCMC). This approach can be used with generic MCMC kernels, but is especially well suited to MetFlow, a novel family of MCMC algorithms we introduce, in which proposals are obtained using Normalizing Flows. The marginal distribution produced by such MCMC algorithms is a mixture of flow-based distributions, thus drastically increasing the expressivity of the variational family. Unlike previous methods following this direction, our approach is amenable to the reparametrization trick and does not rely on computationally expensive reverse kernels

Speaker: Prof. Maxim Panov (Skoltech, HSE)

Panov-presentation.pdf

11 Big GANs Are Watching You: Towards Unsupervised Object Segmentation with Off-the-Shelf Generative Models

Since collecting pixel-level groundtruth data is expensive, unsupervised visual understanding problems are currently an active research topic. In particular, several recent methods based on generative models have achieved promising results for object segmentation and saliency detection. However, since generative models are known to be unstable and sensitive to hyperparameters, the training of these methods can be challenging and time-consuming.
In this work, we introduce an alternative, much simpler way to exploit generative models for unsupervised object segmentation. First, we explore the latent space of the BigBiGAN -- the state-of-the-art unsupervised GAN, which parameters are publicly available. We demonstrate that object saliency masks for GAN-produced images can be obtained automatically with BigBiGAN. These masks then are used to train a discriminative segmentation model. Being very simple and easy-to-reproduce, our approach provides competitive performance on common benchmarks in the unsupervised scenario.

Speaker: Mr Stanislav Morozov (Yandex)

Big_GANs_are_watching.pdf

16:10 Tea Break

Panel discussion

12 How to reveal hidden geometric structures in data

Speaker: Prof. Evgeny Stepanov (Higher School of Economics)

Organization: The end

1-opening.pdf

1-opening.pptx

Evening session: Evening session -- Cancelled