91�Թ���

Ido Tal  was born in Haifa, Israel, in 1975. He received the B.Sc., M.Sc., and Ph.D. degrees in computer science from Technion-Israel Institute of Technology, Haifa, Israel, in 1998, 2003 and 2009, respectively. During 2010–2012 he was a postdoctoral scholar at the University of California at San Diego. In 2012 he joined the Electrical Engineering Department at Technion. His research interests include constrained coding and error-control coding. He received the 91�Թ��� Joint Communications Society/Information Theory Society Paper Award (jointly with Alexander Vardy) for the year 2017.

Information-Theoretic Security: From Information-Theoretic Primitives to Algorithms

Matthieu Bloch (Georgia Institute of Technology)

Abstract

While the foundations of information-theoretic security can be traced back to the works of Shannon (1949), Wyner (1975), Maurer (1993), Ahslwede and Csiszár (1993), the past decade of research on the topic has enabled conceptual simplifications and generalizations, spanning both information and coding theory. This three-part tutorial will leverage these recent advances to provide a modern perspective on the topic. In the first part, we will develop four information-theoretic primitives that will prove central to the study of information theoretic security. In the second part, we will introduce canonical information-theoretic security models and combine information-theoretic primitives to establish their capacity. In the last part, we will show how to translate the insights from information-theory to actual codes and algorithms with manageable complexity. 

Biography

Matthieu R. Bloch is an Associate Professor in the School of Electrical and Computer Engineering, the Georgia Institute of Technology. He received the Engineering degree from Supélec, Gif-sur-Yvette, France, the M.S. degree in Electrical Engineering from the Georgia Institute of Technology, Atlanta, in 2003, the Ph.D. degree in Engineering Science from the Université de Franche-Comté, Besançon, France, in 2006, and the Ph.D. degree in Electrical Engineering from the Georgia Institute of Technology in 2008. In 2008-2009, he was a postdoctoral research associate at the University of Notre Dame, South Bend, IN. Since July 2009, Dr. Bloch has been on the faculty of the School of Electrical and Computer Engineering, and from 2009 to 2013 Dr. Bloch was based at Georgia Tech Lorraine. His research interests are in the areas of information theory, error-control coding, wireless communications, and cryptography. Dr. Bloch has served on the organizing committee of several international conferences; he was the chair of the Online Committee of the 91�Թ��� Information Theory Society from 2011 to 2014, and he has been on the Board of Governors of the 91�Թ��� Information Theory Society and an Associate Editor for the 91�Թ��� Transactions on Information since 2016. He is the co- recipient of the 91�Թ��� Communications Society and 91�Թ��� Information Theory Society 2011 Joint Paper Award and the co-author of the textbook Physical- Layer Security: From Information Theory to Security Engineering published by Cambridge University Press.

Joint Source and Channel Coding: Fundamental Bounds and Connections to Machine Learning

Deniz Gündüz (Imperial College London)

Presentation slides - -

Abstract

Machine learning and communications are intrinsically connected. The fundamental problem of communications, as stated by Shannon, “is that of reproducing at one point either exactly or approximately a message selected at another point,” can be considered as a classification problem. With this connection in mind, I will focus on the fundamental joint source-channel coding (JSCC) problem; starting from the basic information theoretic performance bounds. I will then move to practical implementation of JSCC for wireless image transmission. I will first show some “surprising” performance results with uncoded "analog” transmission, which will then be used to motivate unsupervised learning for JSCC. I will present a "deep joint source-channel encoder” architecture, which behaves similarly to analog transmission, and not only improves upon state-of-the-art digital transmission schemes, but also achieves graceful degradation with channel quality.

In the second part of the lecture, I will talk about another connection between information theory and machine learning, focusing on distributed machine learning, particularly targeting wireless edge networks, and show how ideas from coding theory can help improve the performance of distributed learning across unreliable servers. I will introduce both coded and uncoded distributed stochastic gradient descent algorithms, and study the trade-off between their average computation time and the communication load. Finally, I will introduce the novel concept of "over-the-air stochastic gradient descent" for wireless edge learning, and show that it significantly improves the efficiency of machine learning across bandwidth and power limited wireless devices compared to the standard digital approach that separates computation and communication. This will close the circle, illustrating how JSCC can be used to speed up machine learning at the wireless edge.

Biography

Deniz Gündüz received the B.S. degree from METU, Turkey in 2002, and the M.S. and Ph.D. degrees from NYU Polytechnic School of Engineering (formerly Polytechnic University) in 2004 and 2007, respectively. After his PhD, he served as a postdoctoral research associate at Princeton University, and as a consulting assistant professor at Stanford University. He was a research associate at CTTC in Barcelona, Spain until September 2012, when he joined the Electrical and Electronic Engineering Department of Imperial College London, UK, where he is currently a Reader (Associate Professor) in information theory and communications, and leads the Information Processing and Communications Laboratory (IPC-Lab).

His research interests lie in the areas of communications and information theory, machine learning, and privacy. Dr. Gündüz is an Editor of the 91�Թ��� Transactions on Green Communications and Networking, and a Guest Editor of the 91�Թ��� Journal on Selected Areas in Communications, Special Issue on Machine Learning in Wireless Communication. He served as an Editor of the 91�Թ��� Transactions on Communications from 2013 until 2018. He is the recipient of the 91�Թ��� Communications Society - Communication Theory Technical Committee (CTTC) Early Achievement Award in 2017, a Starting Grant of the European Research Council (ERC) in 2016, 91�Թ��� Communications Society Best Young Researcher Award for the Europe, Middle East, and Africa Region in 2014, Best Paper Award at the 2016 91�Թ��� WCNC, and the Best Student Paper Awards at the 2018 91�Թ��� WCNC and 2007 ISIT. He is serving/ served as the General Co-chair of the 2019 London Symposium on Information Theory, 2016 91�Թ��� Information Theory Workshop, and the 2012 European School of Information Theory.

Sequential Acquisition of Information: From Active Hypothesis Testing to Active Learning to Empirical Function Optimization

Tara Javidi (University of California, San Diego)

Presentation slides

Abstract

This lecture explores an often overlooked connection between the problems of communications with feedback in information theory and that of active hypothesis testing in statistics. This connection, we argue, has significant implications for next generation machine learning algorithms where data is collected actively and/or by cooperative yet local agents. 

Consider a decision maker who is responsible to actively and dynamically collect data/samples so as to enhance the information about an underlying phenomena of interest wh