Marco Mondelli

 

Youth in High Dimensions 2024

Together with Jean Barbier (ICTP), Marylou Gabrié (École Polytechnique) and Sebastian Goldt (SISSA), I am organizing the fifth edition of the conference Youth in high dimensions that will take place at ICTP in Trieste from May 20 to May 23, 2024. Our goal is to give emerging researchers the opportunity to gather and to present their results, focusing on high-dimensional problems arising in machine learning, inference, statistical physics, and neuroscience. Registration is free and possible via our website by March 31 2024. The deadline for applicants requesting financial and/or Visa support is 3 March 2024. We have a great line-up of invited speakers and we will have a poster session during the conference. See you in Trieste in May!

 

Open Positions

• I have open PhD positions. I am actively looking for PhD students with strong mathematical background and interest in the broad areas of machine learning, high-dimensional statistics, information theory and wireless communication systems. Prospective PhD students are encouraged to apply to the IST Austria graduate school.
• IST Austria offers attractive postdoctoral positions via the IST-BRIDGE program and the NOMIS program. In particular, I am actively looking for a postdoctoral scholar to join an ongoing collaboration with Matthew Robinson, at the interface of information theory, high dimensional inference and quantitative genomics. For additional information about this, see also here. More generally, I am willing to support outstanding candidates interested in machine learning with a focus on theoretical foundations of deep learning and non-convex optimization in high dimensions. For questions about the position and the application process, please contact me via email. The application should contain a CV, a publication list and a research statement. Reference letters will be requested at a later stage.

 

Ph.D. Students and Postdocs

I am extremely lucky to be able to work with the following Ph.D. students and postdocs.

• Yihan Zhang (Postdoc)
• Amedeo Esposito (Postdoc)
• Aleksandr Shevchenko (Ph.D. student co-advised with Dan Alistarh)
• Simone Bombari (Ph.D. student)
• Kevin Kögler (Ph.D. student)
• Al Depope (Ph.D. student co-advised with Matthew Robinson)
• Peter Sukenik (Ph.D. student co-advised with Christoph Lampert)
• Diyuan Wu (Ph.D. student)

 

Selected Recent/Future Service

• NeurIPS 2024 (Senior Area Chair), ICML 2024 (Area Chair), ITW 2024 (TPC Member), Youth in High Dimensions 2024 (Co-organizer)
• NeurIPS 2023 (Area Chair), ICML 2023 (Area Chair), COLT 2023 (General PC), ISIT 2023 (TPC Member), ITW 2023 (TPC Member), ISTC 2023 (TPC Member), Youth in High Dimensions 2023 (Co-organizer)
• NeurIPS 2022 (Area Chair), ICML 2022 (Area Chair), ITW 2022 (TPC Member), Youth in High Dimensions 2022 (Co-organizer)
• ISIT 2021 (TPC Member), ISTC 2021 (TPC Member), Youth in High Dimensions 2021 (Co-organizer)
• ITW 2020 (TPC Member), ISTC 2020 (TPC Member), ISIT 2019 (TPC Member), ISIT 2018 (TPC Member)

 

Latest Updates

• March 2024: New paper on arXiv: "Light-speed whole genome association testing and prediction via Approximate Message Passing" (joint with Al and Matt). A preliminary version ("Inference of Genetic Effects via Approximate Message Passing") will be presented at the 2024 IEEE International Conference on Acoustics, Speech and Signal Processing.
• February 2024: New papers on arXiv: "Compression of Structured Data with Autoencoders: Provable Benefit of Nonlinearities and Depth" (joint with Kevin, Alex and Hamed), "Towards Understanding the Word Sensitivity of Attention Layers: A Study via Random Features" (joint with Simone), "Contraction of Markovian Operators in Orlicz Spaces and Error Bounds for Markov Chain Monte Carlo" (joint with Amedeo), and "Average gradient outer product as a mechanism for deep neural collapse" (joint with Daniel, Peter and Misha).
• February 2024: Paper "Concentration without Independence via Information Measures" (joint with Amedeo) accepted in IEEE Transactions on Information Theory. Congratulations Amedeo!
• November 2023: Simone will present our paper on "Privacy at Interpolation: Precise Analysis for Random and NTK Features" at the NeurIPS 2023 Workshop on Mathematics of Modern Machine Learning, and Francesco will present our paper on "Improved Convergence of Score-Based Diffusion Models via Prediction-Correction" (joint with Jan) at the NeurIPS 2023 Workshop on Diffusion Models.
• September 2023: Paper accepted as a spotlight at the 2023 Conference on Neural Information Processing Systems (NeurIPS'23): "Deep Neural Collapse Is Provably Optimal for the Deep Unconstrained Features Model" (joint with Peter and Christoph). Congratulations Peter!
• August 2023: New paper on arXiv: "Spectral Estimators for Structured Generalized Linear Models via Approximate Message Passing" (joint with Yihan, Hong Chang and Ramji).
• July 2023: Paper "Fundamental limits in structured principal component analysis and how to reach them" (joint with Jean, Francesco and Manuel) accepted in the Proceedings of the National Academy of Sciences (PNAS).
• April 2023: Two papers accepted as oral presentations at the 2023 International Conference on Machine Learning (ICML'23): "Beyond the Universal Law of Robustness: Sharper Laws for Random Features and Neural Tangent Kernels" (joint with Simone and Shayan), and "Fundamental Limits of Two-layer Autoencoders, and Achieving Them with Gradient Methods" (joint with Alex, Kevin and Hamed). Congratulations Simone, Alex and Kevin!
• April 2023: Two papers accepted at the 2023 IEEE International Symposium on Information Theory (ISIT'23): "Mismatched estimation of non-symmetric rank-one matrices corrupted by structured noise" (joint with Teng, YuHao, Jean, ShanSuo and TianQi), and "Concentration without Independence via Information Measures" (joint with Amedeo). Congratulations Amedeo!
• March 2023: Tutorial on "Approximate Message Passing for High-Dimensional Inference" (with Cynthia Rush and Ramji Venkataramanan) accepted at the 2023 IEEE International Symposium on Information Theory (ISIT'23).
• February 2023: Paper "Approximate Message Passing for Multi-Layer Estimation in Rotationally Invariant Models" (joint with Yizhou, TianQi and ShanSuo) accepted at the IEEE Information Theory Workshop 2023 (ITW'23).
• February 2023: Paper "Mean-field analysis for heavy ball methods: Dropout-stability, connectivity, and global convergence" (joint with Diyuan and Vyacheslav) accepted to Transactions on Machine Learning Research (TMLR). This was also presented at the OPT 2022 NeurIPS workshop. Congratulations Diyuan!
• November 2022: Two new papers on arXiv: "Finite Sample Identification of Wide Shallow Neural Networks with Biases" (joint with Massimo, Timo and Michael), and "Precise Asymptotics for Spectral Methods in Mixed Generalized Linear Models" (joint with Yihan and Ramji).
• September 2022: Two papers accepted at the 2022 Conference on Neural Information Processing Systems (NeurIPS'22): "The price of ignorance: how much does it cost to forget noise structure in low-rank matrix estimation?" (joint with Jean, TianQi and Manuel), and "Memorization and Optimization in Deep Neural Networks with Minimum Over-parameterization" (joint with Simone and Mohammad). Congratulations Simone and Mohammad! Simone will also present this work as a talk at the 2022 Conference on the Mathematical Theory of Deep Neural Networks.
• September 2022: Paper "Decoding Reed-Muller Codes with Successive Codeword Permutations" (joint with Nghia, Ali and Warren) accepted in IEEE Transactions on Wireless Communications.
• August 2022: Paper "Sharp asymptotics on the compression of two-layer neural networks" (joint with Mohammad, Simone, Rattana and Stefano) accepted at the IEEE Information Theory Workshop 2022 (ITW'22). Congratulations Mohammad, Simone and Rattana!
• June 2022: Paper "Approximate Message Passing with Spectral Initialization for Generalized Linear Models" (joint with Ramji) invited to the 2022 Machine Learning Special Issue of the Journal of Statistical Mechanics, Theory and Experiment (JSTAT). This paper was previously accepted at the 2021 International Conference on Artificial Intelligence and Statistics (AISTATS'21).
• May 2022: Paper "Estimation in Rotationally Invariant Generalized Linear Models via Approximate Message Passing" (joint with Ramji and Kevin) accepted at the 2022 International Conference on Machine Learning (ICML'22). Congratulations Kevin!
• April 2022: Paper "Mean-field Analysis of Piecewise Linear Solutions for Wide ReLU Networks" (joint with Alex and Vyacheslav) accepted to the Journal of Machine Learning Research (JMLR). Congratulations Alex! A poster about this work was presented at the 2021 Conference on the Mathematical Theory of Deep Neural Networks.
• April 2022: Paper "Polar Coded Computing: The Role of the Scaling Exponent" accepted at the 2022 IEEE International Symposium on Information Theory (ISIT'22). Congratulations Dorsa!
• December 2021: Two papers presented at the 2021 Conference on Neural Information Processing Systems (NeurIPS'21): "PCA Initialization for Approximate Message Passing in Rotationally Invariant Models" (joint with Ramji), and "When Are Solutions Connected in Deep Networks?" (joint with Quynh and Pierre).
• November 2021: Paper "Parallelism versus Latency in Simplified Successive-Cancellation Decoding of Polar Codes" (joint with Ali, Arman, Alexander, John and Andrea) accepted in IEEE Transactions on Wireless Communications. This work was presented at the 2021 IEEE International Symposium on Information Theory (ISIT'21).
• November 2021: Paper "Successive Syndrome-Check Decoding of Polar Codes" (joint with Ali, John and Andrea) accepted at the 2021 Asilomar Conference on Signals, Systems, and Computers.
• July 2021: Honored to have received the 2021 Information Theory Society Paper Award jointly with Shrinivas Kudekar, Santhosh Kumar, Henry Pfister, Eren Şaşoǧlu and Rüdiger Urbanke for the paper "Reed–Muller Codes Achieve Capacity on Erasure Channels".
• July 2021: Paper "Optimal Combination of Linear and Spectral Estimators for Generalized Linear Models" (joint with Christos and Ramji) accepted in Foundations of Computational Mathematics (FoCM).
• May 2021: Paper "Tight Bounds on the Smallest Eigenvalue of the Neural Tangent Kernel for Deep ReLU Networks" (joint with Quynh and Guido) accepted at the 2021 International Conference on Machine Learning (ICML'21).
• April 2021: Paper "Sparse Multi-Decoder Recursive Projection Aggregation for Reed-Muller Codes" (joint with Dorsa, Nariman and Ali) accepted at the 2021 IEEE International Symposium on Information Theory (ISIT'21). Congratulations Dorsa!
• December 2020: Paper "Global Convergence of Deep Networks with One Wide Layer Followed by Pyramidal Topology" (joint with Quynh) presented at the 2020 Conference on Neural Information Processing Systems (NeurIPS'20).
• June 2020: Paper "Landscape Connectivity and Dropout Stability of SGD Solutions for Over-parameterized Neural Networks" accepted at the 2020 International Conference on Machine Learning (ICML'20). Congratulations Alex!
• January 2020: Paper "Analysis of a Two-Layer Neural Network via Displacement Convexity" (joint with Adel and Andrea) accepted in Annals of Statistics.
• June 2019: Awarded the 2019 Lopez-Loreta Prize: 1M EUR for the 5-year research project "Foundations of Deep Learning".
• July 2018: Awarded the 2018 EPFL Doctorate Award.