Marco Mondelli

 

IEEE BITS special issue

Together with Ahmad Beirami (Google), Samet Oymak (University of Michigan), Vatsal Sharan (University of Southern California) and Ananda Theertha Suresh (Google), I am editing a Special Issue on Generative Models of IEEE BITS, The Information Theory Magazine. The topics span alignment, compression, in-context learning, data efficiency and more, see the Call for Paper here. You are encouraged to submit an expository paper intended for a broad audience!

 

Open Positions

• I have open postdoctoral, PhD, and internship positions. I am actively looking for candidates with strong mathematical background and interest in (at least one of) the broad areas of machine learning, high-dimensional statistics, information theory and quantitative genetics. I have multiple open positions in the context of an ERC Starting Grant awarded for the project "Inference in High Dimensions: Light-speed Algorithms and Information Limits". Prospective interns are encouraged to contact me directly via email. Prospective PhD students are encouraged to apply to the IST Austria graduate school and via ELLIS. Prospective postdocs are encouraged to contact me directly via email with a CV, a publication list and a research statement; reference letters will be requested at a later stage. IST Austria also offers attractive postdoctoral positions via the IST-BRIDGE program and the NOMIS program.

 

Ph.D. Students and Postdocs

I am extremely lucky to be able to work with the following Ph.D. students and postdocs.

• Yihan Zhang (Postdoc)
• Aleksandr Shevchenko (Ph.D. student co-advised with Dan Alistarh)
• Simone Bombari (Ph.D. student)
• Kevin Kögler (Ph.D. student)
• Al Depope (Ph.D. student co-advised with Matthew Robinson)
• Peter Sukenik (Ph.D. student co-advised with Christoph Lampert)
• Diyuan Wu (Ph.D. student)

...and to have worked with the following alumni!

• Amedeo Esposito, former Postdoc → tenure-track Assistant Professor at the Okinawa Institute of Science and Technology (OIST)

 

Selected Recent/Future Service

• ICLR 2025 (Area Chair)
• NeurIPS 2024 (Senior Area Chair), ISIT 2024 (Data Set Competition Chair), ICML 2024 (Area Chair), ITW 2024 (TPC Member), Youth in High Dimensions 2024 (Co-organizer)
• NeurIPS 2023 (Area Chair), ICML 2023 (Area Chair), COLT 2023 (General PC), ISIT 2023 (TPC Member), ITW 2023 (TPC Member), ISTC 2023 (TPC Member), Youth in High Dimensions 2023 (Co-organizer)
• NeurIPS 2022 (Area Chair), ICML 2022 (Area Chair), ITW 2022 (TPC Member), Youth in High Dimensions 2022 (Co-organizer)
• ISIT 2021 (TPC Member), ISTC 2021 (TPC Member), Youth in High Dimensions 2021 (Co-organizer)
• ITW 2020 (TPC Member), ISTC 2020 (TPC Member), ISIT 2019 (TPC Member), ISIT 2018 (TPC Member)

 

Latest Updates

• November 2024: Congratulations to Simone for receiving a Google PhD Fellowship!
• October 2024: New papers on arXiv: "Wide Neural Networks Trained with Weight Decay Provably Exhibit Neural Collapse" (joint with Arthur, Peter and Zihan), "High-dimensional Analysis of Knowledge Distillation: Weak-to-Strong Generalization and Scaling Laws" (joint with Emrullah, Alperen, Onur and Samet), and "Privacy for Free in the Over-Parameterized Regime" (joint with Simone). Simone will give an oral presentation of the last work at DeepMath 2024.
• September 2024: Thrilled to be a recipient of an ERC Starting Grant for my project "Inference in High Dimensions: Light-speed Algorithms and Information Limits". I will be hiring soon at all levels (postdocs, PhD students and interns).
• September 2024: Three papers accepted at the 2024 Conference on Neural Information Processing Systems (NeurIPS'24): "Average gradient outer product as a mechanism for deep neural collapse" (joint with Daniel, Peter and Misha), "Neural Collapse versus Low-rank Bias: Is Deep Neural Collapse Really Optimal?" (joint with Peter and Christoph), and "Matrix Denoising with Doubly Heteroscedastic Noise: Fundamental Limits and Optimal Spectral Methods" (joint with Yihan). Congratulations Peter and Yihan!
• August 2024: Congratulations Dr. Aleksandr Shevchenko on defending your Ph.D.!
• July 2024: We just updated our preprint on "Joint modelling of whole genome sequence data for human height via approximate message passing" (joint with Al, Jakub and Matt). A preliminary version ("Inference of Genetic Effects via Approximate Message Passing") was accepted at the 2024 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP'24). This work was accepted as an oral presentation at the 52nd European Mathematical Genetics Meeting (Best student presentation award to Al), at the 9th Probabilistic Modeling in Genomics Conference (ProbGen24), at the 7th International Conference of Quantitative Genetics and as a late breaking abstract at the European Human Genetics Conference (ESHG 2024).
• May 2024: New paper on arXiv: "Information limits and Thouless-Anderson-Palmer equations for spiked matrix models with structured noise" (joint with Jean, Francesco and Yizhou).
• May 2024: Paper "Improved Convergence of Score-Based Diffusion Models via Prediction-Correction" (joint with Francesco and Jan) accepted to Transactions on Machine Learning Research (TMLR). This was also presented at the NeurIPS 2023 Workshop on Diffusion Models. Congratulations Francesco!
• May 2024: Two papers accepted at the 2024 Conference on Learning Theory (COLT'24): "Contraction of Markovian Operators in Orlicz Spaces and Error Bounds for Markov Chain Monte Carlo" (joint with Amedeo), "Spectral Estimators for Structured Generalized Linear Models via Approximate Message Passing" (joint with Yihan, Hong Chang and Ramji). Congratulations Amedeo and Yihan!
• May 2024: Three papers accepted at the 2024 International Conference on Machine Learning (ICML'24): "Compression of Structured Data with Autoencoders: Provable Benefit of Nonlinearities and Depth" (joint with Kevin, Alex and Hamed), "Towards Understanding the Word Sensitivity of Attention Layers: A Study via Random Features" (joint with Simone), and "How Spurious Features are Memorized: Precise Analysis for Random and NTK Features" (joint with Simone). The last paper was also presented at the NeurIPS 2023 Workshop on Mathematics of Modern Machine Learning. Congratulations Kevin, Alex and Simone!
• February 2024: Paper "Concentration without Independence via Information Measures" (joint with Amedeo) accepted in IEEE Transactions on Information Theory. Congratulations Amedeo!
• September 2023: Paper accepted as a spotlight at the 2023 Conference on Neural Information Processing Systems (NeurIPS'23): "Deep Neural Collapse Is Provably Optimal for the Deep Unconstrained Features Model" (joint with Peter and Christoph). Congratulations Peter!
• July 2023: Paper "Fundamental limits in structured principal component analysis and how to reach them" (joint with Jean, Francesco and Manuel) accepted in the Proceedings of the National Academy of Sciences (PNAS).
• April 2023: Two papers accepted as oral presentations at the 2023 International Conference on Machine Learning (ICML'23): "Beyond the Universal Law of Robustness: Sharper Laws for Random Features and Neural Tangent Kernels" (joint with Simone and Shayan), and "Fundamental Limits of Two-layer Autoencoders, and Achieving Them with Gradient Methods" (joint with Alex, Kevin and Hamed). Congratulations Simone, Alex and Kevin!
• April 2023: Two papers accepted at the 2023 IEEE International Symposium on Information Theory (ISIT'23): "Mismatched estimation of non-symmetric rank-one matrices corrupted by structured noise" (joint with Teng, YuHao, Jean, ShanSuo and TianQi), and "Concentration without Independence via Information Measures" (joint with Amedeo). Congratulations Amedeo!
• March 2023: Tutorial on "Approximate Message Passing for High-Dimensional Inference" (with Cynthia Rush and Ramji Venkataramanan) accepted at the 2023 IEEE International Symposium on Information Theory (ISIT'23).
• February 2023: Paper "Approximate Message Passing for Multi-Layer Estimation in Rotationally Invariant Models" (joint with Yizhou, TianQi and ShanSuo) accepted at the IEEE Information Theory Workshop 2023 (ITW'23).
• February 2023: Paper "Mean-field analysis for heavy ball methods: Dropout-stability, connectivity, and global convergence" (joint with Diyuan and Vyacheslav) accepted to Transactions on Machine Learning Research (TMLR). This was also presented at the OPT 2022 NeurIPS workshop. Congratulations Diyuan!

Older Updates

• November 2022: Two new papers on arXiv: "Finite Sample Identification of Wide Shallow Neural Networks with Biases" (joint with Massimo, Timo and Michael), and "Precise Asymptotics for Spectral Methods in Mixed Generalized Linear Models" (joint with Yihan and Ramji).
• September 2022: Two papers accepted at the 2022 Conference on Neural Information Processing Systems (NeurIPS'22): "The price of ignorance: how much does it cost to forget noise structure in low-rank matrix estimation?" (joint with Jean, TianQi and Manuel), and "Memorization and Optimization in Deep Neural Networks with Minimum Over-parameterization" (joint with Simone and Mohammad). Congratulations Simone and Mohammad! Simone will also present this work as a talk at the 2022 Conference on the Mathematical Theory of Deep Neural Networks.
• September 2022: Paper "Decoding Reed-Muller Codes with Successive Codeword Permutations" (joint with Nghia, Ali and Warren) accepted in IEEE Transactions on Wireless Communications.
• August 2022: Paper "Sharp asymptotics on the compression of two-layer neural networks" (joint with Mohammad, Simone, Rattana and Stefano) accepted at the IEEE Information Theory Workshop 2022 (ITW'22). Congratulations Mohammad, Simone and Rattana!
• June 2022: Paper "Approximate Message Passing with Spectral Initialization for Generalized Linear Models" (joint with Ramji) invited to the 2022 Machine Learning Special Issue of the Journal of Statistical Mechanics, Theory and Experiment (JSTAT). This paper was previously accepted at the 2021 International Conference on Artificial Intelligence and Statistics (AISTATS'21).
• May 2022: Paper "Estimation in Rotationally Invariant Generalized Linear Models via Approximate Message Passing" (joint with Ramji and Kevin) accepted at the 2022 International Conference on Machine Learning (ICML'22). Congratulations Kevin!
• April 2022: Paper "Mean-field Analysis of Piecewise Linear Solutions for Wide ReLU Networks" (joint with Alex and Vyacheslav) accepted to the Journal of Machine Learning Research (JMLR). Congratulations Alex! A poster about this work was presented at the 2021 Conference on the Mathematical Theory of Deep Neural Networks.
• April 2022: Paper "Polar Coded Computing: The Role of the Scaling Exponent" accepted at the 2022 IEEE International Symposium on Information Theory (ISIT'22). Congratulations Dorsa!
• December 2021: Two papers presented at the 2021 Conference on Neural Information Processing Systems (NeurIPS'21): "PCA Initialization for Approximate Message Passing in Rotationally Invariant Models" (joint with Ramji), and "When Are Solutions Connected in Deep Networks?" (joint with Quynh and Pierre).
• November 2021: Paper "Parallelism versus Latency in Simplified Successive-Cancellation Decoding of Polar Codes" (joint with Ali, Arman, Alexander, John and Andrea) accepted in IEEE Transactions on Wireless Communications. This work was presented at the 2021 IEEE International Symposium on Information Theory (ISIT'21).
• November 2021: Paper "Successive Syndrome-Check Decoding of Polar Codes" (joint with Ali, John and Andrea) accepted at the 2021 Asilomar Conference on Signals, Systems, and Computers.
• July 2021: Honored to have received the 2021 Information Theory Society Paper Award jointly with Shrinivas Kudekar, Santhosh Kumar, Henry Pfister, Eren Şaşoǧlu and Rüdiger Urbanke for the paper "Reed–Muller Codes Achieve Capacity on Erasure Channels".
• July 2021: Paper "Optimal Combination of Linear and Spectral Estimators for Generalized Linear Models" (joint with Christos and Ramji) accepted in Foundations of Computational Mathematics (FoCM).
• May 2021: Paper "Tight Bounds on the Smallest Eigenvalue of the Neural Tangent Kernel for Deep ReLU Networks" (joint with Quynh and Guido) accepted at the 2021 International Conference on Machine Learning (ICML'21).
• April 2021: Paper "Sparse Multi-Decoder Recursive Projection Aggregation for Reed-Muller Codes" (joint with Dorsa, Nariman and Ali) accepted at the 2021 IEEE International Symposium on Information Theory (ISIT'21). Congratulations Dorsa!
• December 2020: Paper "Global Convergence of Deep Networks with One Wide Layer Followed by Pyramidal Topology" (joint with Quynh) presented at the 2020 Conference on Neural Information Processing Systems (NeurIPS'20).
• June 2020: Paper "Landscape Connectivity and Dropout Stability of SGD Solutions for Over-parameterized Neural Networks" accepted at the 2020 International Conference on Machine Learning (ICML'20). Congratulations Alex!
• January 2020: Paper "Analysis of a Two-Layer Neural Network via Displacement Convexity" (joint with Adel and Andrea) accepted in Annals of Statistics.
• June 2019: Awarded the 2019 Lopez-Loreta Prize: 1M EUR for the 5-year research project "Foundations of Deep Learning".
• July 2018: Awarded the 2018 EPFL Doctorate Award.