Item List

3D Shape matching is a fundamental problem in computer vision and graphics with significant applications on biological data. In this project, we will take a closer look at Deep Functional Map (DFM) [1,2] paradigm for 3D shape matching.  You will become familiar with unsupervised DFM frameworks [2] and investigate a couple of directions less explored in the DFM literature. First, most DFM literature strongly relies on precomputed Laplacian Beltrami (LB) eigenbasis. There have been some recent attempts to learn an embedding [3,4] instead. However, it is not entirely clear in which circumstances learned embedding is more robust and useful than LB eigenbasis. Secondly, we will investigate cycle consistency constraints in DFM that provide a strong regularization by jointly optimizing the maps over a collection of shapes. By the end of the project, you should better understand these topics from both theoretical and practical perspectives.

References

[1] Litany et al., Deep Functional Map: Structure prediction for dense shape correspondence, ICCV 2017

[2] Roufosse et al., Unsupervised Deep Learning for 3D shape Matching, ICCV 2019

[3] Marin et al., Correspondence Learning via Linearly Invariant Embedding, Neurips 2020

[4] Sharma & Ovsjanikov, Joint Symmetry Detection and Shape Matching for Non-rigid Point Cloud, arXiv

First described in 1983, the Fine interior is a key combinatorial tool in Mirror Symmetry. Broadly speaking, a convex lattice polytope P corresponds to a hypersurface Z in a toric variety. Associate to P is the Fine interior F(P): the rational polytope given by moving all supporting hyperplanes of P in by lattice distance 1. Many geometric properties of Z can be deduced from combinatorial properties of F(P). For example, there exists a unique canonical model of Z if F(P) is non-empty, and the Kodaira dimension is determined by the dimension of F(P). Computing the Fine interior F(P) is computationally challenging and, despite being so important, almost nothing is known about how the combinatorics of P determines the dimension of F(P). This is an area perfect for investigation via Machine Learning.

In this project we will explore the classification of certain four-dimension lattice simplices -- those containing a single interior lattice point. Each of these 338,752 simplices can be described uniquely by an integer-valued vector (a_0,...,a_4), and in nearly every case we know the Fine interior as a result of brute-force computations totalling many decades of CPU time. We will ask whether Machine Learning can predict the dimension of F(P) directly from the vector (a_0,...,a_4) and, if successful, attempt to understand how the machine is performing this calculation. This should present us with unique insights into the combinatorics of the Fine interior, which in turn will generate a richer understanding of the underlying geometry.

Cohomology is a universal tool in mathematics: from topology and geometry to algebra and number theory, cohomology is used to quantify the possible ways in which a problem fails to meet the naive expectations of the problem solver. As such, estimates of cohomology represent a vital key to problem solving. In areas of theoretical physics such as string theory, cohomology is linked to the low-energy quantum excitations of fields and strings that can be experimentally observed.

In practice, cohomology computations are notoriously difficult to carry out in general. However, it has been recently shown that line bundle cohomology dimensions on several classes of two-dimensional and three-dimensional complex manifolds, including compact toric surfaces, generalised del Pezzo surfaces, K3 surfaces and Calabi-Yau threefolds, are described by closed-form formulae. This new technique for the computation of cohomology uses a combination of algebro-geometric methods and exploratory machine learning methods. The goal of the project will be to go one step further and discover exact generating functions for line bundle cohomology on Calabi-Yau threefolds.

References:

[1] C. Brodie, A. Constantin, J. Gray, A. Lukas, F. Ruehle, Recent Developments in Line Bundle Cohomology and Applications to String Phenomenology, Nankai Symposium on Mathematical Dialogues 2021 Conference Proceedings, arXiv: 2112.12107.

[2] C. Brodie, A. Constantin, R. Deen, A. Lukas, Machine Learning Line Bundle Cohomology, Fortsch.Phys. 68 (2020) 1, 1900087, arXiv: 1906.08730.

[3] C. Brodie, A. Constantin, A. Lukas, Flops, Gromov-Witten invariants and symmetries of line bundle cohomology on Calabi-Yau three-folds, J.Geom.Phys. 171 (2022), arXiv: 2010.06597.

[4] C. Brodie, A. Constantin, Cohomology Chambers on Complex Surfaces and Elliptically Fibered Calabi-Yau Three-folds, arXiv: 2009.01275.

Many machine learning tasks require learning functions that are invariant or equivariant to known symmetries of the input data. Unfortunately, there is a significant challenge in the design of neural network architectures that simultaneously (a) take into account the symmetries, (b) are expressive enough to have small generalization error, and (c) are computationally efficient. Murphy et al. [1, 2] have shown that it is possible to sacrifice (c) --computational efficiency-- for (a) and (b) with the use of symmetrization (Reynolds operator). Recently, Puny et al. [3] have proposed a method (Frame Averaging) to improve the efficiency of symmetrization. However, in some tasks, Frame Averaging can lead to large generalization errors. This is because of a fundamental trade-off between computationally efficient and generalization error in symmetrization. This project will study this trade-off and propose efficient solutions to achieve both computational efficiency and better generalization error in the affected tasks.

References

[1] Murphy, Ryan L., Balasubramaniam Srinivasan, Vinayak Rao, and Bruno Ribeiro. Janossy pooling: Learning deep permutation-invariant functions for variable-size inputs. ICLR 2019.

[2] Murphy, Ryan, Balasubramaniam Srinivasan, Vinayak Rao, and Bruno Ribeiro. Relational pooling for graph representations. In International Conference on Machine Learning, pp. 4663-4673. PMLR, 2019.

[3] Puny, Omri, Matan Atzmon, Heli Ben-Hamu, Edward J. Smith, Ishan Misra, Aditya Grover, and Yaron Lipman. Frame Averaging for Invariant and Equivariant Network Design. ICLR 2022.

Introduction. Graph Neural Networks (GNNs) are increasingly used as powerful tools for representing graph-structured data, such as social, information, chemical, and biological networks. As these models are deployed in critical applications (e.g., drug repurposing and crime forecasting), it becomes essential to ensure that the relevant stakeholders understand and trust their decisions. To this end, several approaches have been proposed in recent literature to explain the predictions of GNNs. Depending on the employed techniques, there are three broad categories: perturbation-based, gradient-based, and surrogate-based methods. While several classes of GNN explanation methods have been proposed, there is little to no work done on showing how to use these explanations to improve the GNN performance. In particular, there is no framework that leverages these explanations on the fly and aids the training of a GNN. This lack of understanding mainly stems from the fact that there is very little work on systematically analyzing the use of explanations generated by state-of-the-art GNN explanation methods 

Proposal. Previous research in GraphXAI has focused on developing post-hoc explanation methods. In this work, we propose in-hoc GNN explanations that act as feedbacks, on the fly, during the training phase of a GNN model. In addition, we aim to use the generated explanations to improve the GNN training. Using explanations, we plan to define local neighborhoods for neural message passing, e.g., for a correctly classified node u, we can generate the most important nodes in its local neighborhood N_u and then use it as a prior or generate augmented samples for guiding the message-passing of similar nodes in the subsequent training stages. To this extent, we propose to have an explanation layer after every message-passing layer which acts as a unit buffer that passes all the information to the upper layers during the forward pass, but propagates the explanation information back to the graph representations.

Cellular sheaves [1, 2] equip graphs with a "geometrical" structure by attaching vector spaces and linear maps to their nodes and edges. It turns out that this additional structure can help mitigate some of the well-known problems of Graph Neural Networks. In a recent paper [3], we studied neural diffusion PDEs on sheaves and proved they have many desirable properties, such as better performance in heterophilic settings than GNNs and robust behaviour in the infinite-time limit (i.e. infinitely many layers).

In this project, we aim to extend this approach to other sheaf-based PDEs that behave differently from diffusion, leading to novel Sheaf Neural Network models. We will be studying the theoretical properties of these neural PDEs (e.g. Do they minimise some energy? Are the dynamics stable?) with the ultimate goal of building practical models for node classification and regression tasks.

References

[1] Jakob Hansen, Robert Ghrist, Toward a Spectral Theory of Cellular Sheaves, Journal of Applied and Computational Topology volume 3, pages 315–358 (2019)

[2] Jakob Hansen, Thomas Gebhart, Sheaf Neural Networks, NeurIPS 2020 Workshop on TDA and Beyond

[3] Bodnar et al., Neural Sheaf Diffusion: A Topological Perspective on Heterophily and Oversmoothing in GNNs, Preprint 2022

Learning to represent and generate music is a highly relevant task for the machine learning field. This data domain is ideal for density estimation tasks and exhibits many interesting properties, such as long-term dependencies and patterns residing at various scales. More crucially, music generation is a main pillar of creative AI, which complements the ML community efforts in pushing scientific progress and gives us the amazing opportunity to assist artists in their creative process [0]!

Existing state-of-the-art approaches for symbolic music generation [1, 2] operate on and produce sequences of tokens. However, the music domain contains structure at several scales: local (e.g. chords, arpeggios, multiple voices being played together in a fugue) and long-term/global (e.g. ABA form [3], the key that a piece is written in, repeating patterns/motifs). In that sense, there have been relatively few studies (e.g. [4, 5, 6, 7, 8]) that investigate explicit representations of structure for modelling and generating music. Each of these works has certain limitations—for instance, each piece modelled in [4] (see Fig. 2) is turned into a single graph by connecting all notes with temporal links, but does not capture the fundamental music-theoretical dependencies and insights that composers most likely used when writing the piece. Alternatively, the work in [7] and a few others look at rhythm-specific encodings only. Perhaps the closest modelling strategy to the one intended for this project is [8], where the authors choose to encode various musical relationships between bars of the score; however, it would be easier to work with more general graph representations of music, building up from first principles such as the circle of fifths / the tonality of a piece (e.g. use a sliding window and compute relationships / a graph within each local window and between windows at discrete points in the score - there are endless ways to think about this!)

This project aims to study the effects of leveraging music-theoretical graph representations in ML models. The goal is to encode symbolic music sequences in a principled manner that reflects the composition process and underlying structure up to a greater extent that previously. This encoding would then be passed as (additional) input to a model [1, 2, 9]. Finally, the effects on model performance would be studied in classification and/or generation scenarios (TBD based on time constraints).

Steps:

1. Download dataset(s), choose task(s)

2. Become familiar with basic music theory concepts and design 1 or more ways of encoding the structure in symbolic music (see Appendix A.2 [1] for a description of event-based MIDI representations; see Chordify in Resources; see Wiki)

3. Set up model codebase and obtain baseline performance on chosen tasks

4. Add relational structure to the model and find suitable ways to encode it

5. Interpret the new results and investigate changes in model processing (e.g. visualising attention in layers, emerging patterns)

6. Open-source code, to allow researchers to preprocess their own music data and build more graph-based ML models for music tasks!

Resources

- Music Transformer GitHub codebase

- MusicVAE GitHub codebase

- Perceiver AR codebase (to be released soon)

- MAESTRO dataset

- MetaMIDIDataset

- Lakh MIDI dataset

- MusPy - A toolkit for symbolic music generation

- Chordify - music21 Documentation

- Symbolic Representations

- Fundamentals of music

- A structural way to encode music

(https://en.wikipedia.org/wiki/Tonnetz) studied by https://ojs.aaai.org/index.php/AAAI/article/view/11880 (here, the resulting encoding was an image)

References

[0] Music 2020 Archives - AI Art Gallery

(http://www.aiartonline.com/category/music-2020/)

[1] Music Transformer

[2] General-purpose, long-context autoregressive modeling with Perceiver AR

[3] Ternary form

[4] Graph Neural Network for Music Score Data and Modeling Expressive Piano Performance

[5] StructureNet: Inducing Structure in Generated Melodies - IR Anthology

[6] Neurosymbolic Deep Generative Models for Sequence Data with Relational Constraints

[7] Pop Music Transformer: Beat-based Modeling and Generation of Expressive Pop Piano Compositions

[8] MELONS: generating melody with long-term structure using transformers and structure graph

[9] A Hierarchical Latent Vector Model for Learning Long-Term Structure in Music

There are many group invariant machine learning models, i.e. learnable functions that give the same output if the input is acted on by a group. One novel approach is using fundamental domain projections - an approach which is particularly useful if the group which acts is very large [1]. There are many large string theory datasets with large symmetry groups, which make good benchmarks for group invariant machine learning models. One such example is the Kreuzer-Skarke dataset of Calabi-Yau three-folds coming from reflexive polyhedra and their Hodge numbers [2].

In this project, we will apply invariant machine learning via fundamental domain projections to the Kreuzer-Skarke dataset and compare this with other group invariant machine learning techniques (e.g. data augmentation and deep sets), as well models that are not group invariant.

References

[1] Group invariant machine learning by fundamental domain projections, Benjamin Aslan, Daniel Platt, David Sheard, arXiv 2022

[2] Calabi-Yau data

Most GNNs are based on the concept of message passing, which is by itself based on information diffusion. In diffusion dynamics, key information lies in closer objects, and distant nodes’ effect is decimated [1]. However, it is not clear that the topological graph structure must dictate the information transfer on the graph. In fact, in many cases, such as combinatorial optimization problems, nodes and edges that are distant from a node may have a major impact on the node’s value or class. To that end, graph rewiring allows adding edges, nodes, or other structures in order to assist information transfer. In practice, it decouples the information graph from the topological (input) graph. In this project, we will investigate how we can (meta) learn to build better information graphs using RL. Specifically, our agent will learn how to modify (add/remove) edges, i.e., perform graph rewiring, to improve learning. Our goal is to publish the results of this project in a top ML conference.

References

[1] Understanding over-squashing and bottlenecks on graphs via curvature, Topping et. al., 2022.

[2] On the bottleneck of graph neural networks and its practical implications, Alon et. al., 2020

In this project we will investigate possible ways to formalize and understand the notion of graph-rewiring in the context of Graph Neural Networks from a geometric perspective. In certain regimes - existence of long-range dependencies that are crucial for the classification task or heterophily of the label (feature) distribution - the graph structure is known not to be beneficial and, in some cases, even harmful. However, a clear understanding of how classical GNNs behave with respect to specific topological transformations is still missing. Providing a clearer picture in this regard is intimately connected with the problem of stability of GNNs with respect to topological perturbations and might also shed some light on tackling expressivity from a different angle. The main goal of the project consists in studying notions of distances among graphs and associated classes of transformations that would allow us to better tackle the problem of graph-rewiring and analyse theoretically its effects on GNNs.

Steerable CNNs [1] and, in particular, Euclidean Steerable CNNs [2, 3] provide a general framework for building neural networks which are equivariant to groups beyond translations, for example reflections and rotations. Euclidean Steerable CNNs rely on standard convolution with steerable filters, i.e. filters satisfying a steerability constraint [2, 3, 4]. Filters are typically parameterized by linear combination of a steerable filter basis [4], obtained by analytically solving the steerability constraint. However, we note that if the filter is parameterized by an implicit MLP [5] (e.g. for point cloud data), the steerability constraint just requires the MLP to be equivariant. This suggests a simpler way to build steerable CNNs, which does not rely on the polar decomposition of the filters and spherical harmonics. Moreover, this strategy enjoys the flexibility of the implicit neural function, supporting wider filters with limited parameter cost. Since no steerable basis is required, this approach can also be easily extended to different equivariance groups with minor changes, provided an equivariant MLP can be built. We will explore applications for point cloud data (with full rotational symmetries or with only azimuthal symmetries) and molecular data.

References

[1] A General Theory of Equivariant CNNs on Homogeneous Spaces, Taco Cohen and Mario Geiger and Maurice Weiler

[2] 3D Steerable CNNs: Learning Rotationally Equivariant Features in Volumetric Data, Maurice Weiler and Mario Geiger and Max Welling and Wouter Boomsma and Taco Cohen

[3] Tensor field networks: Rotation- and translation-equivariant neural networks for 3D point clouds, Nathaniel Thomas and Tess Smidt and Steven Kearnes and Lusann Yang and Li Li and Kai Kohlhoff and Patrick Riley

[4] A Program to Build E(N)-Equivariant Steerable CNNs, Gabriele Cesa and Leon Lang and Maurice Weiler

[5] PointConv: Deep Convolutional Networks on 3D Point Clouds, Wenxuan Wu and Zhongang Qi and Li Fuxin

A common hypothesis in machine learning is that the data lie near a low dimensional manifold which is embedded in a high dimensional ambient space. This implies that shortest paths between points should respect the underlying geometric structure. In practice, we can capture the geometry of a data manifold through a Riemannian metric in the latent space of a stochastic generative model, relying on meaningful uncertainty estimation for the generative process. This enables us to compute identifiable distances, since the length of the shortest path remains invariant under re-parametrizations of the latent space. Consequently, we are able to study the learned latent representations beyond the classic Euclidean perspective.

In this project you will develop methods to learn Riemannian metrics in the latent space of deep generative models. You will then use the learned metrics for computing shortest paths in the representation space and for fitting a statistical model that respects the learned nonlinear geometry of the data.

References

(*) Latent Space Oddity: on the Curvature of Deep Generative Models, Georgios Arvanitidis, Lars Kai Hansen, Søren Hauberg, International Conference on Learning Representations (ICLR), 2018.

(*) A prior-based approximate latent Riemannian metric, Georgios Arvanitidis, Bogdan Georgiev, Bernhard Schölkopf, International Conference on Artificial Intelligence and Statistics (AISTATS), 2022.

(*) Fast and robust shortest paths on manifolds learned from data, Georgios Arvanitidis, Søren Hauberg, Philipp Hennig, Michael Tiemann, International Conference on Artificial Intelligence and Statistics (AISTATS), 2019.

(*) A locally adaptive normal distribution, Georgios Arvanitidis, Lars Kai Hansen, Søren Hauberg, Advances in Neural Information Processing Systems (NeurIPS), 2016.