labml.ai Annotated PyTorch Paper Implementations

A very cool collection of popular deep learning blocks, nicely formatted in the browser with extensive comments. Among others there is a GAT implementation.

Interpretable Deep Learning for New Physics Discovery

In this video, Miles Cranmer (Princeton) discusses a method for converting a neural network into an analytic equation using a particular set of inductive biases. The technique relies on a sparsification of latent spaces in a deep neural network, followed by symbolic regression. In their paper, they demonstrate that they can recover physical laws for various simple and complex systems. For example, they discover gravity along with planetary masses from data; they learn a technique for doing cosmology with cosmic voids and dark matter halos; and they show how to extract the Euler equation from a graph neural network trained on turbulence data.

Fresh picks from ArXiv
This week on ArXiv: SOTA for protein energy prediction, another solution to OGB-LSC challenge, and a new dataset based on Wikipedia 📚

If I forgot to mention your paper, please shoot me a message and I will update the post.

Applications
* X-GGM: Graph Generative Modeling for Out-of-Distribution Generalization in Visual Question Answering
* Structure-aware Interactive Graph Neural Networks for the Prediction of Protein-Ligand Binding Affinity KDD 2021

GNNs
* Local2Global: Scaling global representation learning on graphs via local training
* Ego-GNNs: Exploiting Ego Structures in Graph Neural Networks with William L. Hamilton
* Bridging the Gap between Spatial and Spectral Domains: A Theoretical Framework for Graph Neural Networks
* Large-scale graph representation learning with very deep GNNs and self-supervision with Petar Veličković
* Group Contrastive Self-Supervised Learning on Graphs with Shuiwang Ji

Datasets
* WikiGraphs: A Wikipedia Text - Knowledge Graph Paired Dataset with Oriol Vinyals

Graph Neural Networks User Group: July meeting

This month GNN user group talks about a new release of DGL and applications of GNNs. Please join this Thursday!

4:00 - 4:15 PM (PDT): DGL 0.7 release(Dr. Minjie Wang, Amazon)
4:15 - 4:30 PM (PDT): Storing Node Features in GPU memory to speedup billion-scale GNN training (Dr. Dominique LaSalle, NVIDIA)
4:30 - 5:00 PM (PDT): Locally Private Graph Neural Networks (Sina Sajadmanesh, Idiap Research Institute, Switzerland).
5:00 - 5:30 PM (PDT): Graph Embedding and Application in Meituan (Mengdi Zhang, Meituan).

Header-Only C++ Library for Graph Representation and Algorithms

In case you need the speed of C++ for the well-known graph algorithms there is a nice repo that collects many of them.

Graph Convolutional Neural Networks to Analyze Complex Carbohydrates

A blog post by Daniel Bojar about an application of GNN to analyzing glycan sequences and their proposed GNN architecture called SweetNet. There are other coverages of this work (here and here). The paper is here and the code is here.

Graph Machine Learning research groups: Shuiwang Ji

I do a series of posts on the groups in graph research, previous post is here. The 32nd is Shuiwang Ji, a professor at Texas A&M University. His teams were awarded at OGB-LSC and AI Cures challenges. He also recently advised graph libraries such as MoleculeX and DIG.

Shuiwang Ji (~1982)
- Affiliation: Texas A&M University
- Education: Ph.D. at Arizona State University in 2008 (advisor: Jieping Ye)
- h-index 44
- Interests: GNNs, self-supervised learning, surveys, libraries.
- Awards: best papers at KDD, WWW, ACM Distinguished Member

GNN User Group Videos

Videos from the last Thursday meeting of GNN user group are available now. This includes updates of DGL library, storing node feature for large graphs, and locally private GNNs.

Fresh picks from ArXiv
This week on ArXiv: time series recovery, GNN challenge winning solutions, and benchmark for scene graph generation 🌳

If I forgot to mention your paper, please shoot me a message and I will update the post.

Applications
* Multivariate Time Series Imputation by Graph Neural Networks
* Temporal-Relational Hypergraph Tri-Attention Networks for Stock Trend Prediction
* Graph Constrained Data Representation Learning for Human Motion Segmentation
* The Graph Neural Networking Challenge: A Worldwide Competition for Education in AI/ML for Networks
* Image Scene Graph Generation (SGG) Benchmark
* Structack: Structure-based Adversarial Attacks on Graph Neural Networks

Foundations of Graph Neural Networks Course

A new upcoming course by Zak Jost (you may remember his videos on GNNs) on the foundations of GNN which covers such topics as
- Neural Message Passing
- Fourier Transforms, Graph Wavelets and Spectral Convolutions
- Permutation Symmetries
- Representational capacity of GNNs
- Graph fundamentals like the Laplacian and graph isomorphism.

Graph Neural Networks: Algorithms and Applications

A great presentation by Jian Tang about GNN basics, training many layers, self-supervised learning and statistical relational learning.

Knowledge Graphs in Natural Language Processing @ ACL 2021

A regular update from Michael Galkin on the SOTA applications of KG in the world of words:

Neural Databases & Retrieval
KG-augmented Language Models
KG Embeddings & Link Prediction
Entity Alignment
KG Construction, Entity Linking, Relation Extraction
KGQA: Temporal, Conversational, and AMR.

Essays on Data Science

A great collection of blog posts on machine learning and computer science covering topics such as infinitely wide neural nets, markov models, and graph deep learning.

GDL Course

A course that follows closely the geometric deep learning book. It contains 12 lectures, 2 tutorials, and 4 seminars covering topics such as graphs, sets, grids, groups, geodesics, gauges, and time warping. Videos and slides are available.

Fresh picks from ArXiv
This week on ArXiv: PDE-inspired GNNs, proves to the conjectures, and a new benchmark for graph completion 🧵

If I forgot to mention your paper, please shoot me a message and I will update the post.

GNNs
* Distilling Holistic Knowledge with Graph Neural Networks ICCV 2021
* Fully Hyperbolic Graph Convolution Network for Recommendation CIKM 2021
* Jointly Attacking Graph Neural Network and its Explanations
* LEO: Learning Energy-based Models in Graph Optimization
* PDE-GCN: Novel Architectures for Graph Neural Networks Motivated by Partial Differential Equations
* Shift-Robust GNNs: Overcoming the Limitations of Localized Graph Training data with Bryan Perozzi
* AdaGNN: A multi-modal latent representation meta-learner for GNNs based on AdaBoosting

Math
* Isomorphisms between random graphs
* Edge Partitions of Complete Geometric Graphs (Part 1)

Survey
* Are Missing Links Predictable? An Inferential Benchmark for Knowledge Graph Completion
* Influence Maximization in Social Networks: A Survey of Behaviour-Aware Methods

Awesome Efficient Graph Neural Networks

A new awesome repo by Chaitanya K. Joshi with the curated list of must-read papers on efficient Graph Neural Networks and scalable Graph Representation Learning for real-world applications.

Book: Designing and Building Enterprise Knowledge Graphs (Synthesis Lectures on Data, Semantics, and Knowledge)

A new book by Ora Lassila and Juan Sequeda that guides on designing and building knowledge graphs from enterprise relational databases in practice. It presents a principled framework centered on mapping patterns to connect relational databases with knowledge graphs, the roles within an organization responsible for the knowledge graph, and the process that combines data and people. The content of this book is applicable to knowledge graphs being built either with property graph or RDF graph technologies.

Graph Machine Learning research groups: Ian Davidson

I do a series of posts on the groups in graph research, previous post is here. The 33rd is Ian Davidson, a professor at UC Davis, who works in the areas with societal impacts such as neuroscience, intelligent tutoring systems and social networks.

Ian Davidson (~1973)
- Affiliation: UC Davis
- Education: Ph.D. at Monash University in 2000 (advisor: C.S. Wallace)
- h-index 44
- Interests: fairness, clustering, graphical models.
- Awards: best papers at KDD, SIAM, ICDM

TorchDrug: a powerful and flexible machine learning platform for drug discovery

Jian Tang and his co-workers from MILA open-sourced a new library TorchDrug on drug modeling with machine learning. It includes an easy interface for property prediction, pretrained molecular representations, de-novo molecule design & optimization, knowledge graph reasoning, and more.

Fresh picks from ArXiv
This week on ArXiv: GNNs for lidars, complex reasoning in knowledge graphs, and building AI for drawing graphs 🤖

If I forgot to mention your paper, please shoot me a message and I will update the post.

GNNs
* Generative and Contrastive Self-Supervised Learning for Graph Anomaly Detection
* TabGNN: Multiplex Graph Neural Network for Tabular Data Prediction
* Learning to Match Features with Seeded Graph Matching Network
* EqGNN: Equalized Node Opportunity in Graphs


Applications
* GP-S3Net: Graph-based Panoptic Sparse Semantic Segmentation Network
* Adaptive Graph Convolution for Point Cloud Analysis
* Hyperbolic Hypergraphs for Sequential Recommendation
* Implementation of Sprouts: a graph drawing game

Knowledge graphs
* Fact-Tree Reasoning for N-ary Question Answering over Knowledge Graphs
* UNIQORN: Unified Question Answering over RDF Knowledge Graphs and Natural Language Text

Video: Graph Neural Networks - a perspective from the ground up

A beautiful video about GNNs aimed at CS undergrads that explains what message passing and node embeddings are and gives a link prediction example.