Thermonuclear Fusion

heat the deuterium-tritium fuel to a sufficienly high temperature

Necessary temperature

around 10keV, about 100 million degrees centigrade

• In a tokamak the plasma particles are confined to a toroidal region by a magnetic field, being held by the field in small gyrating orbits.

Energy required for ignition

$$\hat n\tau_E\hat T>5\times10^{21}m^{-3}sKeV$$ $$\hat n\ is\ the\ peak\ ion\ density\ in\ the\ plasma$$ $$\hat T\ is\ the\ peak\ temperature\ in\ the\ plasma$$ $$\tau_E\ is\ the\ energy\ confinement\ time$$

• The nuclei of deuterium and tritium fuse to produce an alpha particle with the release of a neutron $$\begin{aligned} _1D^2\ \ \ \ \ \ +\ \ \ \ \ \ _1T^3\longrightarrow _2H&e^4\ \ \ \ \ \ \ \ +&_0n^1\\ &| &| \\ &35MeV\ \ + &&141MeV=176MeV \end{aligned}$$

Reference Book
Tokamaks by John Wesson

Abstract

Graph neural networks are neural models that capture the dependence of graphs via message passing between the nodes of graphs.

• propose a general design pipeline for GNN models

• discuss the variants of each component

• systematically categorize the applications

• propose four open problems for future research

Introduction

Graphs can be used across various areas

social networks

Graph Convolutional Networks with Markov Random Field Reasoning for Social Spammer Detection, 2020

natural science

Graph Networks as Learnable Physics Engines for Inference and Control, 2018

Interaction Networks for Learning about Objects, Relations and Physics, 2016

protein-protein interaction networks

Protein Interface Prediction using Graph Convolutional Networks, 2017

knowledge graphs

Knowledge Transfer for Out-of-Knowledge-Base Entities: A Graph Neural Network Approach, 2017

other research areas

Learning Combinatorial Optimization Algorithms over Graphs, 2017

The fundamental motivations of graph neural networks

• Recursive Neural Networks are first utilized on directed acyclic graphs

  Supervised neural networks for the classification of structures,1997
  A general framework for adaptive processing of data structures, 1998

• Recurent Neural Networks and Feedforward Neural Networks

  The Graph Neural Network Model, 2009
  Neural Network for Graphs: A Contextual Constructive Approach, 2009

• CNNs result in the rediscovery of GNNs

  Gradient-based learning applied to document recognition, 1998

• the new era of deep learning

  Deep learning, 2015

• geometric deep learning

  Geometric deep learning: going beyond Euclidean data, 2017
  

• graph representation learning

  A Survey on Network Embedding, 2017
  Representation Learning on Graphs: Methods and Applications, 2017
  Network Representation Learning: A Survey, 2017
  A Comprehensive Survey of Graph Embedding: Problems, Techniques and Applications, 2017
  Graph embedding techniques, applications, and performance: A survey,2018

• word representations

  Efficient Estimation of Word Representations in Vector Space, 2013

• DeepWalk

  DeepWalk: Online Learning of Social Representations, 2014

• SkipGram model

  Efficient Estimation of Word Representations in Vector Space, 2013

• node2vec

  node2vec: Scalable Feature Learning for Networks, 2016

• LINE

  LINE: Large-scale Information Network Embedding, 2015

• TADW

  Network representation learning with rich text information, 2015

• drawbacks

  First, no parameters are shared between nodes in the encoder, which leads to computationally inefficiency, since it means the number of parameters grows linearly with the number of nodes.
  Second, the direct embedding methods lack the ability of generalization, which means they cannot deal with dynamic graphs or generalize to new graphs.

several comprehensive reviews on graph neural networks

Geometric deep learning: going beyond Euclidean data, 2017
Graph convolutional networks: a comprehensive review, 2019

• the most up-to-date survey papers on GNNs

  Deep Learning on Graphs: A Survey, 2018
  A Comprehensive Survey on Graph Neural Networks, 2019
  Machine Learning on Graphs: A Model and Comprehensive Taxonomy, 2020

several surveys focusing on some specific graph learning fields

• adversarial learning methods on graphs

  Adversarial Attack and Defense on Graph Data: A Survey, 2018
  A Survey of Adversarial Learning on Graphs, 2020

• a review over graph attention models

  Attention Models in Graphs: A Survey, 2018

• heterogeneous graph representation learning

  Heterogeneous Network Representation Learning: Survey, Benchmark, Evaluation, and Beyond

• existing GNN models for dynamic graphs

  Modeling Complex Spatial Patterns with Temporal Features via Heterogenous Graph Embedding Networks, 2020

• graph embeddings methods for combinatorial optimization.

  Graph Embedding for Combinatorial Optimization: A Survey

contributions

• We provide a detailed review over existing graph neural network models. We present a general design pipeline and discuss the variants of each module. We also introduce researches on theoretical and empirical analyses of GNN models.

• We systematically categorize the applications and divide the applications into structural scenarios and non-structural scenarios. We present several major applications and their corresponding methods for each scenario.

• We propose four open problems for future research. We provide a thorough analysis of each problem and propose future research directions.

General design pipeline of GNNs

Find graph structure

structural scenarios
non-structural scenarios

Specify graph type and scale

Directed/Undirected Graphs

Homogeneous/Heterogeneous Graphs

Static/Dynamic Graphs

Design loss function

Node-level
Edge-level
Graph-level

Supervised setting
Semi-supervised setting
Unsupervised setting

Build model using computational modules

Propagation Module
Sampling Module
Pooling Module

NDCN(Neural Dynamics on Complex Networks) combines ordinary differential equation systems (ODEs) and GNNs.

Instantiations of computational modules

Propagation modules - convolution operator

The emerging field of signal processing on graphs: Extending high-dimensional data analysis to networks and other irregular domains

• Spectral approaches $$\mathscr F(\mathbf x)=\mathbf U^T\mathbf x$$ $$\mathscr F^{-1}(\mathbf x)=\mathbf U\mathbf x$$

the normalized graph Laplacian

$$\mathbf L=\mathbf I_N-\mathbf D^{-\frac{1}{2}}\mathbf A\mathbf D^{-\frac{1}{2}}$$

$$\mathbf D\ is\ the\ degree\ matrix$$ $$\mathbf A is\ the\ adjacency\ matrix\ of\ the\ graph$$

A wavelet tour of signal processing

$$\begin{aligned} \mathbf g\star\mathbf x&=\mathscr F^{-1}(\mathscr F(\mathbf g)\odot\mathscr F(\mathbf x)\\ &=\mathbf U(\mathbf U^T\mathbf g\odot\mathbf U^T\mathbf x) \end{aligned}$$ $$\mathbf U^T\mathbf g\ is\ the\ filter\ in\ the\ spectral\ domain$$ $$\mathbf g\star\mathbf x=\mathbf U\mathbf g_w\mathbf U^T\mathbf x$$

several typical spectral methods which design different filters $\mathbf g_w$

Spectral Network

Spectral Networks and Locally Connected Networks on Graphs

Deep Convolutional Networks on Graph-Structured Data

ChebNet

Wavelets on Graphs via Spectral Graph Theory

Convolutional Neural Networks on Graphs with Fast Localized Spectral Filtering

Convolutional Neural Networks on Graphs with Fast Localized Spectral Filtering

GCN

• Basic spatial approaches

• Attention-based spatial approaches

• General frameworks for spatial approaches

Propagation modules - recurrent operator

Propagation modules - skip connection

Reference Paper
Graph Neural Networks: A Review of Methods and Applications, 2018

Review

• Graph Neural Networks: A Review of Methods and Applications, 2018

• Deep Learning on Graphs: A Survey, 2018

• Relationanl Inductive Biases, Deep Learning, and Graph Neural Networks, 2018

• Geometric Deep Learning: Going beyond Euclidean data, 2017

• Computational Capabilities of Graph Neural Networks, 2009

• Neural Message Passing for Quantum Chemistry, 2017

• Non-local Neural Networks, 2018

• The Graph Neural Network Model, 2009

Model

• A new model for learning in graph domains, 2005

• Graph Neural Networks for Ranking Web Pages, 2005

• Gated Graph Sequence Neural Networks, 2015

• Geometric deep learning on graphs and manifolds using mixture model CNNs, 2016

• Spectral Networks and locally Connected Networks on Graphs, 2013

• Deep Convolutional Networks on Graph-Structure Data, 2015

• Convolutional Neural Networks on Graphs with Fast Localized Spectral Filtering, 2016

• Learning Convolutional Neural Networks for Graphs, 2016

• Semi-Supervised Classification with Graph Convolutional Networks, 2016

• Graph Attention Networks, 2017

• Deep Sets, 2017

• Graph Partition Neural Networks for Semi-Supervised Classification, 2018

• Covariant Compositional Networks For Learning Graphs, 2018

• Modeling Relational Data with Graph Convolutional Networks, 2018

• Stochastic Training of Graph Convolutional Networks with Variance Reduction, 2018

• Learning Steady-States of Iterative Algorithms over Graphs, 2018

• Deriving Neural Architectures from Sequence and Graph Kernels, 2017

• Adaptive Graph Convolutional Neural Networks, 2018

• Graph-to-Sequence Learning using Gated Graph Neural Networks, 2018

• Deeper insights into Graph Convolutional Networks for Semi-Supervised Learning, 2018

• Graphical-Based Learning Environments for Pattern Recognition, 2004

• A Comparison between Recursive Neural Networks and Graph Neural Networks, 2006

• Graph Neural Networks for Object Localization, 2006

• Knowledge-Guided Recurrent Neural Network Learning for Task-Oriented Action Prediction, 2017

• Semantic Object Parsing with Graph LSTM, 2016

• CelebrityNet: A Social Network Constructed from Large-Scale Online Celebrity Images, 2015

• Inductive Representation Learning on Large Graphs, 2017

• Graph Classification using Structural Attention, 2018

• Adversarial Attacks on Neural Networks for Graph Data, 2018

• Large-Scale Learnable Graph Convolutional Networks, 2018

• Contextual Graph Markov Model: A Deep and Generative Approach to Graph Processing, 2018

• Diffusion-Convolutional Neural Networks, 2016

• Neural networks for relational learning: an experimental comparison, 2011

• FastGCN: Fast Learning with Graph Convolutional Networks via Importance Sampling, 2018

• Adaptive Sampling Towards Fast Graph Representation Learning, 2018

Application

• Discovering objects and their relations from entanglrd scene representations, 2017

• A simple neural network module for relational reasoning, 2017

• Attend, Infer, Repeat: Fast Scene Understanding with Generative Models

• Beyond Categories: The Visual Memex Model for Reasoning About Object Relationships

• Understanding Kin Relationships in a Photo

• Graph-Structured Representations for Visual Question Answering

• Spatial Temporal Graph Convolutional Networks for Skeleton-Based Action Recognition

• Few-Shot Learning with Graph Neural Networks

• The More You Know: Using Knowledge Graphs for Image Classification

• Zero-shot Recognition via Semantic Embeddings and Knowledge Graphs

• Rethinking Knowledge Graph Propagation for Zero-Shot Learning

• Interaction Networks for Learning about Objects, Relations and Physics

• A Compositional Object-Based Approach to Learning Physical Dynamics

• Visual Interaction Networks: Learning a Physics Simulator from Video

• Relational neural expectation maximization: Unsupervised discovery of objects and their interactions

• Graph networks as learnable physics engines for inference and control

• Learning Multiagent Communication with Backpropagation

• VAIN: Attentional Multi-agent Predictive Modeling

• Neural Relational Inference for Interacting Systems

• Translating Embeddings for Modeling Multi-relational Data

2D points

$$\mathbf x=(x,y)\in\textit R^2$$ $$\mathbf x=\begin{bmatrix} x \\ y \end{bmatrix}$$

homogeneous vector $\mathbf{\tilde x}$

$$\mathbf{\tilde x}=(\tilde x,\tilde y,\tilde w)\in\textit P^2$$

2D projective space

$$\textit P^2=\textit R^3-(0,0,0)$$

inhomogeneous vector $\mathbf{\overline x}$

$$\mathbf{\tilde x}=\tilde w(x,y,1)=\tilde w\mathbf{\overline x}$$

2D lines

homogeneous vector

$$\tilde{\mathbf{\textit l}}=(a,b,c)$$

line equation

$$\mathbf{\overline{x}}\cdot\tilde{\mathbf{\textit l}}=ax+by+c=0$$

$$\mathbf{\textit l}=(\hat n_x,\hat n_y,d)=(\hat{\mathbf n},d)with\Vert{\hat{\mathbf n}}\Vert=1$$

Reference Book
Computer Vision: Algorithms and Applications by Richard Szeliski

The conditions: E = 0

The equation of motion

$$m\frac{dv}{dt}=q\mathbf{v}\times\mathbf{B}$$

Cyclotron frequency

$$\omega_c\equiv\frac{\lvert q\rvert\mathbf{B}}{m}$$

$$v_x=v_\perp e^{i\omega_c t}$$ $$v_y=\pm iv_\perp e^{i\omega_c t}$$

combine

The Larmor radius

$$r_L\equiv\frac{v_\perp}{\omega_c}=\frac{mv_\perp}{\lvert q\rvert\mathbf{B}}$$

we get

$$x-x_0=r_Lsin\omega_c t$$ $$y-y_0=\pm r_L cos\omega_c t$$

• guiding center ($x_0, y_0$)
• plasmas are diamagnetic

The conditions: finite E

The equation of motion

$$m\frac{d\mathbf v}{dt}=q(\mathbf E+\mathbf v\times\mathbf B)$$

$$v_x=v_\perp e^{i\omega_c t}$$ $$v_y=\pm iv_\perp e^{i\omega_c t}-\frac{\mathbf E_x}{\mathbf B}$$

• The usual circular Larmor gyration
• A drift of the guiding center

• The three-dimensional orbit in space is therefore a slanted helix with changing pitch.

The conditions: gravitational field

$$\mathbf v_g=\frac{m}{q}\frac{\mathbf g\times\mathbf B}{\mathbf B^2}$$

• The magnitude of $v_g$ is usually negligible
• But an effective gravitational force due to centrifugal force is not negligible

Reference Book
Introduction to Plasma Physics and Controlled Fusion by Francis F. Chen

Our Universe is made of 69% dark energy, 27% dark matter, 1% normal matter.

What is plasma?

Plasma is also called the “fourth state of matter”. Solid is heated to become a liquid, liquid is heated to become a gas.
Upon further heating, the gas is ionized into a plasma. Since plasma usually exists only in a vacuum, we need to pump the air out of a vacuum chamber in the laboratory.

The Definition of Plasma

A plasma is a quasineutral gas of charged and neutral particles which exhibits collective behavior.

• The Coulomb force between A and B diminishes as $1/r^2$.
• However, for a given solid angle($\Delta$r/r = constant), the volume of plasma in B that can affect A increases as $r^3$.

The Saha Equation

$$\frac{n_i}{n_n}\approx2.4\times10^{21}\frac{T^{3/2}}{n_i}e^{-U_i/KT}$$ $$n_i\ is\ the\ density\ of\ ionized\ atoms$$ $$n_n\ is\ the\ density\ of\ neutral\ atoms$$ $$K\ is\ Boltzmann's\ constant$$ $$U_i\ is\ the\ ionization\ energy\ of\ the\ gas$$

Physical meaning

• When temperature is raising, the whole value is increasing exponentially with $U_i/KT$.

• The higher value of $n_i$, the lower recombination rate of ionized atoms.

The Maxwellian Distribution

The one-dimensional Maxwellian distribution

$$f(u)=Aexp(-\frac{1}{2}mu^2/KT)$$ $$fdu\ is\ the\ number\ of\ particles\ per\ m^3\ with\ velocity\ between\ u\ and\ u+du$$ $$\frac{1}{2}mu^2\ is\ the\ kinetic\ energy$$

Boltzmann’s constant K

$$K=1.38\times10^{-23}J/^{\circ}K$$

The particles density n

$$n=\int_{-\infty}^{\infty}f(u)du$$

The constant A is related to the density n

$$A=n(\frac{m}{2\pi KT})^{1/2}$$

The average kinetic energy is $\frac{1}{2}KT$

The three-dimensional Maxwellian distribution

$$f(u,v,w)=A_3exp[-\frac{1}{2}m(u^2+v^2+w^2)/KT]$$

Reference Book
Introduction to Plasma Physics and Controlled Fusion by Francis F. Chen

Anaconda Prompt Operation

Create new environment

conda create - -name environment pakage

conda create - -name python3 python=3.8

Activate environment

conda activate environment

Exit environment

deactivate environment

Delete environment

conda remove -n environment - -all

Copy environment

conda create -n environment - -clone existing_environment

View environment information

conda info -e
conda env list
conda info - -envs

View python version

python -V

Install package

conda install package

View package information

conda search package

Install package in a specify environment

conda install -n environment package

Update package in a specify environment

conda update -n environment package

Delete package in a specify environment

conda remove -n environment package

View the installed packages in the current environment

conda list

View the installed packages in a specify environment

conda list -n environment