Machine Learning for Advanced Materials
|Using a combination of theoretical tools (multi-scale modelling), our group is exploring and designing advanced materials. Our goal is to make materials that are, in the words of Mervin Kelly, "better, cheaper, or both".
The long term objective of my research is to teach a computer physical and chemical intuition. A trained artificial intelligence (AI) should be able to make informed decisions about how to solve problems in chemistry, physics, and nanoscience without human intervention. We are approaching an era where machines will learn chemical and physical design rules based on a combination of theoretical models and experience, both simulated and real. Simulation and modelling are used throughout chemistry and physics research - the central question of my research program is how we can enhance these tools using modern AI and deep learning.
So far, we have shown that deep neural networks have the ability to replace both classical [J33, S1] and quantum mechanical operators [J32, J34, J36, S2, S3] (see my CV for full ref). In comparison to other machine learning methods, we demonstrated that convolutional deep neural networks prevailed as the most accurate and best parallel-scaling method for all but the most simple physical problems [J30]. Specifically, we demonstrated the ability of deep neural networks to learn both the mapping from spin-configuration-to-energy (for the case of the ferromagnetic Ising model and screened Coulomb interaction) as well as magnetization for a classical system. We are able to reproduce the temperature-induced order-to-disorder phase transition the Ising model is famous for (at the correct critical temperature, Tc) [J33]. We also showed that generative adversarial networks can be used to efficiently explore phase space (for the case of the spin models), and can be used to provide a confidence level for property prediction [S1]. For a confined quantum particle, our deep neural networks successfully learned the energy of the ground state, first excited state, and kinetic energy [J32]. We then used a similar approach to map the structure of a two dimensional hexagonal crystal lattice to energies computed within the density functional theory [J34, S2], where we trained the system to learn the mapping between a pseudopotential and total energy. Extending this to 3d [S2], we were able to scan through over a billion compositions of a potential light harvesting material (perovskites). All of this was accomplished via “featureless” deep learning, meaning we presented the network with raw, spatial data, without any form of manual feature selection. Overcoming a scaling limitation with traditional deep neural networks, we developed a new architecture and training protocol which naturally enforces extensivity onto the system [J36]. Our new extensive deep neural networks (EDNN) are able to naturally learn the locality length-scales of operators and look extremely promising for providing efficient alternative electronic structure methods to density functional theory [J36], a widely used theoretical and computational framework used in physics, chemistry, and nanoscience. This architecture is an example of how physical insight can be used to improve neural network topology and design - subsequently we showed that it can be used with weakly-supervised data to learn how to localize and count objects within a scene [S5]. I believe that the “inserting physics” into the design of deep networks is a particularly rich area which needs to be further explored. We have also generated and released several open source datasets and reinforcement learning environments for the community (http://clean.energyscience.ca/datasets). These are aimed at exposing these important problems to the wider AI community.
Most recently, our efforts have been focused on understanding and applying techniques from reinforcement learning to physics and chemistry. These include heat engines [S4], chemical reactions, and non-equilibrium self-assembly of nanostructure materials (topics I have previously published on, see CV).
To do this, we are developing deep learning methods which can rapidly approximate the properties of materials based on first principles training data. Ultimately, our goal is to develop a model which is transferrable, generalizable, reliable, and scalable.
Transferability means that the network is able to accurately predict properties for structures which it has not observed during training.
Generalizability means that the approach can be used for arbitrary observables, and that it is not sensitive to the details of the physical system to which it applies. For example, we would like to be able to work with fluids, continuum models, and atomistic systems using the same basic methodologies, model architectures, and training protocols.
By reliable, we mean that the model is able to provide accurate estimates for a property, and, perhaps more importantly, provide the user with a warning or signal when it is uncertain. In our experience, neural networks are much better at interpolating than extrapolating. We have found that whenever a model has difficulty with a particular configuration (i.e. the error is high), the configuration is far away from the training set (in weight space). In practice, one obviously would not have access to the “ground truth” value, therefore a reliable model should be able to signal not only its prediction, but also a measure of confidence.
The final property, scalability, relates to the extent to which the computational cost of evaluating the model can be distributed (e.g. across nodes in a cluster or multiple GPU).
Our latest approaches are generalizable, scalable, and transferrable. Their reliability can be ensured through a GAN-like training procedure
Ab initio electrolysis
Given the ubiquity of electrochemistry as an analytical and industrial tool, the importance of developing a fully first principles description of this system cannot be overstated.
At the same time, the complexity of the problem poses a significant obstacle to the development of an accurate atomistic description.
Obtaining the correct physical picture will require the union of several areas of theory.
A complete treatment will require an accurate description of the electrode-solution interface, both in terms of local geometry and electronic structure.
Methodological development will be important in this field.
Energy harvesting materials
Light from the sun provides an unlimited supply of energy. Our challenge is to harness this energy in an efficient and scalable manner. Using metal-to-metal charge transfer complexes as light absorbers, an integrated, inorganic device shows promise for applications in water splitting and CO2 sequestration.
Through computational modelling, we aim to elucidate electronic processes which occur on the nanoscale, with the ultimate goal of improving efficiency and durability.
Networks and self assembly
Organic-metallic interfaces offer the possibility of novel next-generation photovoltaic devices.
Producing these devices reliably and cheaply poses many challenges, however.
Understanding and controlling self-assembly processes is an important aspect of the realization of this technology. Interestingly, this work has recently spun off in a very different direction: Online social Networks. See our open source tool, #k@ (http://hashkat.org), for more information.