Author(s): Omer Ozgur Deep Learning From https://www.pinterest.ru/pin/801781539891918164/?amp_client_id=CLIENT_ID(_)&mweb_unauth_id=&simplified=true Scientist vs Machine Learning “I am turned into a sort of machine for observing facts and grinding out conclusions.” — Charles Darwin Science cannot work without data. Machine learning is the same. Machine learning and scientific methods are closely related. Scientists identify problems and gather data to understand the universe. Scientists create models from the data collected and improve this process every time. Machine learning will also try to make the most of the information it gathers. This is a model that you can hold in your hands. Human memory and processing power are limited. Artificial intelligence is a revolutionary new way to conduct scientific inquiry. It allows us to automate our research processes. AI is destined to change the face of scientific research. There are still some issues that need to be solved first. Maxwell’s equations are considered science facts, while deep learning models remain a mystery. Interpretability and generalizability play an important part in this situation. We will learn how to compress high-dimensional data into analytic equations later in this article. The Golden Rules Simpler is better than complicated -Einstein Generalizability and Interpretability are key principles for mathematical equations. F=m*a is a great example because it can be easily understood and applied to many situations. It can be used to predict the movement of the moon and apple. Many of the laws in natural science can be described using simple symbolic equations. Examples include work, energy, motion and radiation as well as fluid dynamics and thermodynamics. We are encouraged to use simple equations in solving complex math problems because of our unreasonable mathematical effectiveness. It won’t help to train deep learning models to predict when apples will fall. However, deep learning may not be the answer to all problems. While deep learning is extremely efficient in learning high-dimensional space, it suffers from poor generalizations and interpretability. Symbolic regression, on the other hand is extremely good at generalization but not very good with high-dimensional data. Is there a way that both can be combined? Learning Mechanics From https://astroautomata.com/paper/symbolic-neural-nets/ The Neural Network’s role in our approach is to anticipate targets and split them down into tiny internal functions that operate in low-dimensional spaces. To approximate the function in the deep model, symbolic regression uses an analytic equation. GNNs have been shown to be successful at learning how to solve physics problems. GNNs’ message function can be compared to a force. Newton’s law is comparable to the node-update function. To make the GNN sparse, we reduce each function’s dimensionality. This will allow symbolic regression to better extract the expression. Latent Space To Formula From https://astroautomata.com/paper/symbolic-neural-nets/ Symbolic regression is a kind of regression analysis that looks for the best model in a space of mathematical expressions. To create the initial expression, mathematical building elements like analytic functions constants and variables, are randomized combined. These variables can then be improved using an evolutionary approach. A mutation could replace the operator + with /, or the variable x2 by x3. Model evolution is driven by a fitness function. It takes into account error metrics as well as specific complexity measures. This ensures that final models can explain data’s structure in a way that humans understand. Symbolic regression avoids imposing any previous assumptions. It infers the model directly from data. Traditional regression methods aim to maximize parameters that are specific for a model structure. The high-dimensional search area cannot be optimized by genetic algorithms. To solve the problem, we transform deep neural networks into simplified mathematical equations. Although symbolic Regression is able to discover the laws of Physics, it does not learn from data. Instead, it uses what neural networks have learned. Discovering Unknown From https://astroautomata.com/paper/symbolic-neural-nets/ If we can rediscover dynamics that we know using ML, we can discover new dynamical systems that we do not know. Cosmology studies the evolution of the Universe from its Big Bang through to complex structures such as stars and galaxies. The interactions between different material and energy drive this evolution, even though dark matter accounts for 85% all the matter in the Universe [1].. Dark matter particles are clustered together and form dark matter halos. These serve as gravitational sinks, which draw matter together to form stars or larger structures such as galaxies. Cosmology is dependent on the ability to discern dark matter halos’ characteristics from their environment. The same GNN model that predicted a halo’s excess density was used to predict it. Scientists have not been able to produce an analytical equation comparable to the algorithm that was developed. PySR (Parallelized symbol regression) is a library that was built on Julia and interfaced with Python. Regularized evolution, simulation annealing and gradient-free optimizing are used. SINDY, a sparse analysis package that allows for multiple implementations of the Sparse Identification of Nonlinear Dynamical Systems (SINDY), is available. The key takeaways Machine learning and human intelligence will bring new perspectives to scientific research. No technique is perfect. Each technique has its strengths and flaws. It is essential to combine methods properly in order to advance scientific research. References: [1] https://astroautomata.com/paper/symbolic-neural-nets/ [2]https://en.wikipedia.org/wiki/Symbolic_regression Thanks to Steve Brunton’s Youtube Channel How To Discover The Laws Of Physics With Deep Learning and Symbolic Regression was originally published in on Medium, where people are continuing the conversation by highlighting and responding to this story. Published via
