It has long been suggested that the biological brain operates at some critical point between two different phases, possibly order and chaos, to maximize the information processing power. Investigating the same hypothesis on the ‘artificial’ brains, i.e. the modern computer vision models, we find that they exhibits the same pattern, i.e. highest test accuracy or lowest test loss at the edge of chaos. A theoretical investigation demonstrates that, the best performance is attributed to the maximal metastable states/periodic cycle length near the edge of chaos, where each metastable state can represent an information point. Applied on a very simple network equivalent of the SK spin glass model and Fashion MNIST dataset, we illustrate a simple and principled training method that can achieve both high accuracy and prevent fitting noisy labels automatically.

Papers:

Dr. Feng Ling is a principle scientist at the Institute of High Performance Computing, A*STAR Singapore, leading the Complex Systems group. He is also adjunct assistant professor at Department of Physics at National University of Singapore. He obtained PhD degree from National University of Singapore in 2013, and in 2011-2013 he was a visiting scholar to Boston University in the group lead by H. Eugene Stanley. His research interest is on complexity science and its various applications in social and urban systems, and more recently in the complexity science of AI models.

For more information about the ESD Seminar, please email esd_invite@sutd.edu.sg