The size of data has become enormous. One needs significant mathematical tools to process and abstract information from big data collections. We are living in an information revolution in which processing larger and larger data sets will become common. As the size of data sets increases, more subtle information can be extracted. This talk will illustrate the mathematical background needed to be successful in the information age.
AI is fundamentally transforming every aspect of human life on an unimaginable scale, revolutionizing the making of goods to generating unprecedented wealth or creating brand new forms of interactions. Join internationally renowned AI expert Dr. Kai-Fu Lee, bestselling author of AI Superpowers, as he introduces new predictions of the next five major technology trends of our century. In this illuminating talk, Dr. Lee will guide us through most current breakthroughs in the fields of artificial intelligence, automation & robotics, life sciences, new energy, quantum computing and the cross pollination possibilities across these disciplines. Based on his predictions, AI and automation will change everything from how things are produced to how business decisions are made, leading up to the “age of plentitude." AI coupled with other technology breakthroughs will benefit human well-being and longevity, accelerate new sources of clean energy and safer food. Dr. Lee will also decipher the rise of China under the global technological paradigms on the trajectory to become a deep tech superpower.
This talk presents recent theoretical, algorithmic, and methodological advances for black-box optimization problems for which optimization must be performed in the absence of an algebraic formulation, i.e., by utilizing only data originating from simulations or experiments. We investigate the relative merits of optimizing surrogate models based on generalized linear models and deep learning. Additionally, we present new optimization algorithms for direct data-driven optimization. Our approach combines model-based search with a dynamic domain partition strategy that guarantees convergence to a global optimum. Equipped with a clustering algorithm for balancing global and local search, the proposed approach outperforms existing derivative-free optimization algorithms on a large collection of problems.
Federated learning is an important intersection of AI, big data and privacy computing. How to make federated learning safe, trustworthy and efficient at the same time is the focus of future industry and research. In my lecture, I will systematically review the progress and challenges of AI and introduce federated learning as a secure distributed approach to AI modeling. I will then discuss several important research and application directions.
In this talk we introduce a principled computational framework for learning a compact structured representation for real-world datasets. More specifically, we propose to learn a closed-loop transcription between the distribution of a high-dimensional multi-class dataset and an arrangement of multiple independent subspaces, known as a linear discriminative representation (LDR). We argue that the encoding and decoding mappings of the transcription naturally form a closed-loop sensing and control system. The optimality of the closed-loop transcription, in terms of parsimony and self-consistency, can be characterized in closed-form by an information-theoretic measure known as the rate reduction. The optimal encoder and decoder can be naturally sought through a two-player minimax game over this principled measure. To a large extent, this new framework unifies concepts and benefits of auto-encoding and GAN and generalizes them to the settings of learning a both discriminative and generative representation for multi-class visual data. This work opens many new mathematical problems regarding learning linearized representations for nonlinear submanifolds in high-dimensional spaces, as well as suggests potential computational mechanisms about how visual memory of multiple object classes could be formed jointly or incrementally through a purely internal closed-loop feedback process.
Related papers can be found at: https://arxiv.org/abs/2111.06636, https://arxiv.org/abs/2105.10446, and https://arxiv.org/abs/2202.05411.
Big data contains a tremendous amount of dark knowledge. The community has realized that effectively exploring and using such knowledge is essential to achieving superior intelligence. How can we effectively distill the dark knowledge from ultra-large-scale data? One possible answer is: “through Transformers”. Transformers have proven their prowess at extracting and harnessing the dark knowledge from data. This is because more is truly different when it comes to Transformers. In this talk, I will showcase our recent work on transformers named ViTAE, on many dimensions of “more” including: model parameters, labeled and unlabeled data, prior knowledge, computing resource, tasks, and modalities. Specifically, ViTAE has more model parameters and more input modality support; ViTAE can absorb and encode more data to extract more dark knowledge; ViTAE is able to adopt more prior knowledge in the form of biases and constraints; ViTAE can be easily adapted to larger-scale parallel computing resources to achieve faster training.
ViTAE has been applied to many computer vision tasks and has proven its promise, such as image recognition, object detection, semantic segmentation, image matting, pose estimation, scene text understanding, and remote sensing.
You can find the source code for this work at here.
Federated learning has become a popular distributed machine learning paradigm for developing on-device AI applications. However, the data residing across devices is intrinsically statistically heterogeneous (i.e., non-IID data distribution) and mobile devices usually have limited communication bandwidth to transfer local updates. Such statistical heterogeneity and communication limitation are two major bottlenecks that hinder applying federated learning in practice. In addition, recent works have demonstrated that sharing model updates makes federated learning vulnerable to inference attacks and model poisoning attacks. In this talk, we will present our recent works on novel federated learning frameworks to address the scalability and heterogeneity issues simultaneously. In addition, we will also reveal the essential reason the of adversarial vulnerability of deep learning models and the privacy leakage in federated learning procedures, and provide the defense mechanisms accordingly towards trustworthy federated learning.
The introduction of machine learning (ML) creates unprecedented opportunities for achieving full autonomy. However, learning-based methods in building autonomous systems can be extremely brittle in practice and are not designed to be verifiable. In this talk, I will present several of our recent efforts that combine ML with formal methods and control theory to enable the design of provably dependable and safe autonomous systems. I will introduce our techniques to generate safety certificates and certified decision and control for complex large-scale multi-agent autonomous systems, even when the agents follow nonlinear and nonholonomic dynamics and need to satisfy high-level specifications.
As a flexible nonparametric learning tool, random forests algorithm has been widely applied to various real applications with appealing empirical performance, even in the presence of high-dimensional feature space. Unveiling the underlying mechanisms has led to some important recent theoretical results on the consistency of the random forests algorithm and its variants. However, to our knowledge, all existing works concerning random forests consistency in high dimensional setting were established for various modified random forests models where the splitting rules are independent of the response. In light of this, in this paper we derive the consistency rates for the random forests algorithm associated with the sample CART splitting criterion, which is the one used in the original version of the algorithm (Breiman2001), in a general high-dimensional nonparametric regression setting through a bias-variance decomposition analysis. Our new theoretical results show that random forests can indeed adapt to high dimensionality and allow for discontinuous regression function. Our bias analysis characterizes explicitly how the random forests bias depends on the sample size, tree height, and column subsampling parameter. Some limitations on our current results are also discussed.
This talk introduces OMLT (https://github.com/cog-imperial/OMLT), an open source software package incorporating surrogate models, which have been trained using machine learning, into larger optimisation problems. Computer science applications include maximizing a neural acquisition function and verifying neural networks. Engineering applications include the use of machine learning models to replace complicated constraints in larger design/operations problems. OMLT 1.0 supports GBTs through an ONNX (https://github.com/onnx/onnx) interface and NNs through both ONNX and Keras interfaces. We discuss the advances in optimisation technology that made OMLT possible and show how OMLT seamlessly integrates with the python-based algebraic modeling language Pyomo (http://www.pyomo.org). The literature often presents different optimization formulations as competitors, but in OMLT, competing formulations become alternatives: users can select the best for a specific application. We provide examples including neural network verification, autothermal reformer optimization, and Bayesian optimization.
This work is joint with the Imperial Computational Optimisation Group (Francesco Ceccon, Ruth Misener, Alexander Thebelt, Calvin Tsay), Sandia National Laboratories (Jordan Jalving, Joshua Haddad), and Carnegie Mellon (Carl Laird).
Cyber-physical systems (CPSs) are the mechanisms controlled or monitored by computer-based algorithms, tightly integrated with the internet and its users. CPSs are the central research topic in the era of Industrial 4.0, and continue to be in the forthcoming Industrial 5.0, which have attracted a lot attention in the past years. Undergoing an ever-enriching cognitive process, CPSs deeply integrates control, communication, computation, cloud and cognition. In this talk, we firstly review some basic knowledge with respect to the concepts, history, and some viewpoints on CPS security. Next, some commonly appeared malicious threats will be introduced.
Decentralized optimization techniques offer high quality solutions to various engineering problems, such as resource allocation and distributed estimation and control. Advantages of decentralized optimization over its centralized counterpart lie in that it can provide a flexible and robust solution framework where only locally light computations and peer-to-peer communication are required to minimize a global objective function. In this work, we report the decentralized convex constrained optimization problems in networks. A novel decentralized dual averaging (DDA) algorithm is proposed. In the algorithm, a second-order dynamic average consensus protocol is tailored for DDA-type algorithms, which equips each agent with a provably more accurate estimate of the global dual variable than conventional schemes. Such accurate estimate validates the use of a large constant parameter within the local inexact dual averaging step performed by individual agents. Compared to existing DDA methods, the rate of convergence is improved to $\mathcal{O}({1}/{t})$ where $t$ is the time counter. Finally, numerical results are presented to demonstrate the efficiency of the proposed methods.
It is known that the processes of learning and transfer of what has been learned are important to humans for solving complex problems. However, the study on optimization methodologies via learning from existing solutions and the transfer of what has been learned to help on solving related or unseen problems, has been under-explored in the context of evolutionary computation. This talk will give an overview of evolutionary transfer optimization (ETO), which is an emerging research direction that integrates evolutionary algorithm solvers with knowledge learning and transfer across different problem domains to achieve better optimization efficiency and performance. It will present some recent research work in ETO for solving multi-objective and large-scale optimization problems via high-performance computing. Some discussions on future ETO research directions, including topics such as theoretical analysis and real-world applications, will also be given.
The past four decades witnessed the birth and growth of neurodynamic optimization, which has emerged as a potentially powerful problem-solving tool for constrained optimization due to its inherent nature of biological plausibility and parallel and distributed information processing. Despite the success, almost all existing neurodynamic approaches a few years ago worked well only for optimization problems with convex or generalized convex functions. Effective neurodynamic approaches to optimization problems with nonconvex functions and discrete variables are rarely available. In this talk, a collaborative neurodynamic optimization framework will be presented. Multiple neurodynamic optimization models with different initial states are employed in the framework for scatter local search. In addition, a meta-heuristic rule in swarm intelligence (such as PSO) is used to reposition neuronal states upon their local convergence to escape local minima toward global optima. Experimental results will be elaborated to substantiate the efficacy of several specific paradigms in this framework for nonnegative matrix factorization, supervised learning, vehicle-task assignment, portfolio selection, and energy load dispatching.
Quantum computing is attracting growing interest due to its unique capabilities and disruptive potential. This presentation will briefly introduce quantum computing and its potential applications to systems optimization and machine learning. We will introduce several new algorithms and methods that exploit the strengths of quantum computing techniques to address the computational challenges of classically intractable optimization problems. Applications include molecular design, manufacturing systems operations, and supply chain optimization. In the second half of the presentation, we will focus on quantum machine learning and the emerging hybrid classical-quantum computing paradigm that exploit the strengths of quantum computing techniques to address the computational challenges of important AI-related problems. The presentation will conclude with a novel deep learning model and quantum computing algorithm for efficient and effective fault diagnosis in manufacturing and electric power systems.
Many optimization problems in the real world, by nature, have multiple conflicting objectives. Unlike a single optimization problem, multiobjective optimization problem has a set of Pareto optimal solutions (Pareto front) which are often required by a decision maker. Evolutionary algorithms are able to generate an approximation to the Pareto front in a single run, and many traditional optimization methods have been also developed for dealing with multiple objectives. Combination of evolutionary algorithms and traditional optimization methods should be a next generation multiobjective optimization solver. Decomposition techniques have been well used and studied in traditional multiobjective optimization. Over the last decade, a lot of effort has been devoted to build efficient multiobjective evolutionary algorithms based on decomposition (MOEA/D). In this talk, I will describe main ideas and techniques and some recent development in MOEA/D. I will also discuss some possible research issues in multiobjective evolutionary computation.
In this talk, I will introduce an end-to-end deep learning framework based on a protein–protein interaction (PPI) network to make synergistic anticancer drug combination predictions. The framework, namely GraphSynergy, adapts a spatial-based Graph Convolutional Network component to encode the high-order topological relationships in the PPI network of protein modules targeted by a pair of drugs, as well as the protein modules associated with a specific cancer cell line. The pharmacological effects of drug combinations are explicitly evaluated by their therapy and toxicity scores. An attention component is also introduced in GraphSynergy, which aims to capture the pivotal proteins that play a part in both PPI network and biomolecular interactions between drug combinations and cancer cell lines. GraphSynergy outperforms the classic and state-of-the-art models in predicting synergistic drug combinations on the 2 latest drug combination datasets. Specifically, GraphSynergy achieves accuracy values of 0.7553 (11.94% improvement compared to DeepSynergy, the latest published drug combination prediction algorithm) and 0.7557 (10.95% improvement compared to DeepSynergy) on DrugCombDB and Oncology-Screen datasets, respectively. Furthermore, the proteins allocated with high contribution weights during the training of GraphSynergy are proved to play a role in view of molecular functions and biological processes, such as transcription and transcription regulation. This research indicates that introducing topological relations between drug combination and cell line within the PPI network can significantly improve the capability of synergistic drug combination identification.