Massive data collection and computational capabilities have enabled data-driven scientific discoveries and control of engineering systems. However, there are still several questions that should be answered to understand the fundamental limits of just how much can be discovered with data and what is the value of additional information. I’ll answer this question in two learning tasks. A key insight in both two cases is that using direct plug-in estimators can result in statistically suboptimal inference.

For the first learning task, I focused on variational formulations for differential equation models ( a prototypical Poisson equation). I provide a minimax lower bound for this problem. Based on the lower bounds, I discovered that the variance in the direct plug-in estimator makes sample complexity suboptimal. I also considered the optimization dynamic for different variational forms. Finally, based on our theory, I explain an implicit acceleration of using a Sobolev norm as the objective function for training.

The second learning task discussed is (linear) operator learning, which has wide applications in time series modeling, and conditional probability learning. I built the first min-max lower bound for this problem. The min-max rate has a particular structure where the more challenging parts of the input and output spaces determine the hardness of learning a linear operator. The analysis also shows that an intuitive discretization of the infinite-dimensional operator could lead to a sub-optimal statistical learning rate. Then, I’ll discuss how, by suitably trading off bias and variance, I can construct an estimator with an optimal learning rate for learning a linear operator between infinite dimension spaces. I also illustrate how this theory can inspire a multilevel machine-learning algorithm of potential practical use.

9.22.pdf