Neural Operators vs. Polynomial Surrogates: The Battle for Data Efficiency

the computationally heavy task of evaluating parameter-to-solution maps in parametric partial differential equations, the choice of surrogate models can dramatically impact efficiency and accuracy. to the head-to-head between neural operator surrogates and polynomial surrogate methods.

Neural Operators: The Contenders

Neural operators, including the reduced-basis neural operator and the Fourier neural operator, are making waves. They're evaluated on both linear parametric diffusion and nonlinear parametric hyperelasticity problems. The reality is, in scenarios with rough input fields, where spectral coefficients decay slowly, these neural operators shine. The Fourier neural operator, in particular, showcases the fastest convergence rates when the input fields are rough.

Polynomial Surrogates: Keeping It Smooth

But don't count out polynomial surrogates just yet. For smoother input fields, where the decay rate is higher, polynomial surrogates like the reduced-basis sparse-grid method show superior data efficiency. They're aligning well with theoretical predictions, proving their worth in specific contexts.

Strip away the marketing and you get a clear picture: each method has its strengths and weaknesses. The numbers tell a different story depending on the smoothness of the inputs. For smooth fields, polynomial surrogates lead the pack. For rough fields, neural operators take the crown.

Training Techniques: A Game Changer?

Interestingly, incorporating derivative-informed training into the mix boosts data efficiency, especially for rough inputs in the low-data regime. When you've access to Jacobian information at a reasonable cost, this training technique is a competitive alternative to traditional $L^2_\mu$ training.

Here's what the benchmarks actually show: no single method reigns supreme across all scenarios. It's about matching the surrogate methodology to the problem's regularity, accuracy demands, and computational constraints. So, why should this matter to you? Because choosing the wrong surrogate model can mean wasted resources and missed opportunities for improved performance.

In the end, the architecture matters more than the parameter count. It's the nuances in model selection and training that determine success in computational efficiency and accuracy. Whether neural or polynomial, the choice should be informed by the specifics of your application. Anything less is just running in circles.