Generative Models and the Myth of Hard Constraints

Generative models, employing diffusion and flow matching techniques, have emerged as popular tools for tackling partial differential equation (PDE) inverse problems. These models enforce the physics of the problem as a so-called hard constraint, projecting or guiding to produce samples purportedly representing a Bayesian posterior distribution with calibrated uncertainty.

Color me skeptical, but recent research shows that this approach may be fundamentally flawed. It seems these methods are sampling the wrong distribution altogether. Conditioning a generative model on a PDE constraint isn't as straightforward as it appears. In reality, it's akin to conditioning on a measure-zero manifold, a concept invoking the Borel-Kolmogorov paradox.

The Overlooked Factor

Here's what they're not telling you: the correct sampling should consider a co-area (Fixman) Jacobian factor, denoted as[det(JJ^{\top})]^{-1/2}. This factor, surprisingly, is absent in many current methodologies that rely on projection or guidance, leading to significant biases. The oversight isn't just a minor detail. When omitted, this factor inflates the posterior error by a staggering factor of 20, relative to the sampling-noise baseline. Even techniques like minimal-displacement projection, as seen in PCFM, exhibit biases that reach nine times the baseline.

Introducing CoCoS

Enter CoCoS, a measure-aware constrained sampler designed to address this exact issue. By incorporating the necessary co-area factor, CoCoS aligns much more closely with the true gold-standard posterior, remaining within the bounds of sampling noise. This development underscores a essential distinction: satisfying the physics doesn't equate to accurately sampling the posterior. Without the proper corrections, scientific inferences can be grossly misleading.

So, why should you care about these technical details? If you're relying on these models for scientific predictions or engineering applications, understanding and correcting these biases is imperative. The potential errors aren't just academic. they've real-world implications that could skew results and misinform decisions.

A Wake-Up Call for the Community

I've seen this pattern before: the allure of a seemingly elegant solution overshadowing the need for rigorous validation. The generative model community must take heed and reassess their methodologies. Are we truly advancing towards uncertainty-aware scientific inference, or are we simply falling into another trap of overfitting and cherry-picked results?

The challenge now is clear. Embrace CoCoS or a similar approach to ensure that the physics we so diligently enforce actually leads to the insights we claim to derive. Anything less would be a disservice to the scientific community.