Bayesian updates on infinitesimal evidence, and stochastic calculus

https://math.stackexchange.com/questions/962401/bayes-theorem-with-infinitesimal-evidence

You seem to be describing a situation where you have a variable \(X\), and a stream of information \(Y_t\) for \(t\in\mathbb{R}\) such that the conditional mutual information \(I(X;Y_t\mid Y_{[0,t)})=0\) (or rather infinitesimal). For any particular \(Y_t\) you still apply Bayes’s theorem, but if you already know \(Y_{[0,t)}\) then \(Y_t\) gives you no new information about \(X\).

One natural example is “resolution”: \(Y_t\) are versions of the image at increasing resolution while \(X\) is the true image. A toy model of this: \(X\sim N(0,1)\) and \(Y_t=X+R(1-t)\) for \(t<1\) where \(R(\cdot)\) denotes a random walk wiener process of given length.

We might write:

\[ P(X\mid Y_t)= P(X\mid Y_{t-\epsilon})\frac{ P(Y_t\mid X, Y_{t-\epsilon}) }{ \sum_x P(Y_t\mid X=x, Y_{t-\epsilon})P(X\mid Y_{t-\epsilon}) } \]

The conditional independence assumption implies the numerator is simply \(P(Y_t\mid Y_{t-\epsilon})\) which is equal to the denominator; thus the Bayes factor is 1. But maybe \(P(Y_t\mid X,Y_{t-\epsilon})\) “infinitesimally” depends on the value of \(X\): the true image affects the next infinitesimal step of resolution in infinitesimal ways.

<!—For instance, \(Y_t\) could be the question: is \(X\le t\)? Then given \(X\le t-\epsilon\), we are almost certain \(X\) is not \(\le t\), unless \(X\) is exactly \(t\). So in that case we’d have \(P(Y_t\mid X,Y_{t-\epsilon}=0)\) equal 1 if \(X=t\) (which has infinitesimal probability), and 0 otherwise; and \(P(Y_t\mid X,Y_{t-\epsilon}=1)\) equal 1 regardless of \(X\).—>

In our toy example, \(Y_t\) is slightly more likely to be an infinitesimal step from \(Y_{t-\epsilon}\) in the direction closer to \(X\), than an infinitesimal step in the direction away from it. We can work out that \(P(Y_t\mid Y_{t-\epsilon}, X) \sim \mathbf{N}\left((1-\epsilon)Y_{t-\epsilon}+\epsilon X,\ \epsilon\right)\).

I asked Claude to finish the Bayesian inference, IDK if it is right (simulations agree with it but it seems to have fudged it):

https://claude.ai/share/0cec993b-e141-441a-8bc6-688343ff170c