Bayesian updates on infinitesimal evidence, and stochastic calculus

https://math.stackexchange.com/questions/962401/bayes-theorem-with-infinitesimal-evidence

You seem to be describing a situation where you have a variable X, and a stream of information Y_t for t ∈ ℝ such that the conditional mutual information I(X;Y_t∣Y_[0, t)) = 0 (or rather infinitesimal). For any particular Y_t you still apply Bayes's theorem, but if you already know Y_[0, t) then Y_t gives you no new information about X.

One natural example is "resolution": Y_t are versions of the image at increasing resolution while X is the true image. A toy model of this: X ∼ N(0,1) and Y_t = X + R(1−t) for t < 1 where R(⋅) denotes a random walk wiener process of given length.

We might write:

$$ P(X\mid Y_t)= P(X\mid Y_{t-\epsilon})\frac{ P(Y_t\mid X, Y_{t-\epsilon}) }{ \sum_x P(Y_t\mid X=x, Y_{t-\epsilon})P(X\mid Y_{t-\epsilon}) } $$

The conditional independence assumption implies the numerator is simply P(Y_t∣Y_t − ϵ) which is equal to the denominator; thus the Bayes factor is 1. But maybe P(Y_t∣X,Y_t − ϵ) "infinitesimally" depends on the value of X: the true image affects the next infinitesimal step of resolution in infinitesimal ways.

<!—For instance, Y_t could be the question: is X ≤ t? Then given X ≤ t − ϵ, we are almost certain X is not  ≤ t, unless X is exactly t. So in that case we'd have P(Y_t∣X,Y_t − ϵ=0) equal 1 if X = t (which has infinitesimal probability), and 0 otherwise; and P(Y_t∣X,Y_t − ϵ=1) equal 1 regardless of X.—>

In our toy example, Y_t is slightly more likely to be an infinitesimal step from Y_t − ϵ in the direction closer to X, than an infinitesimal step in the direction away from it. We can work out that P(Y_t∣Y_t − ϵ,X) ∼ N((1−ϵ)Y_t − ϵ+ϵX, ϵ).

I asked Claude to finish the Bayesian inference, IDK if it is right (simulations agree with it but it seems to have fudged it):

https://claude.ai/share/0cec993b-e141-441a-8bc6-688343ff170c