market microstructure - How do you actually solve a stochastic HJB equation in practice?

I've read a number of recent papers on market making. Nearly all of the more recent papers focus on defining the problem in terms of a state and action space, deriving the relevant HJB equations and constraints, and going from there.

I realize this could be viewed as a broad question, but I think that it's worthy from the perspective of a programmer. In practice, what are the general methods to attack the problem of solving an optimal policy in real time? I understand the theoretical underpinnings of the DPP, but there are few libraries or frameworks designed to help with this, and those that exist are extremely light on the documentation. I'm confused as to how viable methods like policy and value iteration are when solving online (in terms of computational cost and resulting added latency in the HFT world). Are there any good examples of someone taking a set of HJB equations for optimal market making (inventory management) or execution and going all the way (preferably with code or pseudocode) to a working mvp?