You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
I recently got interested in this repo & Normal Computing in general. I wanted to contribute in some way so took a stab at this issue. I'm a bit uncertain about some of the math, so I thought I'd implement the 90% that I think I figured out, and ask about the remaining 10%.
Would you be interested in me opening a pull request with the code that I have? It has a unit test that shows it works in the sense that it adheres to the API, but I haven't tested the logic yet because of the parts I'm unsure of.
We should definitely add Stein variational gradient descent (paper, code)
SVGD requires a kernel specification. IMO we don't need to supply a suite of kernels ourselves (aside from maybe a default Gaussian kernel).
I think we should enforce a kernel signature like
where
kernel_params
are any kernel hyperparameters such as bandwidth.To future-proof against more sophisticated kernels that e.g. could use info from the model call via
aux1
andaux2
.Also we should think about how to support adaptive
kernel_params
updates like the median heuristic used in the SVGD [paper](https://proceedings.neurips.cc/paper/2016/file/b3ba8f1bee1238a2f37603d90b58898d-Paper.pdf.The text was updated successfully, but these errors were encountered: