Improvements on base AMISC algorithm

[eckelsjd]

Here is a list of some ideas to improve the base AMISC algorithm. Most are focused on better ways to explore the multi-index search space, which gets quite large when you have >10 inputs.

1. Take more than one candidate multi-index at each iteration. For example, take the one with the biggest change in surrogate prediction and the one with the biggest reduction in test set error.
2. Reduce the index search space by grouping similar inputs and refining all at once when the "super" candidate index is chosen. This will lead to larger improvements per iteration instead of the super granular approach currently used. You could also do some sort of compression in the input space and build a surrogate over the compressed latent space.
3. You could then also incorporate compression accuracy into the adaptive training procedure.
4. Include spatial adaptivity somehow to help fit local features better. Local basis functions can fit discontinuities and spiky regions better than global lagrange interpolants.
5. Always take the first $\alpha$ candidates; sometimes they get lost when the index set gets very anisotropic.
6. Make it easier to include external surrogate training data that is likely unstructured. Right now, the surrogate must be built from scratch in a very orderly fashion -- there should be a way to include existing training data, or data that comes externally after the surrogate was built. Just like neural nets that can keep adding data as it becomes available.