support arbitrary weights for individual streams

[proteneer]

Currently managers can already specify weights of individual targets. Next up is to allow managers to specify the weights of individual streams. It turns out prioritizing longest streams is just a special case of arbitrary weights, where weights is set equal to total number of completed frames. In particular, this will allow F@H managers to actively promote certain streams that are visiting regions of phase space of interest to them.

[proteneer]

This is a rather non-trivial feature to implement. Currently assignments are implemented as a priority queue, where the priority is equal to the number of frames. That is, we prefer longer streams over shorter streams. Unfortunately, a queue structure is not sufficient to support relative weights.

A random weighted sample is best implemented via computation of the normalized cumulative density of the weights, followed by a random sample between [0,1].

In addition, this is complicated by the fact that these weights are only used during assignment time, and that if a CPU core ends up taking a high weight stream, you'd need to force a disengage server-side.

The main use cases I see this are:

1. Adding new streams to an existing target that needs a bit more sampling relative to other existing streams without stopping all the other streams.
2. Lowering the priority of an existing stream that has too much sampling (though you might as well as just stop the relevant stream for this).

If you need more fine grained control, use the existing siegetank stream.stop() and stream.start() methods that forces a core to disengage from a stream and prevents/allows future assignments. So using these two methods, you can more or less get the desired ratio of frames by stopping streams that have too many frames. Albeit you'd need to do this proactively.

What are your use cases?

[proteneer]

(PS this is actually further complicated by the fact that streams's weights are relative only to other streams on the same SCV).