Accessing the general closed-loop workflow from theorist/experimentalist in a module

[Vinwcent]

Hello community,

While developing a new module, I've encountered a small issue.
My aim was to access the general closed-loop workflow from within my theorist estimator. However, due to the estimator utilizing the BaseEstimator class, it's constrained to only receive conditions and observations of the cycle class which does not allow to modify the input of the theorist.

As a temporary workaround, we've considered bypassing the use of the cycle class altogether and directly computing conditions and observations in a simple for loop. Here's an illustrative example:

theo = MyTheorist()

for cycle in range(n_cycles):
      model = theo.fit(conds, obs)
      new_conds = exp_sample(conds, model)
      new_obs = experiment(new_conds)
      conds = np.vstack([conds, new_conds])
      obs = np.vstack([obs, new_obs])

This is more of a "hacky" solution here that may have a proper long-term solution inside of the Cycle class in the future.

[younesStrittmatter]

Hello @Vinwcent,

Maybe I am misunderstanding your question, but the condition and observation objects of the cycle used in your model's fit function already contains all the conditions and observations (not only one of the newest cycle).

[musslick]

Hi @Vinwcent,
The following notebook may illustrate how to maintain a history of events externally and within a sklearn theorist. There should also be a second solution using the cycle logic, but there is still a minor issue we need to resolve with that. In the meantime, this should work:
https://colab.research.google.com/drive/19JlE6blWipYBBpoqsWo28ZbyTnWxK1CQ?usp=sharing

The basic idea illustrated in the notebook is that you can keep a history within the class deriving from BaseEstimator:

import numpy as np
from sklearn.base import BaseEstimator


class ExampleRegressorWithHistory(BaseEstimator):
    """
    This theorist fits a polynomial function to the data.
    """

    def __init__(self, degree: int = 2):
        self.degree = degree
        self.reward_history = [] # this will be used to store history of rewards
        self.coeff_history = [] # this will be used to store history of model coefficients

    def fit(self, conditions, observations):
        # making sure that input is 1d (as required by poly1d)
        if len(conditions.shape) == 2:
          conditions = conditions.flatten()
        if len(observations.shape) == 2:
          observations = observations.flatten()
        
        # fit polynomial
        self.coeff = np.polyfit(conditions, observations, 2)
        self.polynomial = np.poly1d(self.coeff)
        pass

    def predict(self, conditions):
        return self.polynomial(conditions)

    def update_history(self, reward):
        self.reward_history.append(reward)
        self.coeff_history.append(self.coeff)

[hollandjg]

Hi @Vinwcent , the easiest way to do that today is to use the cycle history:
https://autoresearch.github.io/autora/user-guide/workflow/interactive/Accessing%20State%20Dependent%20Properties/

Long story short: you can pass special variables to the "params" argument of the cycle, which are resolved to the observations and conditions each cycle.

For example, here we define a cycle where the experimentalist gets all the observed IVs "%observations.ivs%" each cycle:

cycle_with_state_dep_properties = Cycle(
    variables=variables,
    theorist=example_theorist,
    experimentalist=unobserved_data_experimentalist,
    experiment_runner=example_synthetic_experiment_runner,
    params={
        "experimentalist": {
            "exclude_conditions": {"excluded_conditions": "%observations.ivs%"},
            "custom_random_sampler": {"n": 1}
        }
    }
)

[chadcwilliams]

Closed as this is answered.