Snowfakery Batch Size and Just Once

[acrosman]

Describe the bug

Using Batch with Just Once causes unexpected results. It appears that a list of records created during the first batch of a run is lost as a reference to later batches.

Using the following recipe (my test run was on NPSP, but the main behavior should impact all orgs just with slight differences):

- object: Account
  just_once: true
  count: 5
  fields:
    Name:
      fake: company

- object: Contact
  count: 5
  fields:
    FirstName:
      fake: FirstName
    LastName:
      fake: LastName
    AccountId:
      random_reference: Account

If I run this with a command like:
cci task run generate_and_load_from_yaml --generator_yaml .\datasets\batch.recipe.yml --num_records 450 --num_records_tablename Contact --org myOrg

I will get 450 Contacts in my org, all linked to one of 5 Accounts generated in the recipe.

If I run this with a command that uses batch size:
cci task run generate_and_load_from_yaml --generator_yaml .\datasets\batch.recipe.yml --num_records 450 --num_records_tablename Contact --org myOrg --batch_size 20

The first 20 contact will match that pattern. But the next 430 will not have an Account from the initial list of 5. In NPSP they will have households, in a basic Sales Cloud org I would expect them to fail entirely since AccountId is not set.

Reproduction steps

1. Setup CCI Project, and connect to an org (only tested with a static org, I expect a scratch will have the same behavior)
2. Create a file with the Snowfakery recipe above.
3. Run: cci task run generate_and_load_from_yaml --generator_yaml .\datasets\batch.recipe.yml --num_records 450 --num_records_tablename Contact --org myOrg --batch_size 20
4. The first batch should work, subsequent batches will fail (or use default accounts if there is a package that creates them)

Your CumulusCI and Python versions

• CumulusCI version: 3.85.0 (c:\users\acros.local\bin\cci)
• snowfakery version 3.6.1
• Python version: 3.11.4 (C:\Users\acros.local\pipx\venvs\cumulusci\Scripts\python.exe)

Operating System

Windows 11

Windows environment

PowerShell

CumulusCI installation method

pipx

Error Gist

No response

Additional information

It's hard to be sure if this is a Snowfakery issue or CCI issue. I opened it as a CCI issue because batch-size is an option related to the CCI task used to generate the data.