TSC Meeting (12 Nov 2024)

[cognifloyd]

TSC Meeting (12 November 2024)

November 2024 @StackStorm/tsc 1 hour meeting:

• Tuesday, 12th Nov 2024, 9:30 AM US Pacific / 6:30 PM EU CEST
• See StackStorm TSC Meetings #33 for the schedule and how to join
• TLDR; Jitsi link: https://meet.jit.si/StackStormTSC

Attendees

Present

• AJ guzzijones
• Carlos nzlosh
• Jacob Floyd cognifloyd
• Scott jk464

Absent

• Amanda McGuinness amanda11
• Ankur Singh rush-skills
• Bradley Bishop bishopbm1
• Dan Porter Stealthii
• Dmitri Zimine dzimine
• FileMagic FileMagic
• Khushboo khushboobhatia01
• Marcel Weinberg winem
• Mark Mercado mamercad
• Mick McGrath mickmcgrath13
• Ravi
• Tomaz Muraus Kami
• W Chan m4dcoder

Agenda

No agenda set before the meeting started.

Meeting Minutes

• Pantsbuild-based infrastructure project updates:
  • pants ci: add integration tests jobs st2#6273
    • Work to get Pants to run Integration Tests in CI is nearly complete.
    • This work involves making it safe to run tests in parallel. To do that we have to prevent parallel tests from using the same Mongo database, RabbitMQ Exchanges/Queues, and Redis Keys.
      • For MongoDB, the tests already suffix db_name with ST2TESTS_PARALLEL_SLOT number
      • For RabbitMQ, I tried several approaches before settling on modifying the exchange/queue name prefix from st2. to st2test{ST2TESTS_PARALLEL_SLOT}.. Using vhost was one of the first things I tried, but that had a lot of downsides as the vhost has to be declared before running the tests, so running tests would become more difficult.
        • The prefix is set via a new config option [messaging].prefix and defaults to st2, which matches the current behavior.
      • For Redis, I found that the tooz redis driver has a configurable namespace for the keys it uses. The driver's default namespace is _tooz. So, I added &namespace=_st2_test{ST2TESTS_PARALLEL_SLOT} to the coordination url.
    • This also required giving integration tests a way to pass these modified configuration settings to subprocesses that do not use the config code in st2tests. Luckily, we're already using a newer version of oslo_config that supports loading configuration values from env vars in addition to the conf file(s).
      • So, we will get a new feature that will allow people to set any of the conf settings via env vars
      • The pattern for the vars is ST2_<conf section>__<option name>. For example ST2_MESSAGING__URL and ST2_DATABASE__PASSWORD.
    • Jacob will be splitting this PR into several smaller PRs now that integration tests are passing.
    • This does not yet run self-check tests using pants.
  • Once all of the integration test PRs get merged, the next step will be updating BUILD metadata to build packages (wheels, pex file, and system packages like rpm and deb). Then, we'll need to figure out how to run our package tests on the new packages (the tests are currently in a big tangled mess in the st2-packages repo--Hopefully we'll be able to archive that repo once the pantsbuild infrastructure project is complete).
• Cutting a new release or release candidate:
  • Cutting a release should not have to wait for the pantsbuild infrastructure project to be finished.
  • Someone needs to volunteer as release manager and deal with the (currently onerous) release process to do that.
  • A significant issue is that none of the packages on the exchange have been tested with newer versions of python, so we don't know the impact of a switch to newer python versions.
    • Running stackstorm on one version of python 3 and packs on a different (older) python 3 might be possible using some of the code we used for the python2->3 migration. The gotcha is that pack venvs are tied to the st2 venv right now, treating st2 as if it were the "system" venv so that any packages installed with st2 are available to packs.
  • Carlos will look into what is required for the next release and create an issue documenting things people can do to help (That list will--at a minimum--include an item for someone volunteering to manage the release, and an item for investigating the pack python version issue).
  • Once the pantsbuild infrastructure project is mostly complete, Jacob will be the release manager for whatever version is next to be released. That will serve to simplify the release process and make it easier/faster to make more frequent releases.

[nzlosh]

Does the release process support Release Candidates?

As per the discussion of having a release candidate for v3.9, it does appear the release design has the notion of release candidates as described here https://github.com/StackStorm/st2/wiki/Packaging

Release candidate support is present in the form of the staging-stable repository however these packages are only updated during the release phase and we want feedback before attempting to release.

To get user acceptance and feedback to help decide if we should begin the release phase, the unstable repository has all updates to be included in the upcoming release.

We would need to announce the availability of unstable v3.9 packages (perhaps a specific version such as 3.9-187 should also be mentioned) via the st2 social network account to get people involved in evaluating v3.9 readiness and provide us feedback.

That said, there are no packages being published for Ubuntu 22.04 (Jammy), so evaluating st2 on py3.10 will need this to be fixed first.