Alfoa/rom to pyomo

[alfoa]

---

Pull Request Description

What issue does this change request address? (Use "#" before the issue to link it, i.e., #42.)

Closes #1580

What are the significant changes in functionality due to this change request?

This PR is aimed to address issue #1580.
It adds:

• a new exporting capability of ROMs into a pyomo concrete model, activatable via Outstream system
• a derivative support for ROMs. This derivative support is based on numdiff library and is used both in the Pyomo model and, in the future, in the FMI/FMU exporter for Model exchange.

This uses PyNumero: https://pyomo.readthedocs.io/en/stable/contributed_packages/pynumero/index.html

---

For Change Control Board: Change Request Review

The following review must be completed by an authorized member of the Change Control Board.

• 1. Review all computer code.
• 2. If any changes occur to the input syntax, there must be an accompanying change to the user manual and xsd schema. If the input syntax change deprecates existing input files, a conversion script needs to be added (see Conversion Scripts).
• 3. Make sure the Python code and commenting standards are respected (camelBack, etc.) - See on the wiki for details.
• 4. Automated Tests should pass, including run_tests, pylint, manual building and xsd tests. If there are changes to Simulation.py or JobHandler.py the qsub tests must pass.
• 5. If significant functionality is added, there must be tests added to check this. Tests should cover all possible options. Multiple short tests are preferred over one large test. If new development on the internal JobHandler parallel system is performed, a cluster test must be added setting, in XML block, the node <internalParallel> to True.
• 6. If the change modifies or adds a requirement or a requirement based test case, the Change Control Board's Chair or designee also needs to approve the change. The requirements and the requirements test shall be in sync.
• 7. The merge request must reference an issue. If the issue is closed, the issue close checklist shall be done.
• 8. If an analytic test is changed/added is the the analytic documentation updated/added?
• 9. If any test used as a basis for documentation examples (currently found in raven/tests/framework/user_guide and raven/docs/workshop) have been changed, the associated documentation must be reviewed and assured the text matches the example.

[aalfonsi]

@joshua-cogliati-inl added description.

[aalfonsi]

before merging this, I would wait for the FMI/FMU to be merged, since the serialization in this branch needs to be replaced by the one in that PR

[joshua-cogliati-inl]

@aalfonsi Is there a specific pull request or issue in Pyomo this is waiting on? Thanks.

[joshua-cogliati-inl]

Based on discussion at the Dispatches meeting, this probably was waiting for Pyomo 6 release which is done. So this is probably ready to check.

[joshua-cogliati-inl]

FYI: This is currently waiting on #1608

[moosebuild]

Job Test qsubs on feb0bca : invalidated by @joshua-cogliati-inl

AssertionError: /home/moosetest/.conda/pkgs/yarl-1.6.3-py37h27cfd23_0/info/index.json

[joshua-cogliati-inl]

FYI: #1608 has been merged, so the major need of this pull is a test.

[joshua-cogliati-inl]

Status and work to be done:
The goal is to have a PyNumero ExternalGreyBoxModel that can be loaded by a ExternalGreyBoxBlock. Right now, framework/Models/ROM.py can write python file with a subclass of ExternalGreyBoxModel.
This file is designed to be run from the command line taking a -r ROM and -f framework argument to find a RAVEN ROM and the framework.

Things to be done:

• Make test that exports RAVEN ROM and ExternalGreyBoxModel. (Started doing this in tests/framework/Models/External/serialize_pyomo.xml )
• Figure out how to test this (Note that installing PyNumero requires extra steps beyond installing pyomo ( pyomo download-extensions and pyomo build-extensions )
• Add documentation
• Figure out a good way to have steps export different types of things (See XXX comment in IOStep.py)

[moosebuild]

Job Test qsubs on f80c989 : invalidated by @joshua-cogliati-inl

Conda definitions not found at /home/moosetest/miniconda2/etc/profile.d/conda.sh

[joshua-cogliati-inl]

The test now at least runs (but doesn't do anything useful).

./run_tests --re=pyomo_serialization

To make an environment that can run it (starting from the standard RAVEN environment):

conda install -c conda-forge pyomo glpk coincbc cyipopt
pyomo download-extensions
pyomo build-extensions

To run the exported file (after pyomo_serialization test run):

cd tests/framework/Models/External
python3 SerializePyomo/rom_out.py -r SerializePyomo/rom_pickle.pk -f ../../../..

[moosebuild]

Job Test qsubs on 0a15a53 : invalidated by @joshua-cogliati-inl

civet recipe change

[moosebuild]

Job Test Ubuntu 20-2 Optional on 14d21ae : invalidated by @wangcj05

[wangcj05]

I have a couple of comments for you to consider. I think the manual need to be improved. In addition, if possible please add a section in the manual to describe how to use GreyBoxModel.

[yoshiurr-INL]

The failures shown in commit 3ea5c10 making all optional libraries required proved that some Linux systems were not able to load pynumero. According to Joshua, this is due to those failing Linux systems using older C++ libraries.

[yoshiurr-INL]

The documentation for the user manual is being updated now. The changes should be committed and pushed in the following hours.

[wangcj05]

There are several unresolved comments in this PR, @yoshiurr-INL could you take a look?

[moosebuild]

Job Test qsubs sawtooth on 1852eb9 : invalidated by @yoshiurr-INL

Restart Sawtooth tests

[moosebuild]

Job Test qsubs sawtooth on 1852eb9 : invalidated by @yoshiurr-INL

Restart Sawtooth tests

[yoshiurr-INL]

Congjian, I have addressed the comments given and pushed changes. The sawtooth failure is irrelevant to the commits made and are random failures as discussed in previous RAVEN meetings.

[moosebuild]

Job Test qsubs sawtooth on 1852eb9 : invalidated by @wangcj05

[wangcj05]

PR looks good.

[wangcj05]

Checklist is good. Tests are green.
@yoshiurr-INL @aalfonsi Thanks for your contribution. Great work!