Add support for Greybox model DOF counitng in degrees_of_freedom function #1512

avdudchenko · 2024-10-26T07:02:50Z

Fixes

The current degrees_of_freedom function does not properly account for GreyBoxModel as a set of constraints, resulting in an incorrect number of DOFs being presented.

Summary/Motivation:

With increased usage of GreyBoxModel (specifically Reaktoro-pse) there is a need to support DOF counting when building IDAES model with GreyBoxModels

Changes proposed in this PR:

-add the capability to add unfixed input and output vars on GreyBox model to ComponentSet produced by unfixed_variables_in_activated_equalities_set
-add a counter for the number of "equalities" in gray box model in number_activated_equalities by adding number_grey_box_equalities method

Counting the number of equalities in GreyBox model:
GrayBox models do not provide traditional Pyomo Constraints that bind inputs to outputs as such, we need to assume how many GrayBox model DOFs there are.

In most cases (at least I can't think of any exceptions) GreyBoxes must be 0DOF in the eyes of Pyomo/IPOPT as each output must be some sort of function of provided inputs, regardless if the internal model in the GreyBox is 0DOF, an optimization, or it exposes internal constraints to solve the problem.
As such, it should be safe to assume that the number of outputs equals the number of equality constraints.

Legal Acknowledgement

By contributing to this software project, I agree to the following terms and conditions for my contribution:

I agree my contributions are submitted under the license terms described in the LICENSE.txt file at the top level of this directory.
I represent I am authorized to make the contributions and grant the license. If my employer has rights to intellectual property that includes these contributions, I represent that I have received permission to make contributions and grant the required license on behalf of that employer.

codecov-commenter · 2024-10-26T07:48:11Z

Codecov Report

Attention: Patch coverage is 87.50000% with 12 lines in your changes missing coverage. Please review.

Project coverage is 77.06%. Comparing base (5841433) to head (1bf6ff2).

Files with missing lines	Patch %	Lines
idaes/core/util/model_statistics.py	87.65%	7 Missing and 3 partials ⚠️
idaes/core/util/model_diagnostics.py	86.66%	1 Missing and 1 partial ⚠️

Additional details and impacted files

@@           Coverage Diff           @@
##             main    #1512   +/-   ##
=======================================
  Coverage   77.05%   77.06%           
=======================================
  Files         384      384           
  Lines       62338    62428   +90     
  Branches    10222    10245   +23     
=======================================
+ Hits        48034    48109   +75     
- Misses      11875    11887   +12     
- Partials     2429     2432    +3

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚨 Try these New Features:

Flaky Tests Detection - Detect and resolve failed and flaky tests

dangunter · 2024-10-26T15:37:06Z

idaes/core/util/model_statistics.py

+    ) + number_grey_box_equalities(block)
+
+
+def number_grey_box_equalities(block):


number_of_X is less ambiguous in English.
Can add -> int to indicate return type and, if it's possible, the expected type of block.
In docs: "Method to" is superfluous and at any rate this is a function.

dangunter · 2024-10-26T15:43:38Z

idaes/core/util/model_statistics.py

+
+def number_grey_box_equalities(block):
+    """
+    Method to return the number of equality constraints in GreyBox


I would say, "Compute total number of equality constraints for all GreyBox objects in this block."

andrewlee94

Thanks for this, it is a case we have not considered and Greybox models are becoming more important. To start with, I had a few suggestions for more tests and modularization of the code.

However, a more significant request would be to add these tools and (and some test cases) into the diagnostics toolbox as well. We should at a minimum check that the DoF and model statistics output (number of variables and constraints) accurately reflect the model with a Greybox present. It might also be nice to add additional sub-rows in the model statistics for greybox components when they are present. Finally, we should look for any edge cases that might occur with the rest of the toolbox and if there are other things we need to consider when using greybox models.

andrewlee94 · 2024-10-28T14:27:19Z

idaes/core/util/model_statistics.py

@@ -529,7 +555,7 @@ def deactivated_inequalities_generator(block):
        block : model to be studied

    Returns:
-        A generator which returns all indeactivated equality Constraint
+        A generator which returns all in deactivated equality Constraint


The "in" should be removed here.

andrewlee94 · 2024-10-28T14:40:10Z

idaes/core/util/model_statistics.py

+    # Checks for greyboxes, and if they exist will add
+    # input and output vars to var_set if they are free
+    # inputs and outputs are defined names for greybox class and should always exist
+    for grey_box in _iter_indexed_block_data_objects(


I think it would be good to have a number_of_greybox_variables function to do this so that users can check that separately and so that we can test the code in isolation.

added one for variables, and unfixed variables.

andrewlee94 · 2024-10-28T14:41:17Z

idaes/core/util/tests/test_model_statistics.py

@@ -685,6 +689,42 @@ def test_degrees_of_freedom(m):
    assert degrees_of_freedom(m.b2) == -1


+@pytest.mark.unit


We should also have test for the functions to collect the number of greybox variables and constraints.

added tests

Robbybp · 2024-10-28T16:57:54Z

In most cases (at least I can't think of any exceptions) GreyBoxes must be 0DOF in the eyes of Pyomo/IPOPT as each output must be some sort of function of provided inputs, regardless if the internal model in the GreyBox is 0DOF, an optimization, or it exposes internal constraints to solve the problem.
As such, it should be safe to assume that the number of outputs equals the number of equality constraints.

IIRC, outputs just get translated to equality constraints of the form y = GB(x), but the greybox can define other equality constraints as well. So the number of equality constraints is at least the number of outputs, but can be greater.

avdudchenko · 2024-10-28T19:57:26Z

@Robbybp : Right, so you can include equality constraints in a graybox and send them to the solver (ipopt) - but the variables in those constraints are not actually in pyomo space- and thus we can't count them (hence why here I am only looking at inputs and outputs and ignoring any internal equality constraints). (e.g. looking at this example)

At this point, we just want to look at the graybox as a constraint that binds exposed pyomo inputs to outputs on it, and assume that the users properly setup the internal graybox model to have the correct DOFS... ideally we would have some metadata on the graybox that tells us its internal # of DOFs so we can include it in the total count. But I opt to avoid that for now... unless we want to add support for this now.

Robbybp · 2024-10-28T20:55:32Z

idaes/core/util/model_statistics.py

+    equalities = 0
+    for grey_box in _iter_indexed_block_data_objects(
+        block, ctype=ExternalGreyBoxBlock, active=True, descend_into=True
+    ):
+        equalities += len(grey_box.outputs)
+    return equalities


As written, this will be incorrect for grey box models that have a nonzero number of "explicitly defined equality constraints" (as opposed to the equality constraints that are created to define the outputs). I think all that's needed to correct this is:

equalities += grey_box.get_external_model().n_equality_constraints()

Right, did not think about this scenario! Updated to test graybox with suggestion, this now includes an example constraint that binds 2 inputs together.

def input_names(self): return ["a1", "a2", "a3"] def output_names(self): return ["o1", "o2"] def equality_constraint_names(self): return ["a12"] def evaluate_equality_constraints(self): a1 = self._input_values[0] a2 = self._input_values[1] return [a1 * 0.5 - a2]

avdudchenko · 2024-10-29T00:33:30Z

Thanks for this, it is a case we have not considered and Greybox models are becoming more important. To start with, I had a few suggestions for more tests and modularization of the code.

However, a more significant request would be to add these tools and (and some test cases) into the diagnostics toolbox as well. We should at a minimum check that the DoF and model statistics output (number of variables and constraints) accurately reflect the model with a Greybox present. It might also be nice to add additional sub-rows in the model statistics for greybox components when they are present. Finally, we should look for any edge cases that might occur with the rest of the toolbox and if there are other things we need to consider when using greybox models.

So I started adding display to diagnostics tools

the key question is should we include gray box stats with all others (e.g when counting blocks - should gray blocks count to total block count?, same with variables etc?, or should they be separate category only?)

idaes/core/util/model_statistics.py

lbianchi-lbl · 2024-10-31T18:44:30Z

This is just waiting for @avdudchenko to extend the reporting functionality for diagnostics. This should be in the November release.

avdudchenko · 2024-11-01T04:26:37Z

Ready for another round of reviews, I updated model statistics and diagnostics to try and treat greybox models as part of full model,
so variables and equities from greyboxes will be added with other stats as well, in addition to providing additional grey box model outputs via the diagnostic toolbox, which will show this:

    Activated Blocks: 1 (Deactivated: 0)
    Free Variables in Activated Constraints: 4 (External: 0)
        Free Variables with only lower bounds: 0
        Free Variables with only upper bounds: 0
        Free Variables with upper and lower bounds: 0
    Fixed Variables in Activated Constraints: 3 (External: 0)
    Activated Equality Constraints: 5 (Deactivated: 3)
    Activated Inequality Constraints: 0 (Deactivated: 0)
    Activated Objectives: 0 (Deactivated: 0)
    GreyBox Statistics
        Activated GreyBox models: 1 (Deactivated: 1)
        Activated GreyBox Equalities: 3 (Deactivated: 3)
        Free Variables in Activated GreyBox Equalities: 4 (Fixed: 1)

consistency fix

Robbybp · 2024-11-01T13:36:11Z

I'm curious what happens in the structural analysis when you have a greybox model. I can't imagine the under/overconstrained subsystems are handled correctly. Do you know what happens in this case?

Freeing up my review seeing as I am leaving the project (for now at least).

dallan-keylogic

Looks mostly good, just a few issues with docstrings to resolve.

idaes/core/util/model_statistics.py

dallan-keylogic · 2024-11-22T15:15:24Z

idaes/core/util/model_statistics.py

@@ -195,7 +294,11 @@ def number_total_constraints(block):
    Returns:
        Number of Constraint components in block
    """
-    return sum(1 for _ in activated_block_component_generator(block, ctype=Constraint))
+    number_standard_constraints = sum(


The docstring of this function needs to be updated to explain what's going on with greybox models.

dallan-keylogic · 2024-11-22T15:15:42Z

idaes/core/util/model_statistics.py

@@ -280,7 +383,9 @@ def number_deactivated_constraints(block):
    Returns:
        Number of deactivated Constraint components in block
    """
-    return sum(1 for _ in deactivated_constraints_generator(block))
+    standard_equalities = sum(1 for _ in deactivated_constraints_generator(block))


The docstring of this function needs to be updated to explain what's going on with greybox models.

dallan-keylogic · 2024-11-22T15:16:31Z

idaes/core/util/model_statistics.py

@@ -325,7 +430,9 @@ def number_total_equalities(block):
    Returns:
        Number of equality Constraint components in block
    """
-    return sum(1 for _ in total_equalities_generator(block))
+    standard_equalities = sum(1 for _ in total_equalities_generator(block))
+    greybox_equalities = number_activated_greybox_equalities(block)


The docstring of this function needs to be updated to explain what's going on with greybox models.

dallan-keylogic · 2024-11-22T15:17:05Z

idaes/core/util/model_statistics.py

@@ -377,7 +484,53 @@ def number_activated_equalities(block):
    Returns:
        Number of activated equality Constraint components in block
    """
-    return sum(1 for _ in activated_equalities_generator(block))
+    return sum(


The docstring of this function needs to be updated to explain what's going on with greybox models.

dallan-keylogic · 2024-11-22T15:25:57Z

idaes/core/util/model_statistics.py

+
+def number_activated_greybox_equalities(block) -> int:
+    """
+    Function to compute total number of equality constraints for all GreyBox objects in this block.


"total number of equality constraints" -> "total number of implied equality constraints"

Actually, it might be fine leaving as is. I didn't realize GreyBox models have an n_equality_constraints() attribute. The distinction here is probably that they don't exist as Constraint objects.

dallan-keylogic · 2024-11-22T15:28:08Z

idaes/core/util/model_statistics.py

+    A GreyBox model is always assumed to be 0DOFs where each output[i]==f(inputs)
+    where f is GreyBox model, this should be true regardless if
+    GreyBox model is doing internal optimization or not, as every output
+    is calculated through a the GreyBox internal model using provided inputs.


"through a the GreyBox" -> "through the GreyBox"

Also, is this 0DOFs property of GreyBox models always true, or is it true only if the user has correctly configured their GreyBox model?

dallan-keylogic · 2024-11-22T15:39:59Z

idaes/core/util/model_statistics.py

@@ -423,7 +576,9 @@ def number_deactivated_equalities(block):
    Returns:
        Number of deactivated equality Constraint components in block
    """
-    return sum(1 for _ in deactivated_equalities_generator(block))
+    standard_equalities = sum(1 for _ in deactivated_equalities_generator(block))


Update docstring to reflect inclusion of Greybox.

dallan-keylogic · 2024-11-22T15:44:37Z

idaes/core/util/model_statistics.py

+        A ComponentSet including all unfixed Var components which appear within
+        activated equality Constraints in block


This docstring is incorrect. Also, are greybox_variables Var objects?

dallan-keylogic · 2024-11-22T16:27:27Z

idaes/core/util/tests/test_model_statistics.py

+    assert number_total_equalities(m) == 5
+    assert number_deactivated_equalities(m) == 3
+    assert number_deactivated_constraints(m) == 3
+    report_statistics(m)


What does this line test?

dallan-keylogic · 2024-11-22T16:29:54Z

You also should add a warning to the structural analysis tool that it's not compatible with grey box models, then make sure that report_structural_issues in the diagnostics toolbox has sensible results.

avdudchenko · 2024-11-22T17:49:45Z

You also should add a warning to the structural analysis tool that it's not compatible with grey box models, then make sure that report_structural_issues in the diagnostics toolbox has sensible results.

oof, I have not worked with this tool set deeply enough to even know where to start.

Can we leave this for another PR. At this stage, I doubt many users would try to use diagnostics with Greybox models in general... so it should be okay. I would rather make a new PR after testing and finding all issues with diagnostic tools and then add appropriate doc/warnings.

Robbybp · 2024-11-22T18:42:47Z

Can we leave this for another PR. At this stage, I doubt many users would try to use diagnostics with Greybox models in general... so it should be okay. I would rather make a new PR after testing and finding all issues with diagnostic tools and then add appropriate doc/warnings.

I agree, but I don't want to confidently give wrong analyses if we're supporting GreyBox in one part of the tool (model statistics). Can you raise a "beta warning" or something in report_structural_issues and report_numerical_issues if greybox blocks are detected?

Robbybp · 2024-11-22T18:47:34Z

@avdudchenko Can you also open an issue about handling GBBs in structural and numerical diagnostics? If you provide a MWE to test on, I can try to update incidence analysis to do something reasonable.

idaes/core/util/model_statistics.py

avdudchenko · 2024-11-22T19:51:50Z

@avdudchenko Can you also open an issue about handling GBBs in structural and numerical diagnostics? If you provide a MWE to test on, I can try to update incidence analysis to do something reasonable.

Will do.

avdudchenko · 2024-11-22T19:52:19Z

Can we leave this for another PR. At this stage, I doubt many users would try to use diagnostics with Greybox models in general... so it should be okay. I would rather make a new PR after testing and finding all issues with diagnostic tools and then add appropriate doc/warnings.

I agree, but I don't want to confidently give wrong analyses if we're supporting GreyBox in one part of the tool (model statistics). Can you raise a "beta warning" or something in report_structural_issues and report_numerical_issues if greybox blocks are detected?

I added error raising for those functions for now.

dallan-keylogic · 2024-11-22T20:35:31Z

The Exceptions being raised when report_structural_issues and report_numerical_issues are being called are good, but the user could still call the structural analysis tool by itself and run into issues. Why not just have a single exception when creating the DiagnosticsToolbox (and SVDToolbox and DegeneracyHunter) if the model has any grey box models? You'd have to comment out the tests on model statistics, but leave them in so we can use them later. Also add tests making sure the exception is raised when creating any of those objects.

We can sort out which methods support grey box models and which ones don't in another issue/PR and fine tune our warnings/errors later.

…enko/idaes-pse into add_gray_box_do_counting

avdudchenko · 2024-11-22T21:07:27Z

The Exceptions being raised when report_structural_issues and report_numerical_issues are being called are good, but the user could still call the structural analysis tool by itself and run into issues. Why not just have a single exception when creating the DiagnosticsToolbox (and SVDToolbox and DegeneracyHunter) if the model has any grey box models? You'd have to comment out the tests on model statistics, but leave them in so we can use them later. Also add tests making sure the exception is raised when creating any of those objects.

We can sort out which methods support grey box models and which ones don't in another issue/PR and fine tune our warnings/errors later.

Done, I also did not remove prior tests as they directly call on report function.

idaes/core/util/model_statistics.py

first commit

fb0f8fd

avdudchenko requested review from Robbybp and bknueven October 26, 2024 07:02

avdudchenko requested review from andrewlee94, dallan-keylogic and lbianchi-lbl as code owners October 26, 2024 07:02

typo fixes

a719e47

avdudchenko requested a review from adam-a-a October 26, 2024 07:09

avdudchenko changed the title ~~Add support for Graybox model DOF counitng in degrees_of_freedom function~~ Add support for Greybox model DOF counitng in degrees_of_freedom function Oct 26, 2024

avdudchenko mentioned this pull request Oct 26, 2024

Degrees of freedom utility for GreyBox watertap-org/reaktoro-pse#10

Open

avdudchenko added the WaterTAP label Oct 26, 2024

dangunter reviewed Oct 26, 2024

View reviewed changes

Update model_statistics.py

9a09cc0

andrewlee94 previously requested changes Oct 28, 2024

View reviewed changes

Robbybp requested changes Oct 28, 2024

View reviewed changes

updates to tests

4a25d72

avdudchenko requested a review from agarciadiego as a code owner October 28, 2024 22:06

avdudchenko added 3 commits October 28, 2024 15:44

moved where we count greybox var

2324cbb

add additonal functions to get grayboxblocks

01c3a8a

initial addi tion of grayboxs stats to dignostic tools

e67e56b

Robbybp requested changes Oct 29, 2024

View reviewed changes

idaes/core/util/model_statistics.py Outdated Show resolved Hide resolved

avdudchenko added 3 commits October 28, 2024 18:53

Update test_model_statistics.py

f6a36e6

Merge branch 'main' into add_gray_box_do_counting

880a1be

Merge branch 'main' into add_gray_box_do_counting

75cc0c1

lbianchi-lbl assigned avdudchenko Oct 31, 2024

lbianchi-lbl added the Priority:High High Priority Issue or PR label Oct 31, 2024

udpate to diagonotsics

03291fa

avdudchenko requested review from Robbybp and andrewlee94 November 1, 2024 04:26

avdudchenko added 2 commits October 31, 2024 21:28

Update test_model_diagnostics.py

4b6de4c

consistency fix

Update model_diagnostics.py

d9ff557

Merge branch 'main' into add_gray_box_do_counting

2cd193a

dallan-keylogic reviewed Nov 22, 2024

View reviewed changes

addressign comments IDAES#1

c3498fc

dallan-keylogic reviewed Nov 22, 2024

View reviewed changes

idaes/core/util/model_statistics.py Outdated Show resolved Hide resolved

added errors for greybox

5ce566e

Merge branch 'main' into add_gray_box_do_counting

fe052f3

avdudchenko added 2 commits November 22, 2024 16:06

add exception tests

695b9cf

Merge branch 'add_gray_box_do_counting' of https://github.com/avdudch…

2e01052

…enko/idaes-pse into add_gray_box_do_counting

dallan-keylogic reviewed Nov 22, 2024

View reviewed changes

Update model_statistics.py

1bf6ff2

		) + number_grey_box_equalities(block)


		def number_grey_box_equalities(block):

		@@ -685,6 +689,42 @@ def test_degrees_of_freedom(m):
		assert degrees_of_freedom(m.b2) == -1


		@pytest.mark.unit

		A ComponentSet including all unfixed Var components which appear within
		activated equality Constraints in block

Add support for Greybox model DOF counitng in degrees_of_freedom function #1512

Are you sure you want to change the base?

Add support for Greybox model DOF counitng in degrees_of_freedom function #1512

Conversation

avdudchenko commented Oct 26, 2024 • edited Loading

Fixes

Summary/Motivation:

Changes proposed in this PR:

Legal Acknowledgement

codecov-commenter commented Oct 26, 2024 • edited Loading

Codecov Report

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

andrewlee94 left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Robbybp commented Oct 28, 2024

avdudchenko commented Oct 28, 2024

Choose a reason for hiding this comment

Choose a reason for hiding this comment

avdudchenko commented Oct 29, 2024

lbianchi-lbl commented Oct 31, 2024

avdudchenko commented Nov 1, 2024 • edited Loading

Robbybp commented Nov 1, 2024

dallan-keylogic left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

dallan-keylogic commented Nov 22, 2024

avdudchenko commented Nov 22, 2024

Robbybp commented Nov 22, 2024

Robbybp commented Nov 22, 2024

avdudchenko commented Nov 22, 2024

avdudchenko commented Nov 22, 2024

dallan-keylogic commented Nov 22, 2024

avdudchenko commented Nov 22, 2024

avdudchenko commented Oct 26, 2024 •

edited

Loading

codecov-commenter commented Oct 26, 2024 •

edited

Loading

avdudchenko commented Nov 1, 2024 •

edited

Loading