Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add ecl2df as utility #198

Merged
merged 33 commits into from
Aug 31, 2023
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
33 commits
Select commit Hold shift + click to select a range
7802323
Add utilities submodule
daniel-sol Aug 18, 2023
8442f9f
Add reek test data from ecl2df
daniel-sol Aug 18, 2023
843568e
Add module ecl2csv to utilities
daniel-sol Aug 18, 2023
083ba6e
Make Submodules tuple and dictionary global
daniel-sol Aug 21, 2023
14786d3
Commit first working version
daniel-sol Aug 23, 2023
3cafce6
Add csv files to gitignore
daniel-sol Aug 23, 2023
dce614f
Add code for uploading
daniel-sol Aug 24, 2023
3d6cbc4
Fix some issues for upload
daniel-sol Aug 24, 2023
575fc1f
Fix test for upload
daniel-sol Aug 25, 2023
a06ca24
Make arg_parse give extra help
daniel-sol Aug 25, 2023
386209d
Rename to sim2sumo
daniel-sol Aug 25, 2023
d87977a
Add code for installing SIM2SUMO in ert
daniel-sol Aug 28, 2023
8a0858a
Change CONFIG_PATH TO S2S_CONF_PATH
daniel-sol Aug 29, 2023
042f529
Add ecl2df as requirement
daniel-sol Aug 29, 2023
6f29635
Correct spelling mistake in ecl2df
daniel-sol Aug 29, 2023
4f8ad4c
Remove fmu.config from code, remove tests on parse_args, needs more love
daniel-sol Aug 29, 2023
4c00cef
Add sim2sumo as console script
daniel-sol Aug 29, 2023
cbab960
Correct typo in setup.py
daniel-sol Aug 29, 2023
ff48288
Add fmu-dataio as requirement
daniel-sol Aug 29, 2023
83a26b2
Add again ert as requirement
daniel-sol Aug 29, 2023
c094e04
Fix ert job installation
daniel-sol Aug 30, 2023
bc71b5a
Add line shift
daniel-sol Aug 30, 2023
b1501e2
Merge with remote branch 'add_ecl2df_as_utility'
daniel-sol Aug 30, 2023
bb78d00
Change section ecl2csv to sim2sumo in test data
daniel-sol Aug 30, 2023
5a8f636
add token to sumo connections in tests
equinor-ruaj Aug 30, 2023
f5cae43
fix keywerror ecl2df -> sim2sumo
adnejacobsen Aug 30, 2023
7063af9
Add missing execute in SIM2SUMO, remove a . in upload
daniel-sol Aug 30, 2023
0330350
Change ecl2csv section sim2sumo in tests, add stacktrace for exceptions
daniel-sol Aug 30, 2023
77631fe
Remove unused functions
daniel-sol Aug 30, 2023
6d8bae1
Merge branch 'add_ecl2df_as_utility' of github.com:equinor/fmu-sumo i…
daniel-sol Aug 30, 2023
86705da
Add tidy func to clean up ecl2dfs mess when rft is exctracted
daniel-sol Aug 30, 2023
bb972c4
Comment out test_upload, awaiting refactoring of uploader
daniel-sol Aug 30, 2023
b5e80ce
Add try/except for yaml_load, change to set in _defin_submodules
daniel-sol Aug 31, 2023
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 3 additions & 1 deletion .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -99,4 +99,6 @@ _venv
src/fmu/sumo/version.py

# local expermentation
testing.ipynb
testing.ipynb
# files generated during testing
*.csv
4 changes: 3 additions & 1 deletion requirements/requirements.txt
Original file line number Diff line number Diff line change
Expand Up @@ -5,5 +5,7 @@ sumo-wrapper-python @ git+https://github.com/equinor/sumo-wrapper-python.git@mai
xtgeo
azure-core
pyarrow; python_version > "3.6.1"
# ert
ert
ecl2df
fmu-dataio
OpenVDS; sys_platform != 'darwin'
3 changes: 2 additions & 1 deletion setup.py
Original file line number Diff line number Diff line change
Expand Up @@ -66,7 +66,8 @@ def parse_requirements(fname):
"sumo_upload = fmu.sumo.uploader.scripts.sumo_upload",
],
"console_scripts": [
"sumo_upload=fmu.sumo.uploader.scripts.sumo_upload:main"
"sumo_upload=fmu.sumo.uploader.scripts.sumo_upload:main",
"sim2sumo=fmu.sumo.utilities.scripts.sim2sumo:main",
],
},
cmdclass=CMDCLASS,
Expand Down
25 changes: 25 additions & 0 deletions src/fmu/sumo/config_jobs/SIM2SUMO
Original file line number Diff line number Diff line change
@@ -0,0 +1,25 @@
-- This forward model uploads results from reservoir simulators eclipse or opm
-- to sumo. It is called from the ERT config file as a regular forward model

-- Arguments:
-- S2S_CONF_PATH: path to config
-- SUMO_ENV: sumo environment to upload to

DEFAULT <S2S_CONF_PATH> fmuconfig/output/global_variables.yml
DEFAULT <SUMO_ENV> prod

STDERR sim2sumo.stderr
STDOUT sim2sumo.stdout


EXECUTABLE sim2sumo

ARGLIST execute "--config_path" <S2S_CONF_PATH> "--env" <SUMO_ENV>

MIN_ARG 5
MAX_ARG 5
ARG_TYPE 0 STRING
ARG_TYPE 1 STRING
ARG_TYPE 2 STRING
ARG_TYPE 3 STRING
ARG_TYPE 4 STRING
Empty file.
91 changes: 91 additions & 0 deletions src/fmu/sumo/utilities/scripts/sim2sumo.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,91 @@
#!/usr/bin/env python

"""Upload data to Sumo from FMU."""

import warnings
import os
import argparse
import logging
from pathlib import Path
from ert.shared.plugins.plugin_manager import hook_implementation
from ert.shared.plugins.plugin_response import plugin_response
from ert import ErtScript
from fmu.sumo.utilities.sim2sumo import parse_args, upload_with_config

LOGGER = logger = logging.getLogger(__name__)

DESCRIPTION = """SIM2SUMO uploads results from reservoir simulators directly to sumo.
Typically this is run per realization. The upload is controlled by a yaml config file

SIM2SUMO is implemented both as a FORWARD_JOB and can be called only from this context
when running ERT."""

EXAMPLES = """

FOR full blown example add this to your ert config:

FORWARD_MODEL SIM2SUMO(<S2S_CONF_PATH>= <your config location>, <SUMO_ENV>=<sumo env to upload to>)
<S2S_CONFIG_PATH> refers to the config file that controls the upload. This file can be a regular
fmu config file, or a completely separate file, but needs to be in yaml format, and contain
a section called sim2sumo to produce any results. Defaults to fmuconfig/output/global_variables.yml
<SUMO_ENV> refers to the sumo environment to upload to. Defaults to prod

For minimum amount of clutter in your ert config utilize the defaults. This means that if you
add section sim2sumo to the fmu config file, and store it at the recommendation and upload to
the prod environment for sumo then your call in the ert config can be reduced to

FORWARD_MODEL SIM2SUMO."""


class Sim2Sumo(ErtScript):
"""A class with a run() function that can be registered as an ERT plugin.

This is used for the ERT workflow context."""

# pylint: disable=too-few-public-methods
def run(self):
# pylint: disable=no-self-use
"""Parse with a simplified command line parser, for ERT only,
call sumo_upload_main()"""
logger.debug("Calling run() on Sim2Sumo")
args = parse_args()

upload_with_config(args.config_path, args.env)


# @hook_implementation
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Intentionally left in the code?

# def legacy_ertscript_workflow(config):
# """Hook the Sim2Sumo class into ERT with the name SIM2SUMO,
# and inject documentation"""
# workflow = config.add_workflow(Sim2Sumo, "SIM2SUMO")
# workflow.parser = _get_parser
# workflow.description = DESCRIPTION
# workflow.examples = EXAMPLES
# workflow.category = "export"


def main():
"""Main function, to be executed as console script"""
args = parse_args()
upload_with_config(args.config_path, args.env)


@hook_implementation
@plugin_response(plugin_name="SIM2SUMO")
def job_documentation(job_name):
"""Add job documentation for forward model

Args:
job_name (str): name of job

Returns:
dict: the documentation to be provided
"""
if job_name != "SIM2SUMO":
return None

return {
"description": DESCRIPTION,
"examples": EXAMPLES,
"category": "export",
}
Loading
Loading