Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

w3emc upgrade on WCOSS2? #234

Open
ShelleyMelchior-NOAA opened this issue Jun 25, 2024 · 26 comments
Open

w3emc upgrade on WCOSS2? #234

ShelleyMelchior-NOAA opened this issue Jun 25, 2024 · 26 comments

Comments

@ShelleyMelchior-NOAA
Copy link
Contributor

Hi NCEPlibs team! I am just looking for a bit of guidance on steps forward as they relate to the next upgrade of EVS (v2).

The current operations version of EVS (v1.0.9) uses w3emc/2.9.2 and ip/3.3.3. We could not use ip/4.0.0 b/c of a dependency on ip/3.3.3 to make sure makgds worked properly. Once w3emc is upgraded on WCOSS2 to be w3emc/2.9.3, EVS can use ip/4.0.0.

EVSv2 development is underway and we want to know when w3emc will be upgraded on WCOSS2. We are also curious which version of w3emc will be installed since now I see there are releases up to w3emc.v2.12.0!

Any guidance you can provide is appreciated.
@Hang-Lei-NOAA @edwardhartnett @AliciaBentley-NOAA @PerryShafran-NOAA @aerorahul
@Hang-Lei-NOAA
Copy link
Contributor

Hang-Lei-NOAA commented Jun 26, 2024 via email

@edwardhartnett
Copy link
Contributor

@Hang-Lei-NOAA can we just jump to the latest release, and skip intermediate releases?

@Hang-Lei-NOAA
Copy link
Contributor

Hang-Lei-NOAA commented Jun 26, 2024 via email

@ShelleyMelchior-NOAA
Copy link
Contributor Author

Thanks @Hang-Lei-NOAA and @edwardhartnett . We can do a baseline test of w3emc/2.10.0 on acorn by trying to compile the code that uses makgds, but will it ever be promoted to a prod installation on dogwood or cactus? EVS runs in ops so we cannot use libraries on acorn for operational runs.

@ShelleyMelchior-NOAA
Copy link
Contributor Author

Hi @Hang-Lei-NOAA and @edwardhartnett , I wanted to repeat my question in case it was overlooked with holiday leave.

Is it expected that w3emc/2.10.0, or any other future version, will be promoted to a prod installation on dogwood or cactus?

@Hang-Lei-NOAA
Copy link
Contributor

@ShelleyMelchior-NOAA Have you tested with these versions that I installed on acorn it is under
/lfs/h1/emc
/nceplibs/noscrub/hpc-stack/libs/hpc-stack/modulefiles/compiler/intel/19.1.3.304

Once you confirmed working, we can start the delivery.

@ShelleyMelchior-NOAA
Copy link
Contributor Author

ShelleyMelchior-NOAA commented Jul 10, 2024 •
edited
Loading

Ok, thanks @Hang-Lei-NOAA . That answers my question. We will test the installation on acorn and get back to you.

I am uncertain how to access acorn. Before I task a team member who can access acorn to test the w3emc/2.10.0 installation, can you confirm that all other modules are installed on acorn that the EVS sorc code needs for successful compilation: w3emc, ip, bacio, sp, g2, jasper, libpng, zlib?
If so, are they installed at the same location as w3emc/2.10.0?

@Hang-Lei-NOAA
Copy link
Contributor

Hang-Lei-NOAA commented Jul 11, 2024 via email

@ShelleyMelchior-NOAA
Copy link
Contributor Author

Thank you, @Hang-Lei-NOAA ! I was able to test the compiling of the EVS source code with the test installation for w3emc/2.10.0 and w3emc/2.12.0. I was also pointing to the latest production installation for ip/4.0.0. Good news! Both versions of w3emc compile with no errors and generate the executable files. The EVS team is endorsing the delivery of the w3emc upgrade to NCO.

@Hang-Lei-NOAA
Copy link
Contributor

Hang-Lei-NOAA commented Jul 11, 2024 via email

@ShelleyMelchior-NOAA
Copy link
Contributor Author

Fantastic! Can you keep me apprised of the progress of the release and installation in prod? We will want to update the EVS code in development to point to the upgraded w3emc library once it is installed in operations.

Also, for your awareness, we are targeting this for use in EVSv2, which will be delivered to NCO in May 2025.

@Hang-Lei-NOAA
Copy link
Contributor

Hang-Lei-NOAA commented Jul 11, 2024 via email

@Hang-Lei-NOAA
Copy link
Contributor

Hang-Lei-NOAA commented Jul 22, 2024 via email

@ShelleyMelchior-NOAA
Copy link
Contributor Author

Thank you for the update, @Hang-Lei-NOAA ! I will test the para install and report back.

@ShelleyMelchior-NOAA
Copy link
Contributor Author

Hi @Hang-Lei-NOAA ! When I previously tested the update for w3emc, I did not realize there was an update to g2. My tests were using the prod installation of g2/3.4.5. Compilation errors occur when I use v3.5.1 for g2.

✔️ EVS compiles correctly when I use the cactus installation at /apps/ops/para/libs/modulefiles/compiler/intel/19.1.3.304 for w3emc/2.12.0, ip/4.0.0, and the prod installation for g2/3.4.5.

❌ EVS does not compile correctly when I use the cactus installation at /apps/ops/para/libs/modulefiles/compiler/intel/19.1.3.304 for w3emc/2.12.0, ip/4.0.0, and g2/3.5.1.

Do we have to use the updated g2 to use updated w3emc and ip?

Provided is the compilation error that I see:

ftn -g -traceback -O3 -auto -convert big_endian -I /apps/ops/para/libs/intel/19.1.3.304/g2/3.5.1/include_4 -o sref_precip.x sref_precip.f /apps/ops/prod/libs/intel/19.1.3.304/ip/4.0.0/lib64/libip_4.a /apps/ops/para/libs/intel/19.1.3.304/w3emc/2.12.0/lib64/libw3emc_4.a /apps/ops/prod/libs/intel/19.1.3.304/bacio/2.4.1/lib/libbacio_4.a  /apps/ops/para/libs/intel/19.1.3.304/sp/2.4.0/lib64/libsp_4.a /apps/ops/para/libs/intel/19.1.3.304/g2/3.5.1/lib64/libg2_4.a /apps/spack/jasper/2.0.25/intel/19.1.3.304/sjib74krrorkyczqpqah4tvewmlnqdx4/lib64/libjasper.a /apps/spack/libpng/1.6.37/intel/19.1.3.304/4ohkronuhlyherusoszzrmur5ewvlwzh/lib/libpng.a /apps/spack/zlib/1.2.11/intel/19.1.3.304/hjotqkckeoyt6j6tibalwzrlfljcjtdh/lib/libz.a
/usr/lib64/gcc/x86_64-suse-linux/7/../../../../x86_64-suse-linux/bin/ld: /apps/spack/jasper/2.0.25/intel/19.1.3.304/sjib74krrorkyczqpqah4tvewmlnqdx4/lib64/libjasper.a(jpg_dec.c.o): in function `jpg_decode':
jpg_dec.c:(.text+0x237): undefined reference to `jpeg_std_error'
/usr/lib64/gcc/x86_64-suse-linux/7/../../../../x86_64-suse-linux/bin/ld: jpg_dec.c:(.text+0x251): undefined reference to `jpeg_CreateDecompress'
/usr/lib64/gcc/x86_64-suse-linux/7/../../../../x86_64-suse-linux/bin/ld: jpg_dec.c:(.text+0x261): undefined reference to `jpeg_stdio_src'
/usr/lib64/gcc/x86_64-suse-linux/7/../../../../x86_64-suse-linux/bin/ld: jpg_dec.c:(.text+0x273): undefined reference to `jpeg_read_header'
/usr/lib64/gcc/x86_64-suse-linux/7/../../../../x86_64-suse-linux/bin/ld: jpg_dec.c:(.text+0x323): undefined reference to `jpeg_start_decompress'
/usr/lib64/gcc/x86_64-suse-linux/7/../../../../x86_64-suse-linux/bin/ld: jpg_dec.c:(.text+0x48f): undefined reference to `jpeg_read_scanlines'
/usr/lib64/gcc/x86_64-suse-linux/7/../../../../x86_64-suse-linux/bin/ld: jpg_dec.c:(.text+0x4dc): undefined reference to `jpeg_finish_decompress'
/usr/lib64/gcc/x86_64-suse-linux/7/../../../../x86_64-suse-linux/bin/ld: jpg_dec.c:(.text+0x4e9): undefined reference to `jpeg_destroy_decompress'

@Hang-Lei-NOAA
Copy link
Contributor

Hang-Lei-NOAA commented Jul 22, 2024 via email

@ShelleyMelchior-NOAA
Copy link
Contributor Author

Thanks @Hang-Lei-NOAA . We do load libjpeg/9c, but it is not in the makefile. I added it to the makefile for the impacted source code.
Here is the make command:

ftn -g -traceback -O3 -auto -convert big_endian -I /apps/ops/para/libs/intel/19.1.3.304/g2/3.5.1/include_4 -o sref_precip.x sref_precip.f /apps/ops/prod/libs/intel/19.1.3.304/ip/4.0.0/lib64/libip_4.a /apps/ops/para/libs/intel/19.1.3.304/w3emc/2.12.0/lib64/libw3emc_4.a /apps/ops/prod/libs/intel/19.1.3.304/bacio/2.4.1/lib/libbacio_4.a  /apps/ops/para/libs/intel/19.1.3.304/sp/2.4.0/lib64/libsp_4.a /apps/ops/para/libs/intel/19.1.3.304/g2/3.5.1/lib64/libg2_4.a /apps/spack/jasper/2.0.25/intel/19.1.3.304/sjib74krrorkyczqpqah4tvewmlnqdx4/lib64/libjasper.a /apps/spack/libpng/1.6.37/intel/19.1.3.304/4ohkronuhlyherusoszzrmur5ewvlwzh/lib/libpng.a /apps/spack/libjpeg/9c/intel/19.1.3.304/jkr3isi257ktoouprwaxcn4twtye747z/lib/libjpeg.a /apps/spack/zlib/1.2.11/intel/19.1.3.304/hjotqkckeoyt6j6tibalwzrlfljcjtdh/lib/libz.a

And here are a few errors that are spawned:

/usr/lib64/gcc/x86_64-suse-linux/7/../../../../x86_64-suse-linux/bin/ld: /apps/spack/libjpeg/9c/intel/19.1.3.304/jkr3isi257ktoouprwaxcn4twtye747z/lib/libjpeg.a(jcapimin.o): in function `jpeg_CreateCompress':
/tmp/root/spack-stage/spack-stage-libjpeg-9c-jkr3isi257ktoouprwaxcn4twtye747z/spack-src/jcapimin.c:68: undefined reference to `__cray_dset_detect'
/usr/lib64/gcc/x86_64-suse-linux/7/../../../../x86_64-suse-linux/bin/ld: /apps/spack/libjpeg/9c/intel/19.1.3.304/jkr3isi257ktoouprwaxcn4twtye747z/lib/libjpeg.a(jcmaster.o): in function `validate_script':
/tmp/root/spack-stage/spack-stage-libjpeg-9c-jkr3isi257ktoouprwaxcn4twtye747z/spack-src/jcmaster.c:209: undefined reference to `__cray_sset_detect'

@Hang-Lei-NOAA
Copy link
Contributor

Hang-Lei-NOAA commented Jul 22, 2024 via email

@ShelleyMelchior-NOAA
Copy link
Contributor Author

Yes! I can try this. One question. I don't see w3emc in the provided installation path. Should I use the para installation for w3emc/2.12.0?

$ ls /lfs/h2/emc/eib/save/hang.lei/forgdit/nco_wcoss2/install2/modulefiles/compiler/intel/19.1.3.304
g2  g2tmpl  hpc-cray-mpich

@Hang-Lei-NOAA
Copy link
Contributor

Hang-Lei-NOAA commented Jul 22, 2024 via email

@ShelleyMelchior-NOAA
Copy link
Contributor Author

Thanks Hang! I tried to recompile using your installation. Here is the make command:

ftn -g -traceback -O3 -auto -convert big_endian -I /lfs/h2/emc/eib/save/hang.lei/forgdit/nco_wcoss2/install2/intel-19.1.3.304/g2/3.5.1/include_4 -o sref_precip.x sref_precip.f /apps/ops/prod/libs/intel/19.1.3.304/ip/4.0.0/lib64/libip_4.a /lfs/h2/emc/eib/save/hang.lei/forgdit/nco_wcoss2/install2/intel-19.1.3.304/w3emc/2.12.0/lib64/libw3emc_4.a /apps/ops/prod/libs/intel/19.1.3.304/bacio/2.4.1/lib/libbacio_4.a  /apps/ops/prod/libs/intel/19.1.3.304/sp/2.4.0/lib64/libsp_4.a /lfs/h2/emc/eib/save/hang.lei/forgdit/nco_wcoss2/install2/intel-19.1.3.304/g2/3.5.1/lib64/libg2_4.a /apps/spack/jasper/2.0.25/intel/19.1.3.304/sjib74krrorkyczqpqah4tvewmlnqdx4/lib64/libjasper.a /apps/spack/libpng/1.6.37/intel/19.1.3.304/4ohkronuhlyherusoszzrmur5ewvlwzh/lib/libpng.a /apps/spack/libjpeg/9c/intel/19.1.3.304/jkr3isi257ktoouprwaxcn4twtye747z/lib/libjpeg.a /apps/spack/zlib/1.2.11/intel/19.1.3.304/hjotqkckeoyt6j6tibalwzrlfljcjtdh/lib/libz.a

But the same errors occur.

I reverted back to the prod install of g2/3.4.5 (and ip/4.0.0 and para installation of w3emc/2.12.0) and EVS compiled correctly.

@Hang-Lei-NOAA
Copy link
Contributor

Hang-Lei-NOAA commented Jul 22, 2024 via email

@edwardhartnett
Copy link
Contributor

Could it be that two different versions of jpeg are involved? That is, g2 was build with one version, and some other library includes a different version?

@Hang-Lei-NOAA
Copy link
Contributor

Hang-Lei-NOAA commented Jul 22, 2024 via email

@ShelleyMelchior-NOAA
Copy link
Contributor Author

Thanks for the help and discussion! EVSv2 will plan for ip/4.0.0 and w3emc/2.12.0, while using g2/3.4.5.

@Hang-Lei-NOAA , can you let me know whenw3emc/2.12.0 has been promoted to a prod installation?

@Hang-Lei-NOAA
Copy link
Contributor

Hang-Lei-NOAA commented Jul 22, 2024 via email

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
3 participants
@edwardhartnett @Hang-Lei-NOAA @ShelleyMelchior-NOAA