Intel(R) SHMEM Library (ISHMEM) 1.0.0 doc pages

Signed-off-by: sys_shmem <[email protected]>

.gitignore

@@ -0,0 +1,3 @@
+.*.sw*
+build
+tags

LICENSE

@@ -0,0 +1,30 @@
+Copyright (c) 2023 Intel Corporation.
+
+Redistribution and use in source and binary forms, with or without
+modification, are permitted provided that the following conditions are
+met:
+
+1. Redistributions of source code must retain the above copyright
+notice, this list of conditions and the following disclaimer.
+
+2. Redistributions in binary form must reproduce the above copyright 
+notice, this list of conditions and the following disclaimer in the 
+documentation and/or other materials provided with the distribution.
+
+3. Neither the name of the copyright holder nor the names of its 
+contributors may be used to endorse or promote products derived from 
+this software without specific prior written permission.
+
+THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS 
+“AS IS” AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT 
+LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR 
+A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT 
+HOLDER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, 
+SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT 
+LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, 
+DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY 
+THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT 
+(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE 
+OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+
+SPDX-License-Identifier: BSD-3-Clause

README.md

@@ -0,0 +1,155 @@
+# Intel® SHMEM <!-- omit in toc --> <img align="right" width="100" height="100" src="https://spec.oneapi.io/oneapi-logo-white-scaled.jpg">
+
+[Installation](#installation)&nbsp;&nbsp;&nbsp;|&nbsp;&nbsp;&nbsp;[Usage](#usage)&nbsp;&nbsp;&nbsp;|&nbsp;&nbsp;&nbsp;[Release Notes](RELEASE_NOTES.md)&nbsp;&nbsp;&nbsp;|&nbsp;&nbsp;&nbsp;[Documentation](https://oneapi-src.github.io/ishmem/intro.html)&nbsp;&nbsp;&nbsp;|&nbsp;&nbsp;&nbsp;[How to Contribute](CONTRIBUTING.md)&nbsp;&nbsp;&nbsp;|&nbsp;&nbsp;&nbsp;[License](LICENSE)
+
+Intel® SHMEM provides an efficient implementation of GPU-initiated communication on systems with Intel GPUs.
+
+## Table of Contents <!-- omit in toc -->
+
+- [Prerequisites](#prerequisites)
+- [Installation](#installation)
+- [Usage](#usage)
+  - [Launching Example Application](#launching-example-application)
+- [Additional Resources](#additional-resources)
+  - [OpenSHMEM Specification](#openshmem-spec)
+  - [Specification](#ishmem-spec)
+
+## Prerequisites
+
+- Linux OS
+- Intel® oneAPI DPC++/C++ Compiler 2024.0 or higher.
+
+### SYCL support <!-- omit in toc -->
+Intel® oneAPI DPC++/C++ Compiler with Level Zero support.
+
+To install Level Zero, refer to the instructions in [Intel® Graphics Compute Runtime for oneAPI Level Zero and OpenCL™ Driver repository](https://github.com/intel/compute-runtime/releases) or to the [installation guide](https://dgpu-docs.intel.com/installation-guides/index.html) for oneAPI users.
+
+## Installation
+Intel® SHMEM requires a host SHMEM back-end to be used for host-sided operations support. In particular, it relies on an a collection of extension APIs (`shmemx_heap_create`, `shmemx_heap_preinit`, and `shmemx_heap_postinit`) to coordinate the Intel® SHMEM and OpenSHMEM heaps. We recommend [Sandia OpenSHMEM](https://github.com/Sandia-OpenSHMEM/SOS) for this purpose.
+
+### Building Sandia OpenSHMEM (SOS)
+Download the SOS repo to be configured as a back-end for Intel® SHMEM.
+
+```
+git clone https://github.com/Sandia-OpenSHMEM/SOS.git SOS
+```
+
+Build SOS following instructions below. `FI_HMEM` support in the provider is required for use with Intel® SHMEM. To enable `FI_HMEM` with a supported provider, we recommend a specific set of config flags. Below are two examples for configuring and building SOS with two providers supporting `FI_HMEM`. To configure SOS with the `verbs;ofi_rxm` provider, use the following instructions:
+
+```
+cd SOS
+./autogen.sh
+./configure --prefix=<sos_dir> --with-ofi=<ofi_installation> --enable-pmi-simple --enable-ofi-mr=basic --disable-ofi-inject --enable-ofi-hmem --disable-bounce-buffers --enable-hard-polling
+make -j
+make install
+```
+To configure SOS with the HPE Slingshot provider `cxi`, please use the following instructions:
+```
+cd SOS
+./autogen.sh
+./configure --prefix=<sos_dir> --with-ofi=<ofi_installation> --enable-pmi-simple --enable-ofi-mr=basic --disable-ofi-inject --enable-ofi-hmem --disable-bounce-buffers --enable-ofi-manual-progress --enable-mr-endpoint
+make -j
+make install
+``` 
+Please choose an appropriate PMI configure flag based on the available PMI client library in the system. Please check for further instructions on [SOS Wiki pages](https://github.com/Sandia-OpenSHMEM/SOS/wiki). Optionally, users may also choose to add `--disable-fortran` since fortran interfaces will not be used.
+
+
+### Building Intel® SHMEM
+Check that the SOS build process has successfully created an `<sos_dir>` directory with `include` and `lib` as subdirectories. Please find `shmem.h` and `shmemx.h` in `include`. 
+
+Build Intel® SHMEM using the following instructions:
+
+```
+cd ishmem
+mkdir build
+cd build
+cmake .. -DSHMEM_INSTALL_PREFIX=<sos_dir> -DCMAKE_INSTALL_PREFIX=<ishmem_install_dir>
+make -j
+```
+
+## Usage
+
+### Launching Example Application
+
+Validate that Intel® SHMEM was built correctly by running an example program.
+
+1. Add the library path for SOS to the environment:
+
+```
+export LD_LIBRARY_PATH=<sos_dir>/lib:$LD_LIBRARY_PATH
+```
+
+2. Run the example program or test on an allocated node using a process launcher:
+
+```
+mpiexec.hydra -n 2 -hosts <allocated_node_id> ./scripts/ishmrun ./test/unit/SHMEM/int_get_device
+```
+
+- *Note:* Current supported launchers include: MPI process launchers (i.e. `mpiexec`, `mpiexec.hydra`, `mpirun`, etc.), Slurm (i.e. `srun`, `salloc`, etc.), and PBS (i.e. `qsub`).
+
+- *Note:* Intel® SHMEM execution model requires applications to use a 1:1 mapping between PEs and GPU devices. Attempting to run an application without the ishmrun launch script may result in undefined behavior if this mapping is not maintained.
+  - For further details on the device selection, please see [the ONEAPI_DEVICE_SELECTOR](https://github.com/intel/llvm/blob/sycl/sycl/doc/EnvironmentVariables.md#oneapi_device_selector).
+
+3. Validate the application ran succesfully; example output:
+
+```
+Selected device: Intel(R) Data Center GPU Max 1550
+Selected vendor: Intel(R) Corporation
+Selected device: Intel(R) Data Center GPU Max 1550
+Selected vendor: Intel(R) Corporation
+No errors
+No errors
+```
+
+### Launching Example Application w/ CTest
+
+`ctest` can be used to run Intel® SHMEM tests that are generated at compile-time. To see a list of tests available via `ctest`, run:
+
+```
+ctest -N
+```
+
+To launch a single test, execute:
+
+```
+ctest -R <test_name>
+```
+
+Alternatively, all the tests in a directory (such as `test/unit/SHMEM/`) can be run with the following command:
+
+```
+ctest --test-dir <directory_name>
+```
+
+By default, a passed or failed test can be detected by the output:
+```
+    Start 69: sync-2-gpu
+1/1 Test #69: sync-2-gpu .......................   Passed    2.29 sec
+
+100% tests passed, 0 tests failed out of 1
+```
+
+To have a test's output printed to the console, add either the `--verbose` or `--output-on-failure` flag to the `ctest` command
+
+### Available Scheduler Wrappers for Jobs Run via CTest
+The following values may be assigned to `CTEST_SCHEDULER` at configure-time (ex. `-DCTEST_SCHEDULER=mpi`) to set which scheduler will be used to run tests launched through a call to `ctest`:
+ - srun (default)
+   - Launches CTest jobs on a single node using Slurm's `srun`.
+ - mpi
+   - Uses `mpirun` to launch CTest jobs with the appropriate number of processes.
+ - qsub
+   - Launches CTest jobs on a single node using `qsub`. If this option is being used on a system where a reservation must be made (i.e. via `pbsresnode`) prior to running a test, assign the `JOB_QUEUE` environment variable to the queue associated with your reservation:
+   ```
+   export JOB_QUEUE=<queue>
+   ```
+
+## Additional Resources
+
+### OpenSHMEM Specification
+
+- [OpenSHMEM](http://openshmem.org/site/)
+- [Specification](http://openshmem.org/site/sites/default/site_files/OpenSHMEM-1.5.pdf)
+
+### Intel® SHMEM Specification
+
+- [Intel® SHMEM Specification](https://oneapi-src.github.io/ishmem/intro.html)

docs/_sources/acknowledgements.rst

@@ -0,0 +1,96 @@
+.. _acknowledgements:
+
+----------------
+Acknowledgements
+----------------
+
+Intel acknowledges the OpenSHMEM specification and its many contributors. For
+more details on OpenSHMEM, visit http://openshmem.org.
+
+Some of the Intel® SHMEM documentation and library routines are
+derived from the OpenSHMEM specification.
+The OpenSHMEM specification belongs to Open Source Software Solutions, Inc.
+(OSSS), a nonprofit organization, under an agreement with HPE.
+Permission to copy without fee all or part of this material is granted,
+provided the OSSS notice and the title of this document appear, and notice is
+given that copying is by permission of OSSS.
+
+All software downloads produced by the OpenSHMEM project on the OpenSHMEM
+website are provided under the following BSD 3-clause license.
+
+Copyright (c) 2016 - 2020
+Stony Brook University
+
+Copyright (c) 2015 - 2020
+Los Alamos National Security, LLC.
+
+Copyright (c) 2011 - 2016
+University of Houston System and UT-Battelle, LLC.
+
+Copyright (c) 2009 - 2016
+Silicon Graphics International Corp. SHMEM is copyrighted by Silicon Graphics International Corp. (SGI) The OpenSHMEM API (shmem) is released by Open Source Software Solutions, Inc., under an agreement with Silicon Graphics International Corp. (SGI).
+
+Copyright (c) 2016-2020
+Open Source Software Solutions (OSSS), Inc.
+
+Copyright (c) 2016-2017
+Hewlett Packard Enterprise Development LP.
+
+Redistribution and use in source and binary forms, with or without
+modification, are permitted provided that the following conditions are met:
+
+* Redistributions of source code must retain the above copyright notice, this
+  list of conditions and the following disclaimers.
+
+* Redistributions in binary form must reproduce the above copyright notice,
+  this list of conditions and the following disclaimer in the documentation
+  and/or other materials provided with the distribution.
+
+* Neither the name of the University of Houston System, UT-Battelle, LLC. nor
+  the names of its contributors may be used to endorse or promote products
+  derived from this software without specific prior written permission.
+
+* Neither the name of Los Alamos National Security, LLC, Los Alamos National
+  Laboratory, LANL, the U.S. Government, nor the names of its contributors may
+  be used to endorse or promote products derived from this software without
+  specific prior written permission.
+
+THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND
+ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED
+WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE
+DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE
+FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
+DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR
+SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER
+CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY,
+OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
+OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+
+---------------------
+Notices & Disclaimers
+---------------------
+
+|Intel_Disclaimers|
+
+.. |Intel_Disclaimers| raw:: html
+
+    <a href="https://www.intel.com/LegalNoticesAndDisclaimers" target="_blank">Legal Notices and Disclaimers</a>
+
+Intel technologies may require enabled hardware, software or service activation.
+
+No product or component can be absolutely secure.
+
+Your costs and results may vary.
+
+© Intel Corporation.  Intel, the Intel logo, and other Intel marks are trademarks of Intel Corporation or its subsidiaries.  Other names and brands may be claimed as the property of others.
+
+---------
+Copyright
+---------
+
+|Creative_Commons_BY_4|
+
+.. |Creative_Commons_BY_4| raw:: html
+
+    <p xmlns:cc="http://creativecommons.org/ns#" xmlns:dct="http://purl.org/dc/terms/">The <a property="dct:title" rel="cc:attributionURL" href="https://oneapi-src.github.io/ishmem">Intel® SHMEM specification</a> © 2023 is licensed under <a href="http://creativecommons.org/licenses/by/4.0/?ref=chooser-v1" target="_blank" rel="license noopener noreferrer" style="display:inline-block;">CC BY 4.0<img style="height:22px!important;margin-left:3px;vertical-align:text-bottom;" src="https://mirrors.creativecommons.org/presskit/icons/cc.svg?ref=chooser-v1"><img style="height:22px!important;margin-left:3px;vertical-align:text-bottom;" src="https://mirrors.creativecommons.org/presskit/icons/by.svg?ref=chooser-v1"></a></p>
+