Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feature: parallel solve subspace diagonalization in dav_subspace #5549

Open
wants to merge 8 commits into
base: develop
Choose a base branch
from

Conversation

pxlxingliang
Copy link
Collaborator

  • Add function Diago_HS_para() to diago the H/S by genelpa or scalapack. The function will perform 2D block distribution of data, call elpa/scalapack for solution, and collect eigenvectors to rank0.
  • Add a unified function interface pxxxgvx_diag() for calling scalapack to solve the generalized eigenvalue problem with double/complex/float/complex.
  • Add a new INPUT parameter diag_subspace_method, which support to set the subspace diagonalization method in dav_subspace to use elpa or scalapack to solve.

Reminder

  • Have you linked an issue with this pull request?
  • Have you added adequate unit tests and/or case tests for your pull request?
  • Have you noticed possible changes of behavior below or in the linked issue?
  • Have you explained the changes of codes in core modules of ESolver, HSolver, ElecState, Hamilt, Operator or Psi? (ignore if not applicable)

Linked Issue

Fix #5480

Unit Tests and/or Case Tests for my changes

  • A unit test is added for each new feature or bug fix.

What's changed?

  • Example: My changes might affect the performance of the application under certain conditions, and I have tested the impact on various scenarios...

Any changes of core modules? (ignore if not applicable)

  • Example: I have added a new virtual function in the esolver base class in order to ...

- **Default**: 4
- **Default**: 4

### diag_subspace_method
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would suggest using "diag_subspace" and delete "_method"

const int* n, float* A, const int* ia, const int* ja, const int*desca, float* B, const int* ib, const int* jb, const int*descb,
const float* vl, const float* vu, const int* il, const int* iu,
const float* abstol, int* m, int* nz, float* w, const float*orfac, float* Z, const int* iz, const int* jz, const int*descz,
float* work, int* lwork, int*iwork, int*liwork, int* ifail, int*iclustr, float*gap, int* info);
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

at least leave a blank line between two functions

const int diag_subspace_method,
const int block_size)
{
int myrank;
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

initialize the variable


template <typename T>
void Diago_HS_para(
T* h,
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would suggset using clang-format before you pr a new code, here T is not aligned with the following variables

work, lwork,rwork, lrwork, iwork, liwork, ifail, iclustr, gap, info);
}

// post processing for pdsygvx/pzhegvx/pdsygvx/pzhegvx
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

explain the functions of pdsygvx/pzhegvx/pdsygvx/pzhegvx

typename GetTypeReal<T>::type *const ekb,
T *const wfc_2d)
{
int nprow, npcol, myprow, mypcol;
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

initialize one variable for a line

int ndim_global = desc[2];
const char jobz = 'V', range = 'I', uplo = 'U';
const int itype = 1, il = 1, iu = nbands, one = 1;
int M = 0, NZ = 0, lwork = -1, lrwork = -1, liwork = -1, info = 0;
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

one line for a variable

}

// template instantiation
template void pxxxgvx_diag(const int *const desc,
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

explain the functions of the four templates

@mohanchen mohanchen added the Diago Issues related to diagonalizaiton methods label Nov 22, 2024
@haozhihan
Copy link
Collaborator

Diago_HS_para implements parallel solution for the subspace process in dav-subspace. If this is efficient, can we just fix one solution method to reduce the burden of users?

@haozhihan
Copy link
Collaborator

I have a brief insight regarding this PR:

This process involves a transformation of parallel strategy.

  • from basis parallelism to 2D block parallelism
  • from 2D block parallelism to basis parallelism

Can this transformation of parallel strategy be more general?

If the H and S matrices of LCAO are solved by iterative method (like cg, dav, and so on, usually used for plane wave basis), it will also involve almost the same transformation of parallel strategy.

@pxlxingliang
Copy link
Collaborator Author

Diago_HS_para implements parallel solution for the subspace process in dav-subspace. If this is efficient, can we just fix one solution method to reduce the burden of users?

The parallel diagonization is not always more efficient, it is related to the system size, parallel cores, and efficiency of parallel communication, etc. I have done some tests in #5480.

@pxlxingliang
Copy link
Collaborator Author

pxlxingliang commented Nov 22, 2024

I have a brief insight regarding this PR:

This process involves a transformation of parallel strategy.

  • from basis parallelism to 2D block parallelism
  • from 2D block parallelism to basis parallelism

Can this transformation of parallel strategy be more general?

If the H and S matrices of LCAO are solved by iterative method (like cg, dav, and so on, usually used for plane wave basis), it will also involve almost the same transformation of parallel strategy.

The transformation of different 2D block distribution can be realized easily by call the scalapack function Cpigemr2d() (a uniform interface for different data type is here: https://github.com/deepmodeling/abacus-develop/blob/develop/source/module_base/scalapack_connector.h#L158). While the transformation for basis parallelism is strongly related to the self-defined class in ABACUS (like psi?), may be a class function of psi to do the transformation is better.

*/

template <typename T>
void pxxxgvx_diag(const int* const desc,
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Does p stand for "parameter"?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

pxxxgvx is the wrapper of pdsygvx_/pzhegvx_/pdsygvx_/pzhegvx_ in scalapack

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Diago Issues related to diagonalizaiton methods
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Need subspace diagonalization with parallel
4 participants