DO NOT SQUASH Add PyOP2 to Firedrake repository

[JDBetteridge]

Description

The big merge!

Closes #3877

Key changes introduced by this PR

• PyOP2 is ingested into this repository (TSFC will be done later)
• Existing tests are moved into a tests/firedrake subdirectory
• Adds a pyproject.toml
• pip install git+https://github.com/firedrakeproject/firedrake.git becomes a viable install method (subject to setting a number of environment variables). This ultimately will be our path to a Firedrake wheel.
• Removes firedrake_configuration. The information is all accessible by inspecting PETSc variables.

Outstanding tasks

• CI passing
• Make sure that firedrake-install, firedrake-update, firedrake-status and firedrake-clean pass locally (these are not fully tested in the CI).
• git rebase -i to try and keep a clean history.
• Fix firedrake-zenodo as PyOP2 repo is going away.
• Check manual for possible inaccuracies about install and Zenodo.

Future work

• Also include TSFC in Firedrake repo (and put GEM and FInAT into FIAT).
• pip install libsupermesh (nearly ready to go: JDBetteridge/pip install libsupermesh#10)

[miguelcoolchips]

I tested installing Firedrake using this branch on my Mac M1. I installed PETSc independently and set the PETSC_DIR and PETSC_ARCH variables. Then I ran

CXXFLAGS="-std=c++11" pip install -v ./firedrake/

I needed CXXFLAGS="-std=c++11" for installing tinyasm. In addition, I edited the setup.py

diff --git a/setup.py b/setup.py
index c0192b703..73418a6c6 100644
--- a/setup.py
+++ b/setup.py
@@ -176,7 +176,7 @@ spatialindex_ = ExternalDependency(
 supermesh_dir = Path(supermesh.__path__._path[0]).absolute()
 supermesh_so = next(supermesh_dir.glob('*.so'))
 supermesh_ = ExternalDependency(
-    libraries=[f":{supermesh_so.name!s}"],
+    libraries=[],
     include_dirs=[str(supermesh_dir.joinpath("include"))],
     library_dirs=[str(supermesh_dir)],
     runtime_library_dirs=[os.path.join(site.getsitepackages()[0], "supermesh")]

Otherwise I would get this error

  mpicc -bundle -undefined dynamic_lookup -L/opt/homebrew/opt/ncurses/lib -L/opt/homebrew/opt/readline/lib -L/opt/homebrew/opt/readline/lib -L/Users/miguel/.pyenv/versions/3.11.8/lib -Wl,-rpath,/Users/miguel/.pyenv/versions/3.11.8/lib -L/opt/homebrew/opt/gcc/lib/gcc/13 -Wl,-rpath,/opt/homebrew/opt/gcc/lib/gcc/13 -lgcc_s -L/opt/homebrew/lib -Wl,-rpath,/opt/homebrew/lib -L/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/lib -L/opt/homebrew/opt/ncurses/lib -L/opt/homebrew/opt/readline/lib -L/opt/homebrew/opt/readline/lib -L/Users/miguel/.pyenv/versions/3.11.8/lib -Wl,-rpath,/Users/miguel/.pyenv/versions/3.11.8/lib -L/opt/homebrew/opt/gcc/lib/gcc/13 -Wl,-rpath,/opt/homebrew/opt/gcc/lib/gcc/13 -lgcc_s -L/opt/homebrew/lib -Wl,-rpath,/opt/homebrew/lib -L/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/lib -L/opt/homebrew/opt/gcc/lib/gcc/13 -Wl,-rpath,/opt/homebrew/opt/gcc/lib/gcc/13 -lgcc_s build/temp.macosx-12.4-arm64-cpython-311/firedrake/cython/supermeshimpl.o -L/Users/miguel/repos/petsc-firedrake-fork/default/lib -L/private/var/folders/s3/9xkzjvy531138v895vytyfh40000gn/T/pip-build-env-kuy2zid3/overlay/lib/python3.11/site-packages/supermesh -L/Users/miguel/.pyenv/versions/3.11.8/lib -Wl,-rpath,/Users/miguel/repos/petsc-firedrake-fork/default/lib -Wl,-rpath,/Users/miguel/repos/firedrake-pip-install-venv/lib/python3.11/site-packages/supermesh -lpetsc -l:supermesh.cpython-311-darwin.so -o build/lib.macosx-12.4-arm64-cpython-311/firedrake/cython/supermeshimpl.cpython-311-darwin.so
  ld: library not found for -l:supermesh.cpython-311-darwin.so
  clang: error: linker command failed with exit code 1 (use -v to see invocation)
  error: command '/opt/homebrew/bin/mpicc' failed with exit code 1
  error: subprocess-exited-with-error

  × Building wheel for firedrake (pyproject.toml) did not run successfully.
  │ exit code: 1
  ╰─> See above for output.

  note: This error originates from a subprocess, and is likely not a problem with pip.
  full command: /Users/miguel/repos/firedrake-pip-install-venv/bin/python3 /Users/miguel/repos/firedrake-pip-install-venv/lib/python3.11/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py build_wheel /var/folders/s3/9xkzjvy531138v895vytyfh40000gn/T/tmpkrmci25p
  cwd: /Users/miguel/repos/firedrake
  Building wheel for firedrake (pyproject.toml) ... error
  ERROR: Failed building wheel for firedrake
Failed to build firedrake
ERROR: Could not build wheels for firedrake, which is required to install pyproject.toml-based projects

although when I run the tests, libsupermesh cannot be found

ImportError while importing test module '/Users/miguel/repos/firedrake/tests/firedrake/supermesh/test_assemble_mixed_mass_matrix.py'.
Hint: make sure your test modules/packages have valid Python names.
Traceback:
../../.pyenv/versions/3.11.8/lib/python3.11/importlib/__init__.py:126: in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
tests/firedrake/supermesh/test_assemble_mixed_mass_matrix.py:3: in <module>
    from firedrake.supermeshing import *
../firedrake-pip-install-venv/lib/python3.11/site-packages/firedrake/supermeshing.py:5: in <module>
    from firedrake.cython.supermeshimpl import assemble_mixed_mass_matrix as ammm, intersection_finder
E   ImportError: dlopen(/Users/miguel/repos/firedrake-pip-install-venv/lib/python3.11/site-packages/firedrake/cython/supermeshimpl.cpython-311-darwin.so, 0x0002): symbol not found in flat namespace '_libsupermesh_tree_intersection_finder_get_output'

[connorjward]

Thanks @miguelcoolchips for trying this out! I am hoping to get access to a Mac to test this out on today (and in the near future we should have Mac CI runners as well).

[dham]

Think about contact.

[connorjward]

To not violate single source of truth we should somehow link this to https://github.com/firedrakeproject/firedrake/blob/master/docs/source/team.ini

Since we're not yet publishing this to PyPI I think this can be handled at a later point and just removed from this PR.

[connorjward]

Though the team.ini file distinguishes between further things like "active" and "inactive" core team members as well as having institutions and links. The authors/maintainers field in the pyproject.toml can only store names and emails.

It might be simplest just to have all of the current core developers listed as maintainers and to not have any authors specified.