Roadmap 1.0 and beyond

[fieker]

Deadlines

• Oscar book needs to be ready 31 January 2024 (or maybe even earlier)
• anything relevant for the book (and examples in it) should be settled before
• the proposal is submitted 1 May 2024, so 1.0 should be out before that
• some improvements can be done afterwards but still should be done ASAP so that referees will see them

Plan

Kaiserslautern: two regular meetings:

• triage: Wednesday 11:30-12:20 (planning; review and resolve issues and PRs on Oscar, Nemo, AA, ...)
• code sprint: TBD (possibly Wednesday or Thursday 13:30-16:00)
• it will be possible to join either via Zoom

Things critical for the book (end of January!!)

Examples in the book should match what we get in 1.0 even though 1.0 won't be finished in time for the book deadline. That means a few items should be done before; and if we can't do them, then we may have to "lie" in the book in a few examples (i.e. edit output to match what it is supposed to be in 1.0, not what it is right now)

• NEW HackMD for renaming
• finalize type and function names (at least any user facing)
  • relevant issues:
    • The big renaming... #1826
    • More things we should perhaps rename Nemocas/AbstractAlgebra.jl#1278
    • More things to rename (?) Nemocas/Nemo.jl#1394
    • Should we rename dense_matrix_type to matrix_type ? Nemocas/AbstractAlgebra.jl#1306
    • HackMD
  • Abs.. in type names can indicate "abstract" or "absolute". It should be only one
    • there are many abstract types without the Abs, what is the plan?
      • abstract type ProjSpc, but abstract type AbsAffineAlgebraicSet
      • abstract type ModuleFP, abstract type AbstractFreeModElem
      • abstract type OscarMap
      • AbstractLat
    • concrete types AbstractVariety etc.
  • acb, arb and friends and relatives in Nemo need proper names
  • CoveringMorphism -> CoveringMor? rules for Mor(phism)?
  • Inv and Invar in type names
  • _dec vs Dec
  • Mod and Module
  • FPModule vs ModuleFP
  • GrpAb -> AbGroup? and other Grp in Hecke and below
• improve printing for groups Print group generators in detailed printing #2878
• printing of modules etc., Adjust printing of modules and related stuff to follow OSCAR printing guidelines #2891 (@jankoboehm @wdecker @HechtiDerLachs @simonbrandhorst )
• sub/quo/etc. dispatches should be consistent in the return values (with/without map, order, etc.)

Things critical for 1.0 (e.g. to avoid breaking semver in 1.1)

• Hecke and Oscars GModules and their relation
• sanitize exported vs non-exported stuff
  • if you are aware of things that are not exported but should be (or vice-versa), please report it
  • never export a function starting with an underscore (if user should use it, give it a proper name)
  • if in doubt, don't export it (it is easy to add export; but hard to remove them again; people will expect that anything exported is official and here to stay)
• remove large code blocks in comments?
• turn more structs into immutable ones?
• various quo and residue_ring, residue_field constructors to return proper fields, not generic ones
• Hecke CrvEll and relation to PlaceCrv and Scheme
• unify automorphism_group_generators and friends
  • automorphism_list
  • automorphism_group
  • automorphisms
• revisit Lie theory
  • come up with a plan for future extensions now so that things will at least have a chance of fitting together
  • talk with Uli T, Lars G. and others in Aachen, Laura, ...
  • Dynkin diagrams as graphs, visualization, ...
  • (applications in many places, e.g. in F-Theory, ....)
• representation theory
  • finish the G-module stuff, migrate it from Experimental, ...
  • e.g. extension of scalars etc. etc.
• PcGroup vs FpGroup and their relation and definition (see also PcGroups as FPGroups #952, Rename FPGroups resp. clarify documentation #1139, Add SubPcGroup type #2672)
• graphs (Lars K)
• better mpoly interface (dynamic polynomial interface for universal polynomials)
  • see e.g. https://github.com/JuliaAlgebra/DynamicPolynomials.jl:
  • promote UniversalPolynomialRing more, make it easy to find/use
  • "nicer" substitution syntax, e.g.:
    • subs(p, y=>x^2)
    • p((x, y)=>(y, x))
• lin alg interface: revise them
  • (Tommy started a document on this)
  • (more) functions for efficient base operations (addrow, swaprow, etc. etc.)
  • standard names for certain operations that cover all cases
  • consistent behavior
  • decide things like result as matrix vs. result as vector space object
  • sparse stuff....
• Streamline methods for direct_sum and tensor_product throughout Oscar  #3059
• ... more to be added here

Important for 1.0 in the sense of "should be there to look good", but not in a technical sense (not breaking semver)

These are things we want for the grant application reviewers, but technically they are separate from 1.0

• AbstractAlgebra documentation:
  • mention Oscar
  • mention limitations
• precompile on installation or install via Aaruni's toolkit
• documentation
  • QQbar
  • ... in general, improve, proof read, criticize...
  • state definitions in more places
  • ideally every function docstring has at least one example
  • but not everything needs a docstring (e.g. no need for 1 million + docstrings)
  • ring arithmetic etc. is better showcased (if at all) in a docstring for the ring type and/or in the manual section for that ring; or even not at all
• Oscar web-site
  • news
  • publications
  • tutorials -> Martin Bies
  • ref to book
  • identify existing data sets within OSCAR and bring their metadata to the MaRDI portal

Important but could be deferred to 1.x:

• POSets
• some more vprint, hassert throughout the system
  • maybe resolve Move Assertions.jl to AbstractAlgebra thofma/Hecke.jl#1264
• document and use some of the exception types
  • e.g. Hecke provides BadPrime
  • also use those provided by Julia (e.g. InexactError)
• replace some (many) MapFromFunc by proper types?
• CalciumQQBar vs qqbar vs. Nemo.QQBar
  • decide how to expose this (e.g. algebraic_closure(QQ))
  • rename?
  • AlgClosure(QQ) as QQbar and some support and docu for it
  • provide conversion between QQBar elements and genuine number fields
• nmod_poly_factor and friends: should be useless and unreachable
• Nemo.GaussianIntegers and Nemo.GaussianRationals
  • should we just get rid of them? (they don't fit into the number field hierarchy)
• serialization immediate in memory: test case (parallelization)
• serialization:
  • group theory types missing
  • in some cases that's caused by missing serialization features
  • allow "easy" attribute serialization... ? but warn about dangers
• docstrings for more (all?) types, at least concrete types
  • ideally including examples showing how to produce instances of that type

[YueRen]

Regarding combinatorics in OSCAR, it would be nice if the following types would be more "consistent" relative to itself and each other:

• graphs
• polyhedra and polyhedral complexes
• cones and polyhedral fans
• subdivision of points

With "consistent" I vaguely mean the following (will update if something new springs to my mind):

• polyhedral_fan and subdivision_of_points are incidence matrix last, while polyhedral_complex is incidence matrix first
• all constructors above are named after their types, while the constructor that creates a graph from an incidence matrix is called graph_from_adjacency_matrix
• the commands nv and ne should be renamed to n_vertices and n_edges to be more consistent with n_vertices, n_rays, etc.
• Both PolyhedralComplex and PolyhedralFan have rays_modulo_lineality, but the corresponding function for vertices in PolyhedralComplex and Polyhedron is called minimal_faces and not vertices_modulo_lineality

[fieker]

We should also look at some foundational discussions in Nemo/ AA:

• compose Update argument order in compose Nemocas/Nemo.jl#1521
• mutability fix aliasing in matrix algebra constructor Nemocas/AbstractAlgebra.jl#692, Document allowed mutation behaviour for Nemo, especially matrix inplace operations Nemocas/Nemo.jl#278, Memory allocation in diagonal_matrix Nemocas/AbstractAlgebra.jl#1108

[lkastner]

* all constructors above are named after their types, while the constructor that creates a graph from an incidence matrix is called [`graph_from_adjacency_matrix`](https://docs.oscar-system.org/stable/Combinatorics/graphs/#graph_from_adjacency_matrix)

There are two "incidence matrices" for a graph, one encodes which edge contains which vertices, and the other encodes which vertices are adjacent. This command takes the second kind, which still has the input type IncidenceMatrix, nevertheless in this case it encodes the adjacency.

* the commands [`nv`](https://docs.oscar-system.org/stable/Combinatorics/graphs/#nv-Union%7BTuple%7BGraph%7BT%7D%7D,%20Tuple%7BT%7D%7D%20where%20T%3C:Union%7BDirected,%20Undirected%7D) and [`ne`](https://docs.oscar-system.org/stable/Combinatorics/graphs/#ne-Union%7BTuple%7BGraph%7BT%7D%7D,%20Tuple%7BT%7D%7D%20where%20T%3C:Union%7BDirected,%20Undirected%7D) should be renamed to `n_vertices` and `n_edges`  to be more consistent with `n_vertices`, `n_rays`, etc.

We are not trying to be consistent with these commands, we are trying to be consistent with what is in Graphs.jl. Furthermore @lgoettgens already added nedges and nvertices for graphs:

julia> g = complete_graph(4)
Undirected graph with 4 nodes and the following edges:
(2, 1)(3, 1)(3, 2)(4, 1)(4, 2)(4, 3)

julia> nedges(g)
6

julia> nvertices(g)
4

Once there is GraphsBase.jl we will use this to derive our graphs from it. The commands n_vertices, n_edges and n_rays seem not to exist.

* Both `PolyhedralComplex` and `PolyhedralFan` have [`rays_modulo_lineality`](https://docs.oscar-system.org/stable/PolyhedralGeometry/polyhedral_complexes/#rays_modulo_lineality-Union%7BTuple%7BPolyhedralComplex%7BT%7D%7D,%20Tuple%7BT%7D%7D%20where%20T%3C:Union%7BFloat64,%20FieldElem%7D), but the corresponding function for vertices in `PolyhedralComplex` and `Polyhedron` is called [`minimal_faces`](https://docs.oscar-system.org/stable/PolyhedralGeometry/polyhedral_complexes/#minimal_faces-Union%7BTuple%7BPolyhedralComplex%7BT%7D%7D,%20Tuple%7BT%7D%7D%20where%20T%3C:Union%7BFloat64,%20FieldElem%7D) and not `vertices_modulo_lineality`

I'd rather rename rays_modulo_lineality to something more sensible, but minimal_faces did not make sense for a fan, since this would consist of only the lineality space.

[fieker]

sort out some of the mutating operators, esp for generic matrices. They are not really inplace... und unneccessarily so

[lkastner]

Would it make sense to have a milestone for 1.0?

[YueRen]

And would it make sense to have a coding sprint before the release of 1.0?

Regarding the names, I personally like the *_modulo_lineality names, since I find it descriptive and both gfan and polymake do things modulo lineality space by default (i.e., $F->RAYS will not return something empty just because $F has lineality).

[lgoettgens]

finalize type and function names (at least any user facing)

What about more or less weirdly types in Hecke that currently do not fit into our naming scheme, e.g. NfRelNS, GrpAbFinGen or AlgMat?

[YueRen]

The plan mentions two meetings in Kaiserslautern, what are the dates?

[micjoswig]

For some weird reason, I did not see this discussion before. Let me comment briefly, even if 1.0 is complete now.

With "consistent" I vaguely mean the following (will update if something new springs to my mind):

* [`polyhedral_fan`](https://docs.oscar-system.org/stable/PolyhedralGeometry/fans/#polyhedral_fan) and [`subdivision_of_points`](https://docs.oscar-system.org/stable/PolyhedralGeometry/subdivisions_of_points/#subdivision_of_points-Union%7BTuple%7BT%7D,%20Tuple%7BType%7BT%7D,%20Union%7BMatElem,%20AbstractMatrix%7D,%20IncidenceMatrix%7D%7D%20where%20T%3C:Union%7BFloat64,%20FieldElem%7D) are incidence matrix **last**, while [`polyhedral_complex` ](https://docs.oscar-system.org/stable/PolyhedralGeometry/polyhedral_complexes/#polyhedral_complex) is incidence matrix **first**

That's probably unresolved, and that's bad.

* all constructors above are named after their types, while the constructor that creates a graph from an incidence matrix is called [`graph_from_adjacency_matrix`](https://docs.oscar-system.org/stable/Combinatorics/graphs/#graph_from_adjacency_matrix)

This function takes any matrix and interprets the thing as an adjacency matrix; so that's correct. Adjacency refers to node vs node, whereas the incidence matrix of a graph is node versus edge. In particular, the former is always square, whereas the latter is not, in general. The data type IncidenceMatrix is appropriate for both. Sorry if this is confusing. I do not know of a good way to avoid this.

* the commands [`nv`](https://docs.oscar-system.org/stable/Combinatorics/graphs/#nv-Union%7BTuple%7BGraph%7BT%7D%7D,%20Tuple%7BT%7D%7D%20where%20T%3C:Union%7BDirected,%20Undirected%7D) and [`ne`](https://docs.oscar-system.org/stable/Combinatorics/graphs/#ne-Union%7BTuple%7BGraph%7BT%7D%7D,%20Tuple%7BT%7D%7D%20where%20T%3C:Union%7BDirected,%20Undirected%7D) should be renamed to `n_vertices` and `n_edges`  to be more consistent with `n_vertices`, `n_rays`, etc.

Done. But nv and ne are kept for graphs for compatibility with standard graph packages in Julia.

* Both `PolyhedralComplex` and `PolyhedralFan` have [`rays_modulo_lineality`](https://docs.oscar-system.org/stable/PolyhedralGeometry/polyhedral_complexes/#rays_modulo_lineality-Union%7BTuple%7BPolyhedralComplex%7BT%7D%7D,%20Tuple%7BT%7D%7D%20where%20T%3C:Union%7BFloat64,%20FieldElem%7D), but the corresponding function for vertices in `PolyhedralComplex` and `Polyhedron` is called [`minimal_faces`](https://docs.oscar-system.org/stable/PolyhedralGeometry/polyhedral_complexes/#minimal_faces-Union%7BTuple%7BPolyhedralComplex%7BT%7D%7D,%20Tuple%7BT%7D%7D%20where%20T%3C:Union%7BFloat64,%20FieldElem%7D) and not `vertices_modulo_lineality`

The term "minimal face" is standard in optimization textbooks. Hence it is chosen here. This is in line with the global decision to pick function names according to textbooks.