Reimplement rendering

[ElementW]

Right now rendering is slow and suffers from high overhead and GPU synchronization issues. In the light of more performant rendering APIs and imrovement of existing ones (AZDO on OpenGL), the whole rendering backend should be reimplemented from scratch based on more modern design decisions, focusing on batching geometry, minimizing state changes, and better resource management.

Ideally, rendering should work with Vulkan as well as OpenGL, although Vulkan is not a target right now and ought to be used as design guide rather than actual implementation.

The major proposed changes are:

• Switching from direct API calls to graphics command collection
  • This enables rendering reordering in order to both accelerate opaque rendering by allowing early fragment discard, and make rendering transparent/translucent objects possible
• Clearly defining in which direction(s) render planning is done: can individual entities render themselves and/or should a Renderer do it?

[ElementW]

As with most changes in existing codebases, we'll have to decide on what names to use. Here I'm splitting up things in resource management to distinguish between data on the CPU and data on the GPU.

For example, right now we have Texture representing a GPU bitmap, and Mesh both being a std::vector<Vector3f> vertices and a int handle (pointing to a GL VAO, currently the reference to the associated VBO is lost & resources leaked).

Proposed Radix (R∃) wording; note the Radix names are API agnostic (i.e. interface names) and underlying implementation may use API-specific features and a different name:

OpenGL	Vulkan	R∃ CPU	R∃ GPU
Texture	Image (+Sampler)	Bitmap	Texture
ARRAY "VBO" Buffer	VERTEX_BUFFER_BIT Buffer	Mesh	VertexBuffer
ELEMENT_ARRAY "IBO" Buffer	INDEX_BUFFER_BIT Buffer	Mesh	IndexBuffer
"VAO" VertexAttribArray	VertexInputBindingDescription + CmdBind{Index,Vertex}Buffers	Mesh	∃?
"UBO" UniformBlock	UNIFORM_BUFFER_BIT Buffer	-	UniformBuffer
FRAMEBUFFER "FBO" Framebuffer	Framebuffer	-	FrameBuffer
-	CommandBuffer	CommandBuffer	-

"∃?" = should it exist as class?

[hhirsch]

@ElementW I know that framebuffer is a noun and correctly written like you wrote it. What do you think about that we still call it FrameBuffer for consistency?

[ElementW]

@hhirsch Good point. Changed.

[wow2006]

This a good guide Doom III BFG
and Renderer

[kungfooman]

This project would be way more interesting if it could be used as a collection of useful gamedev components. But using developers time and energy to basically just reimplement just another game engine is kinda... wasteful... unless the goal is simply to learn it.

E.g. why not just use the Godot renderer and their bullet physics? Then all of their goodies can be reused, e.g. proper standardized 3D formats like glTF 2.0 for maps/models/skeletal-animations/morphs which comes with Khronos Blender importer/exporter and many thirdparty tools which also support it.

[hhirsch]

@kungfooman I agree with bullet physics being better than a custom made physics engine. Dorian has replaced our own physics engine with bullet a while ago.

I also agree that the way we work is not a fast way. If speed was my main concern I would've discarded the whole source the project is based on when I've started and used one of the stock engines. My intent when I adopted the project was to learn something about how games are built and to save a game that was no longer being developed. An interesting aspect about it was that a lot of things where implemented from ground up without using external libraries.

I do agree with splitting the project into smaller modules. A while ago the game and engine where the same thing. So Dorian already gave the project a big nudge in the right direction by splitting the game and engine and he did the same for the game entities. And I hope we can do the same with the renderer, soon (only once we are back into release cycles).

As for Godot it did not come up yet. Our renderer architecture would not support it atm. However I think we'll get smarter about it by discussion. Just know that the way it works isn't that somebody comes up with a good idea and three people from the team start working on it immediately. Usually you have to implement it yourself and you will likely find one or two people that are motivated to pull your branch and help to test it and review your PRs. However you might be able to get more devs on board with this as it is a neat idea.

Thank you for caring enough to contribute in the discussion. I hope my reply did not discourage you.

[ElementW]

Dorian already gave the project a big nudge in the right direction

but also somewhat "left" the project in a state that is not really working, with some refactoring only half-finished. Also while I integrated Bullet I completely broke the portal physics, a thing I was working on but eventually put on hold because of other priorities, university not helping me have more time on the project.

Fortunately I am still here and might come back to work on glPortal, as my attention span is sort of rotating in a round-robin fashion over the set of projects I can work on, and it's been a while since I've done game engine-related stuff.

As for the Godot renderer, I'm definitely not against using it, however I don't know if it is available as a "stand-alone" rendering package without pulling the whole Godot engine in, and there's another concern, which also arises with other bring-your-own-engine renderers like bgfx; how well will it Think With Portals™? If getting portal rendering working is an utter pain or has prohibitively slow performance I'm afraid we'll have to go for our own renderer.

In addition, while it's been a while since I stopped brainstorming on that, I yet have to voice myself as to how I think rendering should be structured, and what exact role Renderers should fulfill: do the entity instances draw themselves through them? Do they do all the rendering themselves from the entities to display, requiring a renderer per entity type (in turn limiting render effect composition)?
I'm increasingly thinking about using and well defining a scene graph for the game to work around some limitations. Being tied to the max 2/3-level rendering tree of Source/Quake-like engines (visleaves → entities (→ particle systems)) might not be the best choice.