allocAppendable() and getCapacity(). #284

dsm9000 · 2023-08-15T03:51:08Z

Appendability metadata, currently for large-sized allocs only. All large allocs are now considered appendable.
allocAppendable(): requests an appendable block of given size and saves the used capacity in the extent metadata.
getCapacity(): computes the capacity of a memory segment per the rules of https://dlang.org/spec/arrays.html#capacity-reserve .
realloc(): modified to behave correctly on appendable allocs.
Tests.

deadalnix · 2023-08-15T12:17:54Z

sdlib/d/gc/extent.d

+			void* finalizer; // Reserved, not used currently
+			ubyte[47] _unused;


These are not useful.

deadalnix

It seems like there is a lot of extraneous content in this that is not justified by how things are used.

There is a serious lack of tests. This obviously would inform on the previous point while also ensuring the details of what remains are right.

It would also be great if we could avoid opening numerous PR without addressing what can be addressed in the other ones.

deadalnix · 2023-08-15T12:27:50Z

sdlib/d/gc/extent.d

+		// Metadata for non-slab (large) size classes
+		struct large {
+			ulong allocSize; // Actual alloc size, stored when required
+			bool canAppend; // Is the alloc appendable?


This does not seems to be useful either, because i can't think of a case where allocSize is going to be zero.

If you're saying that allocSize must always store the alloc size, then canAppend is required to represent appendability. (Even before we have finalizer and it will be necessary to store allocSize when finalization but not appendability is in use for a given alloc.)

You can just store zero.

deadalnix · 2023-08-15T12:28:02Z

sdlib/d/gc/extent.d

-	Bitmap!512 _slabData = void;
+	union _meta {
+		// Slab occupancy (constrained by freeSlots, so usable for all classes)
+		Bitmap!512 slabOccupy;


Why change a perfectly good name?

deadalnix · 2023-08-15T12:28:27Z

sdlib/d/gc/extent.d

+		Bitmap!512 slabOccupy;
+
+		// Metadata for non-slab (large) size classes
+		struct large {


Types are Capitalized.

deadalnix · 2023-08-15T12:28:41Z

sdlib/d/gc/extent.d

@@ -99,7 +99,20 @@ private:
 	Links _links;

 	import d.gc.bitmap;
-	Bitmap!512 _slabData = void;
+	union _meta {


This does not follow the naming convention.

deadalnix · 2023-08-15T12:29:41Z

sdlib/d/gc/extent.d

+	void clearLargeMeta() {
+		meta.large.allocSize = 0;
+		meta.large.canAppend = false;
+	}


After cleaning up the large data structure, this will end up being only one assignement, and it has one call site, so it can go.

deadalnix · 2023-08-15T12:38:36Z

sdlib/d/gc/tcache.d

+			auto ptr = arena.allocLarge(emap, aSize, false);
+			auto pd = getPageDescriptor(ptr);
+			pd.extent.allocSize = 0;
+			pd.extent.appendable = true;


And because it's not 0, we don't need this.

deadalnix · 2023-08-15T12:43:16Z

sdlib/d/gc/extent.d

+
+	@property
+	ref bool appendable() {
+		assert(!isSlab(), "appendable accessed on slab!");


There are enough !isSlab all over to call for an isLarge predicate. Also, there is slab feature section int here, so why isn't there a large feature section now that we are adding features for large extents?

deadalnix · 2023-08-15T12:44:43Z

sdlib/d/gc/tcache.d

+		// Clear metadata:
+		if (!pd.isSlab())
+			pd.extent.clearLargeMeta();


It's unclear to me why this matters...

deadalnix · 2023-08-15T12:57:54Z

sdlib/d/gc/tcache.d

+		// Transfer metadata (currently only support large allocs) :
+		auto pdNew = getPageDescriptor(newPtr);
+		if (!pd.isSlab() && !pdNew.isSlab())
+			pd.extent.copyLargeMeta(pdNew.extent);


This must be wrong, as the copied capacity may not even fit in the new allocation's size!

deadalnix · 2023-08-15T13:07:09Z

sdlib/d/gc/tcache.d

+	// Determine whether given alloc is appendable
+	bool is_appendable(void* ptr) {
+		if (ptr is null)
+			return false;
+
+		auto pd = getPageDescriptor(ptr);
+		// Not supports slab allocs yet
+		if (pd.isSlab())
+			return false;
+
+		return pd.extent.appendable;
+	}
+
+	// Get the current fill of an appendable alloc.
+	// If the alloc is not appendable, returns 0.
+	ulong get_appendable_fill(void* ptr) {
+		if (ptr is null)
+			return 0;
+
+		auto pd = getPageDescriptor(ptr);
+		// Not supports slab allocs yet
+		if (pd.isSlab() || !pd.extent.appendable)
+			return 0;
+
+		return pd.extent.allocSize;
+	}
+
+	// Get the current free space of an appendable alloc.
+	// If the alloc is not appendable, returns 0.
+	ulong get_appendable_free_space(void* ptr) {
+		if (ptr is null)
+			return 0;
+
+		auto pd = getPageDescriptor(ptr);
+		// Not supports slab allocs yet
+		if (pd.isSlab() || !pd.extent.appendable)
+			return 0;
+
+		return cast(ulong) pd.extent.size - pd.extent.allocSize;
+	}
+
+	// Change the current fill of an appendable alloc.
+	// If the alloc is not appendable, or the requested
+	// fill exceeds the available space, returns false.
+	bool set_appendable_fill(void* ptr, ulong fill) {
+		if (ptr is null)
+			return false;
+
+		auto pd = getPageDescriptor(ptr);
+		// Not supports slab allocs yet
+		if (pd.isSlab())
+			return false;
+
+		if (fill <= pd.extent.size) {
+			pd.extent.allocSize = fill;
+			return true;
+		}
+
+		return false;
+	}


This API has numerous problems, a number of which would have easily been detected if we had tests for it. For instance, what happens when it is called on a pointer that isn't managed by the GC? Or what about interior pointers?

Consider:

uint[] slice = (new uint[123])[0 .. 123]; slice = slice[10 .. $]; slice ~= 42;

If there is room, there is no reason for this to reallocate.

Having tests that exercise the API to achieve this kind of result would tell you if the current API is the right one or not. I predict there will be superfluous methods in there, and that the remaining one will have to be amended substantially.

I MR'd this prematurely, apologies.

deadalnix · 2023-08-15T13:37:20Z

Also title doesn't really match what this does, and there is no description whatsoever...

…size (todo)

deadalnix

If this is going to make progress, this must start from the use case. Building a bunch of feature really isn't improving the code if these feature do not add value in some way. In fact, it makes the code bigger, more complex, probably buggier or slower.

deadalnix · 2023-08-16T13:11:16Z

sdlib/d/gc/extent.d

+		Bitmap!512 _slabData;
+
+		// Metadata for non-slab (large) size classes
+		size_t allocSize; // Actual alloc size, 0 if not appendable


Do not put comment on the same line as declarations.

deadalnix · 2023-08-16T13:12:52Z

sdlib/d/gc/extent.d

@@ -202,6 +227,10 @@ public:
 		return ec.isSlab();
 	}

+	bool isLarge() const {


Why is that in the middle of slab features? Make a large features section and put the large feature related stuff in it.

deadalnix · 2023-08-16T13:13:20Z

sdlib/d/gc/extent.d

@@ -99,7 +99,15 @@ private:
 	Links _links;

 	import d.gc.bitmap;
-	Bitmap!512 _slabData = void;
+	union Meta {


deadalnix · 2023-08-16T13:14:01Z

sdlib/d/gc/extent.d

-	Bitmap!512 _slabData = void;
+	union Meta {
+		// Slab occupancy (constrained by freeSlots, so usable for all classes)
+		Bitmap!512 _slabData;


The underscore isn't necessary here anymore.

deadalnix · 2023-08-16T14:08:12Z

sdlib/d/gc/tcache.d

 		auto copySize = size;
 		auto pd = getPageDescriptor(ptr);

+		// TODO: should realloc work with pointers outside of GC?


How would it know how big the previous allocation is?

It couldn't. That comment should go.

deadalnix · 2023-08-16T14:15:26Z

sdlib/d/gc/tcache.d

+	}
+
+	// Get appendable extent of ptr, or null if this is impossible
+	Extent* maybeGetAppendableExtent(void* ptr) {


Why is this method necessary?

It is needed because any invocation of appendable API could have been given a non-appendable (slab alloc, or pointer unknown to GC.)

deadalnix · 2023-08-16T14:16:30Z

sdlib/d/gc/tcache.d

@@ -54,6 +64,39 @@ public:
 		pd.arena.free(emap, pd, ptr);
 	}

+	// Determine whether given alloc is appendable
+	bool is_appendable(void* ptr) {


This is not used anywhere, and is just a helper around another function, that doesn't seems worth including.

This is intended to be part of the exported API for appendables, rather than a helper.
What precisely do we intend to do with appendables anyway? This was not discussed yet.

deadalnix · 2023-08-16T14:25:30Z

sdlib/d/gc/tcache.d

+
+	// Get the current fill of an appendable alloc.
+	// If the alloc is not appendable, returns 0.
+	size_t get_appendable_fill(void* ptr) {


This won't work with appendable pointer. This API is still a more or less random set of features that do not seems to be connected with the actual use case.

The actual use case is that D has slices, and slices can be appended to, and reallocations needs to be avoided when unnecessary.

I don't know how to do any of this with the current API, considering it doesn't work with interior pointers.

So, what happens when I allocate a slice? What happens when I want to append to a slice? What happens when the slice is not on the GC heap? This is what matters here. I have no API do this.

I interpreted #266 as concerning the GC strictly, rather than connecting the mechanism with the rest of SDC. This may be the root of our misunderstanding and the reason you've replied with "random set of features that do not seems to be connected" to all of my PR. Could I persuade you to fully describe the requirements for satisfying #266 ? Slices, for instance, were mentioned nowhere in #266.

"The API of the GC needs to be extended with a method that allows us to query the size stored via the appendable mechanism."

This is all that was clear from #266 as originally given.

Thinking about slices: as I understand, if we're doing an append operation on a slice, we'll get the pointer to the parent array from the GC heap (or from the calling code, which already has it, when we're not doing so from the GC.)

deadalnix · 2023-08-16T14:26:16Z

sdlib/d/gc/extent.d

+
+	@property
+	ulong allocSize() {
+		return isLarge() ? _meta.allocSize : 0;


It clearly a bug to call this on slabs.

Well no, because e.g. is_appendable and the related API functions can very well end up being called on a slab alloc, and must return the correct answer (i.e. that the alloc is not appendable, and appendability operations are prohibited.)

sdlib/d/gc/tcache.d

deadalnix

The allocSize/fill/capacity lingo is very confused and confusing. This is usually a tell that the design is more complex than it needs to be.

It seems to me that this would benefit from focusing on storing and keeping track of the actual allocation size, in addition of the size of the extent itself, and maintaining the right numbers accross various operation such as realloc.

That is clearly a problem that is probably somewhat solved in this PR (although i think it is not correct), and needs to be solved in the PR, so let's build something solid we can build on.

deadalnix · 2023-08-21T15:08:02Z

sdlib/d/gc/tcache.d

+	// Get the capacity of the array slice denoted by slice and length.
+	// It includes the space occupied by the slice itself.
+	size_t getArrayCapacity(void* sliceAddr, size_t sliceLen) {
+		if ((sliceAddr is null) || (!sliceLen))


Why would a slice of length 0 would necessarily have 0 capacity? Also, I don't see why you need to special case null, as the page descriptor will be empty.

Slice of length 0 has capacity of 0. See here.

deadalnix · 2023-08-21T15:11:48Z

sdlib/d/gc/tcache.d

+
+	// Get the capacity of the array slice denoted by slice and length.
+	// It includes the space occupied by the slice itself.
+	size_t getArrayCapacity(void* sliceAddr, size_t sliceLen) {


The GC has no notion of array, only allocation, and this is better this way, IMO. You have allocation, and you have the ability to extend some allocation to fit more in them. The rest doesn't have to be handled by the GC, but can be handled by the rest of the runtime (and is largely out of scope here).

The API functions deal on the level of bytes strictly, the array terminology may be out of place here.

deadalnix · 2023-08-21T15:13:16Z

sdlib/d/gc/tcache.d

+
+		auto pd = maybeGetPageDescriptor(sliceAddr);
+		// Return zero if slice is unknown to GC or is to a non-appendable block:
+		if ((pd.extent is null) || (!pd.extent.allocSize))


You don't need to check if the alloc size is zero, as the next check will fail anyway.

deadalnix · 2023-08-21T15:15:25Z

sdlib/d/gc/tcache.d

+			return 0;
+
+		// Otherwise, capacity is the block size minus the slice length:
+		return pd.extent.size - sliceLen;


This is incorrect, the slice doesn't have to start at the beginning of the extent.

That isn't what we're checking here. Per the D docs, slice has to end at the same place as the valid data in order to have a capacity.

This is clearly wrong. Remove everything else, write the proper tests, and it'll be obvious in no time.

deadalnix · 2023-08-21T15:17:31Z

sdlib/d/gc/tcache.d

+	// Append array slice denoted by rAddr and rBytes to the one
+	// denoted by lAddr and lBytes, using the former's appendable capacity
+	// if possible, otherwise allocating a new array.
+	void* appendArray(void* lAddr, size_t lBytes, void* rAddr, size_t rBytes) {


This is the wrong abstraction level, once again. the GC could not care less about arrays. The runtime does. What you want is to be able to extend an existing allocation.

There is no reason for the GC itself to do memcopies and whatnot. All of that is language level semantic that have nothing to do in there.

If the memcopies etc. were to take place outside of the GC, the latter will need to export an API for the containsPointers logic and so forth. Do we want this? IMHO "append in place if possible, otherwise reallocate" belongs in the same place as alloc et al.

deadalnix · 2023-08-21T15:21:35Z

sdlib/d/gc/tcache.d

-			if (alignUp(size, PageSize) == esize) {
+			oldFill = pd.extent.allocSize;
+
+			// Prohibit resize below appendable fill:


I don't think this is a sensible design decision.

Why isn't it? If we don't have a stored fill, we have to copy the entire alloc space when reallocing.

deadalnix · 2023-08-21T15:26:19Z

sdlib/d/gc/tcache.d

+
+			auto oldSize = oldFill ? oldFill : pd.extent.size;
+
+			if (alignUp(size, PageSize) == alignUp(oldSize, PageSize)) {


You are short circuiting the whole thing when the size you'd get is the same. Yet, later on, you recompute the size.

deadalnix · 2023-08-21T15:28:09Z

sdlib/d/gc/tcache.d

 		}

 		containsPointers = (containsPointers | pd.containsPointers) != 0;
-		auto newPtr = alloc(size, containsPointers);
+		auto useSize = oldFill ? upsizeToLarge(size) : size;


Here you recompute the size, so the previous short circuit is incorrect. This will also lead to quadratic behavior as the size will only be bumped up one page at the time instead of exponentially, which is linear.

What kind of bump increment do we want for realloc?

Incidentally upsizeToLarge does not recompute the size unless the offered size is below the minimum for a large alloc. The size passed to alloc still needs to be aligned.

deadalnix · 2023-08-21T15:30:34Z

sdlib/d/gc/tcache.d

+
+	assert(!getCap(a));
+	assert(!getCap(b));
+	assert(!getCap(c));


Capacity is an integer. If you expect it to be 0, then check for that.

…mpty appendable allocs.

deadalnix

We are still in complexity land here. This has a few side effects:

It becomes difficult to convince oneself that all bases are covered by tests (and in fact, it doesn't looks like this is the case here).
The amount of branches grows and grows. For instance, excluding gating, realloc has 5 branches. This feature adds 3 (!!). alloc had 1 branch outside gating, now it has 2 more (!!) This is bad for perfs, testability, understandability, ...
API become nonsensical, with conditionally used arguments, or magic behavior in realloc. One pretty much has to know what the implementation of these method is to know what to expect from them, but this cannot scale to any program other than trivially small ones.

deadalnix · 2023-08-22T22:59:12Z

sdlib/d/gc/tcache.d

+	/**
+	 * Reallocation.
+	 */
+
 	void* realloc(void* ptr, size_t size, bool containsPointers) {


Keep the C API together.

deadalnix · 2023-08-23T00:02:29Z

sdlib/d/gc/tcache.d

+	void* alloc(size_t size, bool containsPointers, bool isAppendable = false,
+	            size_t spareCapacity = 0) {


The fact spareCapacity is only used in one branch is a good sign that this API is not the right one.

deadalnix · 2023-08-23T00:02:45Z

sdlib/d/gc/tcache.d

-		}
+	void* alloc(size_t size, bool containsPointers, bool isAppendable = false,
+	            size_t spareCapacity = 0) {
+		// spareCapacity is ignored if alloc is not appendable.


See you noticed it. The model is wrong. Don't say it's wrong in a comment, make it right.

deadalnix · 2023-08-23T00:10:01Z

sdlib/d/gc/sizeclass.d

@@ -43,6 +43,7 @@ enum ClassCount {
 enum SizeClass {
 	Tiny = getSizeFromClass(ClassCount.Tiny - 1),
 	Small = getSizeFromClass(ClassCount.Small - 1),
+	Large = getSizeFromClass(ClassCount.Small),


The semantic of that entry is different. It doesn't belong in there (and this can anyway be computed easily using getAllocSize).

deadalnix · 2023-08-23T00:10:26Z

sdlib/d/gc/tcache.d

@@ -1,5 +1,6 @@
 module d.gc.tcache;

+import d.gc.extent;


Why is this needed?

deadalnix · 2023-08-23T00:23:58Z

sdlib/d/gc/tcache.d

+	// Enlarging the spare capacity :
+	p0 = threadCache.realloc(p0, 16385, false);
+	assert(threadCache.getCapacity(p0, 100) == 36864);


This seems to be out of scope in general, but holly molly, an increase by a factor of almost 3x? That doesn't seem reasonable at all.

What multiplier do you consider reasonable here? Is 2x reasonable?

The size classes already provide what's needed.

Nevertheless it is out of scope for the PR.

deadalnix · 2023-08-23T00:27:14Z

sdlib/d/gc/tcache.d

+	// Reduce again to minimum:
+	p0 = threadCache.realloc(p0, 100, false);
+	assert(threadCache.getCapacity(p0, 100) == 16384);
+
+	// Enlarge the empty (will double in total capacity) :
+	p2 = threadCache.realloc(p2, 16385, false);
+	assert(threadCache.getCapacity(p2, 0) == 32768);


At no point do you test the case for which the realloc is done to a size smaller than the capacity, which definitively should impact the capacity tracking, consider realloc's semantic.

// This realloc will have no effect, as the requested size // is below the appendable fill size: auto p3 = threadCache.realloc(p0, 99, false); assert(p3 == p0);

Again, under what circumstances would it be correct to permit this??

deadalnix · 2023-08-23T00:32:36Z

sdlib/d/gc/tcache.d

+	// Zero-length segment of alloc where data exists in front of it:
+	assert(threadCache.getCapacity(p0, 0) == 0);


The obvious missing case is length 0 at the end of the segment.

deadalnix · 2023-08-23T00:34:19Z

sdlib/d/gc/tcache.d

+			// New block will be appendable if the old one was:
+			appendable = pd.extent.isAppendable();
+
 			// TODO: Try to extend/shrink in place.
-			import d.gc.util;
-			copySize = min(size, esize);
+			if (appendable) {
+				// Prohibit resize below appendable fill:
+				if (size < pd.extent.allocSize)
+					return ptr;
+				copySize = pd.extent.allocSize;
+				allocateSize = copySize;
+				// If enlarging an appendable, boost spare capacity:
+				if (esize < size)
+					spareCapacity = max(size, esize * 2);
+			} else {
+				copySize = min(size, esize);
+			}


A lot of this doesn't seem to relate to keeping track of capacity.

deadalnix · 2023-08-23T00:55:26Z

sdlib/d/gc/tcache.d

+				// Prohibit resize below appendable fill:
+				if (size < pd.extent.allocSize)
+					return ptr;


Under what circumstances would one want to realloc below the size of the live data?!

I don't know, maybe because the data is not needed anymore. What matter is that we were asked to.

Accumulating more and more special cases trying to guess what to do in each instance is the recipe for exploding complexity.

If we are requested to reallocate into something smaller, we ought to do it. That's really all there is to consider.

deadalnix · 2023-08-24T00:57:04Z

Please update title/description and this can go in. Next step is extend.

bare-bones appendables (not tested yet)

3a28488

deadalnix reviewed Aug 15, 2023

View reviewed changes

dsm9000 marked this pull request as draft August 15, 2023 13:21

deadalnix requested changes Aug 15, 2023

View reviewed changes

tests; recommended corrections; afaik everything but adjustment on re…

bb26af1

…size (todo)

dsm9000 changed the title ~~bare-bones appendables~~ Appendability for large allocs. Aug 15, 2023

dsm9000 added 10 commits August 15, 2023 15:52

realloc handling for appendables; tests for same

6cdf9b7

realloc handling for appendables; tests for same

2afbad9

realloc handling for appendables; tests for same

781dd37

realloc handling for appendables; tests for same

af06b86

realloc handling for appendables; tests for same

fb86c4f

realloc handling for appendables; tests for same (simplified)

20850f6

realloc handling for appendables; tests for same (simplified)

6bcd254

realloc handling for appendables; tests for same (simplified)

0de7276

realloc handling for appendables (with tests for unknown-to-gc ptrs)

2e782e6

realloc handling for appendables (with tests for unknown-to-gc ptrs)

43b61bd

dsm9000 marked this pull request as ready for review August 16, 2023 13:57

deadalnix requested changes Aug 16, 2023

View reviewed changes

dsm9000 added 2 commits August 16, 2023 12:50

corrections

852d98f

Merge branch 'master' of github.com:dsm9000/sdc into basic_appendables

960ccf6

dsm9000 marked this pull request as draft August 21, 2023 04:10

dsm9000 added 6 commits August 21, 2023 00:37

in-place appending api.

e5c753e

in-place appending api.

5c276ff

in-place appending api.

23abbcd

in-place appending api. (corrections)

e697fba

in-place appending api. (corrections, tests)

d4d9899

in-place appending api. (corrections, tests)

3b439dd

deadalnix requested changes Aug 21, 2023

View reviewed changes

getCapacity(), realloc(), and tests for same; support for initially-e…

f8d87ba

…mpty appendable allocs.

dsm9000 added 5 commits August 22, 2023 14:46

getCapacity(), realloc(), and tests for same; support for initially-e…

2037ca8

…mpty appendable allocs.

getCapacity(), realloc(), and tests for same; support for initially-e…

103e7fe

…mpty appendable allocs.

getCapacity(), realloc(), and tests for same; support for initially-e…

978238c

…mpty appendable allocs.

getCapacity(), realloc(), and tests for same; support for initially-e…

54dcc43

…mpty appendable allocs.

getCapacity(), realloc(), and tests for same; support for initially-e…

ef01fee

…mpty appendable allocs.

dsm9000 marked this pull request as ready for review August 22, 2023 19:36

dsm9000 added 6 commits August 22, 2023 16:29

getCapacity() docs

9066a34

getCapacity() docs

4e7db41

getCapacity() docs

f15d0ac

getCapacity() docs

5bce071

getCapacity() docs

eac9095

getCapacity() corrections

2cd70e3

deadalnix requested changes Aug 23, 2023

View reviewed changes

deadalnix force-pushed the master branch 2 times, most recently from f94baa8 to 87a9baf Compare August 23, 2023 22:14

dsm9000 marked this pull request as draft August 23, 2023 23:29

dsm9000 added 3 commits August 23, 2023 19:31

various corrections

0c73f63

various corrections

514481e

corrections and simplifications suggested by deadalnix.

733feca

dsm9000 marked this pull request as ready for review August 24, 2023 00:08

dsm9000 changed the title ~~Appendability for large allocs.~~ allocAppendable() and getCapacity(). Aug 24, 2023

deadalnix merged commit 0196479 into snazzy-d:master Aug 24, 2023
3 checks passed

		void* finalizer; // Reserved, not used currently
		ubyte[47] _unused;


		auto oldSize = oldFill ? oldFill : pd.extent.size;

		if (alignUp(size, PageSize) == alignUp(oldSize, PageSize)) {

		void* alloc(size_t size, bool containsPointers, bool isAppendable = false,
		size_t spareCapacity = 0) {

		// Zero-length segment of alloc where data exists in front of it:
		assert(threadCache.getCapacity(p0, 0) == 0);

allocAppendable() and getCapacity(). #284

allocAppendable() and getCapacity(). #284

Conversation

dsm9000 commented Aug 15, 2023 • edited Loading

Choose a reason for hiding this comment

deadalnix left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

deadalnix commented Aug 15, 2023

deadalnix left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

deadalnix left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

deadalnix left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

deadalnix commented Aug 24, 2023

dsm9000 commented Aug 15, 2023 •

edited

Loading