[Neo VM] adopt light gc from neogo #3581
Conversation
Jim8y
requested review from
cschuchardt88,
shargon,
vncoelho,
AnnaShaleva and
Hecate2
|
A few unit test was updated to adapt light GC. Need help from neogo to review this pr, @roman-khimov @AnnaShaleva. |
There was a problem hiding this comment.
- It should be a part of Echidna, we can't easily throw away the old RC.
- Probably the best way to test is to keep both RCs in the code and run through testnet/mainnet tx set, comparing values after every instruction. My expectation is they will be the same (at least if don't have any circular references in contract code).
src/Neo.VM/ReferenceCounterV2.cs
Outdated
| if (item is CompoundType compoundType) | ||
| { | ||
| // Increment the item's stack references by the specified count. | ||
| compoundType.StackReferences--; |
There was a problem hiding this comment.
In circular references this counter will become negative
There was a problem hiding this comment.
for stack item A, B:
-
A->C, B->D => ref.A = 1, ref.B = 1, ref.C = 1, ref.D = 1
-
C->D => ref.C = 1, ref.D = 2.
-
D->C => ref.C = 2, ref.D = 2
when we try to remvoe C from A, we will have:
- C--; => ref(C) == 1
5.then, remve D from B, we will have:
D--; => ref.D = 1, // from here the operation related to D and C will be over, the RC will never be decreased anymore.
- but if we revert step 5, and try to remove C from D, we will have:
C--; => ref.C =0,
causing D--; ref.D = 1.
- following setp 6, remove D from B,
D--; ref.D =0, but since C is removed from D in step 6, no further operation will be taken
There was a problem hiding this comment.
I don't think it can become negative, rather it will have some positive count that is not exactly true (like 1 in RC with no elements on the stack), but this is a safe behavior.
|
For me this change seems good, but the test changed to fit this version, so if we want to bee 100% safe we will need a HF and be able to execute with the previous version |
should be the same if properly implementated, some ut was updated cause gc no longer work the same way, for instance, in this pr, you dont have to worry about adding a compound type whose subitems are not RC counted to an array, the GC will do it,,, so the test for adding RC uncounted compound type will no longer work. |
|
more tests. running on mainnet and testnet. |
| { | ||
| if (!map.TryGetValue(key, out var value1)) | ||
| { | ||
| engine.ReferenceCounter.AddStackReference(key); |
There was a problem hiding this comment.
the key is added multiple times? @roman-khimov
There was a problem hiding this comment.
the key is added if it does not exsit in the dictionary, when the value is removed from the dic, the key is removed as well. So no value also means no key.
There was a problem hiding this comment.
But if i add twice the key is only one and is counted as two
There was a problem hiding this comment.
if you add twice, the key wont be changed, key will only be added once, then it will only be removing the value, and add the value.
The complete logic here is:
key exists:
remove old value ref, add new value ref
key does not exist:
add key ref, add new value ref.
There was a problem hiding this comment.
The key don't change in the map, but it's called twice to engine.ReferenceCounter.AddStackReference(key);
| break; | ||
| case Map map: | ||
| map.Remove(key); | ||
| var old = map.RemoveKey(key); | ||
| if (old != null && engine.ReferenceCounter.Version == RCVersion.V2) |
There was a problem hiding this comment.
This is checked that can return false in neo-go? @roman-khimov
| Array array = new(new StackItem[] { 1, 2, 3, 4 }); | ||
| engine.CurrentContext.EvaluationStack.Push(array); | ||
| Assert.AreEqual(array.Count + 1, engine.ReferenceCounter.Count); |
There was a problem hiding this comment.
We should try to avoid the UT changes, in this case, what's wrong exactly?
There was a problem hiding this comment.
this is because of how we process reference counter for compound type, previously we can test the RC without the engine, cause stackitem itself will process the RC, but in the new version, only executionstack will process the stack, as a result, we must push the item to the stack to get a RC count.
That is why it was 4 in the past, as the array is not in the stack. 4 = Array.subitems.
While new version its 5 (4+1), cause we have to push the array itself to the stack. 5 = Array.subitems+Array
|
I already made my optimizations/review, just we should ensure that it works with tons of UT and checking all the UT that was not working with the old logic |
|
|
||
| if (item is CompoundType compoundType) | ||
| { | ||
| // Increment the item's stack references by the specified count. |
| /// <inheritdoc/> | ||
| public void RemoveStackReference(StackItem item) | ||
| { | ||
| // Increment the reference count by the specified count. |
| _referencesCount += count; | ||
|
|
||
| if (_referencesCount > _limits.MaxStackSize) | ||
| throw new IndexOutOfRangeException("Circular reference detected, execution stopped."); |
There was a problem hiding this comment.
Not the best exception message. It can be simply caused by too many objects
Description
This pr tries to adopt the light GC solution from neogo.
It:
Fixes # (issue)
Type of change
Test Configuration:
Checklist: