Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

The PLY file generated is not an octree. #30

Open
ZhiyeTang opened this issue Jun 18, 2024 · 11 comments
Open

The PLY file generated is not an octree. #30

ZhiyeTang opened this issue Jun 18, 2024 · 11 comments

Comments

@ZhiyeTang
Copy link

I've run the released code. During training, the anchors will be adjusted adaptively. However, the anchors in the generated PLY file can not be constructed as an octree because of ``children-node duplication''. I've read the logic of anchor_growing function defined in class `GaussianModel`. There're two stages of each `cur_level`. The first stage is to generate new anchors for level `cur_level`, and the second one generates anchors for level `cur_level + 1`, resulting in variables `candidate_anchor` and `candidate_anchor_ds` respectively. However, for `candidate_anchor_ds`, the `get_remove_duplication` should consider its duplication between `candidate_anchor_ds` with the anchors of `cur_level + 1` (as they eventually will be assigned a level value of `cur_level + 1`), while in the released code, is `cur_level`. I wonder the reason of setting the code like this.

@ZhiyeTang
Copy link
Author

What‘s more, I've noticed that some anchors do not have a parent anchor.

@tongji-rkr
Copy link
Contributor

Thank you for your detailed suggestions. The problem you mentioned stems from the details of the anchor growth strategy, and we will modify it as soon as possible.

@ZhiyeTang
Copy link
Author

I modified your code to prevent duplicate anchors from growing. Afterward, I re-trained bicycle on your released code, finding that there are numerous ``orphan'' anchors with no parent anchors, and each parent anchor typically has only one child.
image

@tongji-rkr
Copy link
Contributor

The former is due to the anchor removing strategy, and the latter relates to the minimum and maximum length of the octree voxels.

@ZhiyeTang
Copy link
Author

ZhiyeTang commented Jun 19, 2024

The existence of orphan anchors might be due to the growing strategy, which does not consider whether the candidates' parents exist. As to the latter, I don't understand why the minimum and maximum voxel sizes relate to it.

Otherwise, I've also analyzed the number of duplicated anchors when I am re-training bicycle on your released code while setting use_chunk=False in the function get_remove_duplication. After 10,000 iterations of training, the number of duplicated anchors is up to 120,000, among a total of 480,000 anchors. This indicates that the wrong setting in the mentioned problem is significant to the experimental results reported in your submitted paper since duplicated anchors do contribute to the final rendering images. I kindly recommend you and your team re-evaluate Octree-GS using the correct code ASAP.

@tongji-rkr tongji-rkr reopened this Jun 19, 2024
@tongji-rkr
Copy link
Contributor

For duplicate anchors, we will update the code and weights file within the week. Besides, we have a general strategy for the orphan anchor problem, but it will take some time to adjust to get the best results.

@ZhiyeTang
Copy link
Author

ZhiyeTang commented Jun 19, 2024

I've run the fixed code on bicycle (removing duplication using the correct level). The PSNR, SSIM, and LPIPS scores are similar to those you reported in your paper. That seems like good news. However, since you've got a general strategy for orphan anchors, I'm curious about it. Can you please state how you are going to deal with this? Thanks a lot!

@tongji-rkr
Copy link
Contributor

tongji-rkr commented Jun 19, 2024

Simply, when you add a new anchor, you (1) recursively complete all parent node (with the possibility of adding more than one anchor) or (2) recursively find the anchor and add the child anchor (with the guarantee of adding one anchor at a time). For example, when adding anchors in the same layer of the 5th layer, the corresponding corresponding position only has the anchor of the 2nd layer, adding only the point of the 5th layer results in orphan anchors, (1) adding the anchor of the 3rd, 4th and 5th layer (2) adding only the 3rd layer. At the same time, all child anchors are recursively removed when an anchor is removed.

@ZhiyeTang
Copy link
Author

Thank you for your kind share. Looking forward to your updated release!!

@tongji-rkr
Copy link
Contributor

image
This is the change in performance after resolving the duplicate anchor. The metrics are down but the storage is less. We are currently moving on to the next stage of the problem, so we will not update the code for now.

@ZhiyeTang
Copy link
Author

ZhiyeTang commented Jun 26, 2024

image This is the change in performance after resolving the duplicate anchor. The metrics are down but the storage is less. We are currently moving on to the next stage of the problem, so we will not update the code for now.

Congrats! I think such a slight decrease in visual performance is acceptable. Anyway, thank you for your effort in the community. Hopefully your brand new Octree-GS can be released soon! Moreover, I wrote a CUDA module that can search the parent-children relationship quickly, check here. I hope this simple tool helps you.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants