Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Option to delay video for more accurate background removal #605

Open
pd431 opened this issue Nov 5, 2024 · 0 comments
Open

Option to delay video for more accurate background removal #605

pd431 opened this issue Nov 5, 2024 · 0 comments
Labels
enhancement New feature or request

Comments

@pd431
Copy link

pd431 commented Nov 5, 2024

Issue I face
Using both MediaPipe and RVM, the background removal is fine when I am static, but feels like it is calculated a few frame too late when moving.
This means if I'm moving my head to the right, the rightmost portion of my head is cut out, and the background left of my head is visible.
This happens on CPU or GPU accelerated removal. The effect can be lessened by setting Temporal smoothing to 0, but that means the contour may flicker a lot more than if set to 0.1

Proposed solution

As an alternative to fiddling with temporal smoothing, I would prefer giving a few frames of delay to the engine to allow more accurate matting to be applied. Given the models do a really decent job statically, I wonder whether giving it 2 extra frames to work with might be a possible solution

Possible alternatives

Switching temporal smoothing to zero makes this a lot better (although doesn't fix the issue entirely). Getting a higher quality camera where temporal smoothing set to zero is not quite as flickery might also help.

Additional context
Running Win10, Inference running on Intel Arc 770LE GPU

@pd431 pd431 added the enhancement New feature or request label Nov 5, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

No branches or pull requests

1 participant