VOTS 2023 Submission of READMem-MiVOS

⚠️ This is our Tracker submission for the VOTS2023 Challenge. ❗❗

---

🚧 TODO list:

• Clean the code (simplify and more readable)
• Clean the README file
• Clean Conda_environment.txt" ( Filter the list to the essential packages only)

---

Setting up the environment:

Create a conda environment and install the needed packages.

conda create --name Tracker_READMem_MiVOS python=3.10
conda install --name Tracker_READMem_MiVOS --file Conda_environment.txt
conda activate Tracker_READMem_MiVOS

Setting up the tracker

Clone this repository and download the propagation model of MiVOS

git clone https://github.com/Vujas-Eteph/Tracker_READMem_MiVOS
cd Tracker_READMem_MiVOS/MiVOS
python download_model.py

Evaluate on the VOTS 2023 dataset

Supposing you already have the VOT toolkit, Trax package and the integration installed. Set up the workspace with the vots_2023 stack and integrate/test the tracker

cd ../../
vot initialize vots2023 --workspace workspace_VOTS_2023
cp Tracker_READMem_MiVOS/trackers.ini workspace_VOTS_2023/trackers.ini
cd workspace_VOTS_2023
ln -s ../Tracker_READMem_MiVOS/ Tracker_READMem_MiVOS

Before going further, adapt the paths:

• in trackers.ini
• and in READMem_Tracker.py where the path of the MiVOS propagation model and the configuration file should be changed.

Now test the tracker on 4 small sequences.

vot test Tracker_READMem_MiVOS

You should get the following output message: Test concluded successfully

Now let's evaluate the tracker on the vots_2023 stack

cd ..
vot evaluate --workspace workspace_VOTS_2023 Tracker_READMem_MiVOS

After a while (approx. 35 hours), we can run the analysis and pack the results to sent to the server:

vot analysis --workspace workspace_VOTS_2023 Tracker_READMem_MiVOS
vot pack --workspace workspace_VOTS_2023 Tracker_READMem_MiVOS

History

Here is a link to the number of attempts and their specificity - i.e., what change we made to the original READMem_miVOS tracker that made the performance decrease or increase.

Credits

• READMem for the memory management
• MiVOS for the core-architecture
• VOT challenge for providing the dataset

Exhaustive and useful information:

Official to VOT

• The official VOT Challenge website: https://www.votchallenge.net/
• The official VOT Challenge Support section (Always refer to this page when in trouble): https://www.votchallenge.net/howto/integration_multiobject.html
• The official VOT ToolKit (Python 3 version) GitHub page (don't forget to look up issues (sometimes usefull information)): https://github.com/votchallenge/toolkit
• The official VOT Challenge Support FORUM (Only for discussions): https://groups.google.com/g/votchallenge-help
• Official terminology (under Key concepts): https://github.com/votchallenge/toolkit/blob/master/docs/overview.rst
• More information on when submitting the results to the VOT server: votchallenge/toolkit#102

---

If you find this work helpful/useful, please consider citing the original paper:

@misc{vujasinović2023readmem,
      title={READMem: Robust Embedding Association for a Diverse Memory in Unconstrained Video Object Segmentation}, 
      author={Stéphane Vujasinović and Sebastian Bullinger and Stefan Becker and Norbert Scherer-Negenborn and Michael Arens and Rainer Stiefelhagen},
      year={2023},
      eprint={2305.12823},
      archivePrefix={arXiv},
      primaryClass={cs.CV}
}