step: Matching patches across datasets. Error num_samples = 0

[luda1013]

Hallo Pros,

i am trying to implement the pipeline and want to try to train and test it with PfD Dataset.
all is good until i am at step: matching patches across datasets: python epe/matching/feature_based/collect_crops.py PfD pfd_files.csv

the code tells to create csv file: "The csv files for the datasets contain per row the paths to the images of each dataset."
so i created it only for 5 images first:
Here is my csv looks like (only contains 5 lines):
/Home/PEcon2/PfD/01_images/images/00001.png
/Home/PEcon2/PfD/01_images/images/00002.png
/Home/PEcon2/PfD/01_images/images/00003.png
/Home/PEcon2/PfD/01_images/images/00004.png
/Home/PEcon2/PfD/01_images/images/00005.png

then saved it, run the line: python epe/matching/feature_based/collect_crops.py PfD pfd_files.csv
i got error said: ValueError: num_samples should be a positive integer value, but got num_samples=0

then i tried to run the corresponding python scripst line per line in jupyter notebook. what i found:

1.) in collect_crops.py
the error begins at dataset = ImageDataset(args.name, read_filelist(args.img_list,1,False)
this line gives out result 0, which is led to this Error num_sample = 0

The function read_filelist(args.img_list,1,False) is okay, i run it in notebook, i got this as result:
[(PosixPath('/Home/PEcon2/PfD/01_images/images/00001.png'),),
(PosixPath('/Home/PEcon2/PfD/01_images/images/00002.png'),),
(PosixPath('/Home/PEcon2/PfD/01_images/images/00003.png'),),
(PosixPath('/Home/PEcon2/PfD/01_images/images/00004.png'),),
(PosixPath('/Home/PEcon2/PfD/01_images/images/00005.png'),)]

That means my problem is in the imageDataset

2.) in image_datasets.py: Class ImageDataset
in line 26 and 27:
img_paths = [Path(p[0] if type(p) is tuple else p) for p in img_paths]
self.paths = sorted([p for p in img_paths if p.is_file() and p.suffix in ['.jpg', '.png']])

the result from the function read_filelist i saved it in other variable, lets say = read
should the path(p) tuple or not? because tried to print (type(read) is tuple) returns False
and the line 27 which is sorted([p for ..... ) returns [ ]

i think this is what makes the command dataset = ImageDataset(args.name, read_filelist(args.img_list,1,False) returns 0

so did i make mistake in the csv File? i also tried to directly run the training command:
python epe/EPEExperiment.py train ./config/train_pfd2cs.yaml --log=info

but i assume it will not work until i run the all the command above the training section, right?

Thanks for your help!

[metaiintw]

I also encountered the same problem, and I think you might be able to solve your problem by observing this code:

image_datasets.py: Class ImageDataset
line 27:
self.paths = sorted([p for p in img_paths if p.is_file() and p.suffix in ['.jpg', '.png']])

There are only two reasons that could cause num_samples to be 0, either p.is_file() or p.suffix in ['.jpg', '.png'] is False. In my case, the problem was caused by p.is_file() being False, and the reason was that I had typos in my image paths. After I corrected the file paths, the problem of num_samples = 0 was solved successfully.

I hope this information is helpful to you.

[luda1013]

ya, i already solved it, i made it a lil bit complicated, i modified the function and defined new Dataset class.. but in the end i didnt need it, it is just about the path, before i used absolute path and the python couldnt access it, but i tried with relative path and it works.
now i am at the MSeg, need to make label maps using Mseg for PfD and cityscapes, and then maybe i can get it to train. wish me luck :)

[luda1013]

@metaiintw Hi, so btw did you already train the EPE-algo? because i am still stuck at MSeg.
I dont know why, but i used the pretrained weight for MSeg, was using it for segmenting PfD-Datasets.
Using the universaldemo.py for a folder, i only got the gray scale image. And 1 image took like 10 minutes!!!

So in your case, were u succesfull at MSeg and at the end training the EPE? i want to know your step by step :)
Thanks