Hi! I am now working on your SeqNet project, and meeting some problems through it. Currently I can run the codes and get recall results using nordland and oxford dataset. Now I am trying to use other datasets like hilti: https://hilti-challenge.com/dataset.html . I wonder how I can fit this dataset into your program. Should I also generate .db AND .npy files as well? If my original data format is .jpg, what should be my process? the .npy files under /netvlad-pytorch file are descriptors generated by netVLAD, am I right?

Thank you for your patience and waiting for your kindly help!

Hi! Thanks for your work! I have some doubts about the nordland image descriptors.

1. The nordland image descriptors are from netvlad. Whether the netvlad model was trained in the nordland?
2. Which one is a fixed distance sample between oxford-pnv and oxford-v1.0

Kind regards!

Hi @oravus , thanks for your help and I have generated the needed db files using pitts250k dataset. Generally, I use the whole dataset to generate descriptors and saved both query and ref .npy files. That is : ===> Loading dataset(s) All Db descs: (254064, 4096) All Qry descs: (24000, 4096)

Next, I use the first 10000 images as train sets, followed by 3000 images as validation and 3000 images as test sets. Also, I generated 3 .db files as train_mat_file, test_mat_file, val_mat_file.

Thereafter, I wrote the specification in get_datasets.py. The indexes are defined as nordland dataset format: trainInds, testInds, valInds = np.arange(10000), np.arange(10000,13000), np.arange(13000,16000)

And I think I can test the dataset using your pretrained model, but things comes out that still errors occur regarding index.

////////////////////////////////////////// Restored flags: ['--optim', 'SGD', '--lr', '0.0001', '--lrStep', '50', '--lrGamma', '0.5', '--weightDecay', '0.001', '--momentum', '0.9', '--seed', '123', '--runsPath', './data/runs', '--savePath', './data/runs/Jun03_15-22-44_l10_l10_w5_seqnetEnv/checkpoints', '--patience', '0', '--pooling', 'seqnet', '--w', '5', '--outDims', '4096', '--margin', '0.1'] Namespace(batchSize=16, cacheBatchSize=24, cachePath='./data/cache', cacheRefreshRate=0, ckpt='latest', dataset='pitts250k', descType='netvlad-pytorch', evalEvery=1, expName='0', extractOnly=False, lr=0.0001, lrGamma=0.5, lrStep=50.0, margin=0.1, mode='test', momentum=0.9, msls_trainCity='melbourne', msls_valCity='austin', nEpochs=200, nGPU=1, nocuda=False, numSamples2Project=-1, optim='SGD', outDims=4096, patience=0, pooling='seqnet', predictionsFile=None, resultsPath=None, resume='./data/runs/Jun03_15-22-44_l10_w5/', runsPath='./data/runs', savePath='./data/runs/Jun03_15-22-44_l10_l10_w5_seqnetEnv/checkpoints', seed=123, seqL=5, seqL_filterData=None, split='test', start_epoch=0, threads=8, w=5, weightDecay=0.001) ===> Loading dataset(s) All Db descs: (254064, 4096) All Qry descs: (24000, 4096) ===> Evaluating on test set ====> Query count: 800 ===> Building model => loading checkpoint './data/runs/Jun03_15-22-44_l10_w5/checkpoints/checkpoint.pth.tar' => loaded checkpoint './data/runs/Jun03_15-22-44_l10_w5/checkpoints/checkpoint.pth.tar' (epoch 200) ===> Running evaluation step ====> Extracting Features 20%|████████████████████████████████████▉ | 49/249 [00:00<00:01, 127.41it/s]==> Batch (50/250) 39%|█████████████████████████████████████████████████████████████████████████▏ | 97/249 [00:00<00:01, 148.19it/s]==> Batch (100/250) 58%|████████████████████████████████████████████████████████████████████████████████████████████████████████████▉ | 145/249 [00:01<00:00, 154.13it/s]==> Batch (150/250) 78%|████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████▉ | 193/249 [00:01<00:00, 156.80it/s]==> Batch (200/250) 97%|█████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████▋ | 242/249 [00:01<00:00, 158.98it/s]==> Batch (250/250) Average batch time: 0.006786982536315918 0.009104941585046229
torch.Size([3000, 4096]) torch.Size([3000, 4096]) ====> Building faiss index ====> Calculating recall @ N Using Localization Radius: 25 Traceback (most recent call last): File "main.py", line 133, in recallsOrDesc, dbEmb, qEmb, rAtL, preds = test(opt, model, encoder_dim, device, whole_test_set, writer, epoch, extract_noEval=opt.extractOnly) File "/home/lx/lx/Seqnet_new/test.py", line 138, in test rAtL.append(getRecallAtN(n_values, predictions, gtAtL)) File "/home/lx/lx/Seqnet_new/test.py", line 37, in getRecallAtN if len(gt[qIx]) == 0: IndexError: list index out of range

///////////////////////////////////////// I am writing to ask :

1. how should the index should be defined? Could you use oxford dataset or nordland dataset to explain the details for me?
2. should npy files contain the same number of descriptors as I generate the dataset as 3 db files for train, test, and val?

THANKS FOR HELP!

Hi! Thanks for your great work on SeqNet. And I met a problem when I trained the sequential descriptors on the MSLS dataset. I just run the command: python main.py --mode train --pooling seqnet --dataset msls --msls_trainCity melbourne --msls_valCity austin --seqL 5 --w 3 --outDims 4096 --expName "msls_w3". But ther is a problem: No such file or directory: './data/descData/netvlad-pytorch/msls_melbourne_databast.npy'. Maybe I did not find any corresponding ***.npy files about the MSLS dataset. Would you mind providing the corresponging download link of the MSLS dataset.? Kind regards!

Hello @oravus ,

Thanks for your fantastic job providing a nice way to fuse information from multiple frames. Now I only use SeqNet (Conv+SAP+L2-Norm) without SeqMatch, trained with my own dataset for LIDAR-based place recognition. However, I found SeqNet did not work well once the seq_len is small (< 20) and was hard to train to get a strong model. Thus I guess that whether SeqNet without SeqMatch works well largely depends on the distribution of training set and test set, or it largely depends on the output module of the raw descriptor generation algorithm. Is this right? Have you met some dataset where SeqNet can not work well?

Best wishes!

Hi, I would like to replicate the results on Mapillary. Thank you for adding the code to generate the .npy files containing the descriptors for any dataset. Could you please suggest how to generate the .db files for MSLS and then replicate the results in your paper? Thank your help!

Hi May I ask what the sequential descriptor really encodes?

As far as I understand, I can use LSTM to estimate both self-motion as well as motion stereo depth.

Does LSTM here only encodes the changes in locomotion? or does it also encode the overall 3D structure prior?

If it encodes either motion or map, I can use short-duration odometry as additional field input or something like LIDAR projected depth map to speed up the seqNet process to make it multi-modality seqNet?

Hi, thanks for your remarkable work! Could you please provide the original images dataset for training SeqNet? such as Nordland, Oxford, Brisbane, and MSLS? Thanks.

Hi! Thanks for your great work! The argument, “seqL_filterData”, has always defaulted to None, What does it do and under what conditions is it used? Kind regards!

Hello @oravus ,

Thanks for your fantastic job providing a nice way to fuse information from multiple frames. Now I only use SeqNet (Conv+SAP+L2-Norm) without SeqMatch, trained with my own dataset for LIDAR-based place recognition. However, I found SeqNet did not work well once the seq_len is small (< 20) and was hard to train to get a strong model. Thus I guess that whether SeqNet without SeqMatch works well largely depends on the distribution of training set and test set, or it largely depends on the output module of the raw descriptor generation algorithm. Is this right? Have you met some dataset where SeqNet can not work well?

Best wishes!

Hello @oravus ,

Thanks for your fantastic job providing a nice way to fuse information from multiple frames. Now I only use SeqNet (Conv+SAP+L2-Norm) without SeqMatch, trained with my own dataset for LIDAR-based place recognition. However, I found SeqNet did not work well once the seq_len is small (< 20) and was hard to train to get a strong model. Thus I guess that whether SeqNet without SeqMatch works well largely depends on the distribution of training set and test set, or it largely depends on the output module of the raw descriptor generation algorithm. Is this right? Have you met some dataset where SeqNet can not work well?

Best wishes!

Hi! Thanks for your great work on SeqNet. With your kindly help in issue #12 opened by myself, I have obtained the corresponding descriptor data for the Nordland, Oxford, and MSLS datasets, and achieved the same results as the paper. Then, I found that the Bribane dataset is not open-source, such that I cannot directly obtain the corresponding descriptor through the original dataset imsges. I have found your reply--"For the Brisbane dataset, we can only release the descriptor data." in issue #1 that the descritpor data can be released. Would you mind providing the descriptor data of the Brisbane dataset for me? I just want to reproduce all the results in the paper and improve this remarkable work. I promise it will never be commercial but for research. Kind regards!