Hi all

After I executed the command python examples/resnet.py -d viper -b 64 -j 2 --loss oim --logs-dir logs/resnet-viper-oim I encountered the following errors:

Process Process-4: Traceback (most recent call last): File "/root/miniconda2/lib/python2.7/multiprocessing/process.py", line 258, in _bootstrap self.run() File "/root/miniconda2/lib/python2.7/multiprocessing/process.py", line 114, in run self._target(*self._args, **self._kwargs) File "/root/miniconda2/lib/python2.7/site-packages/torch/utils/data/dataloader.py", line 45, in _worker_loop data_queue.put((idx, samples)) File "/root/miniconda2/lib/python2.7/multiprocessing/queues.py", line 392, in put return send(obj) File "/root/miniconda2/lib/python2.7/site-packages/torch/multiprocessing/queue.py", line 17, in send ForkingPickler(buf, pickle.HIGHEST_PROTOCOL).dump(obj) File "/root/miniconda2/lib/python2.7/pickle.py", line 224, in dump self.save(obj) File "/root/miniconda2/lib/python2.7/pickle.py", line 286, in save f(self, obj) # Call unbound method with explicit self File "/root/miniconda2/lib/python2.7/pickle.py", line 554, in save_tuple save(element) File "/root/miniconda2/lib/python2.7/pickle.py", line 286, in save f(self, obj) # Call unbound method with explicit self File "/root/miniconda2/lib/python2.7/pickle.py", line 606, in save_list self._batch_appends(iter(obj)) File "/root/miniconda2/lib/python2.7/pickle.py", line 639, in _batch_appends save(x) File "/root/miniconda2/lib/python2.7/pickle.py", line 286, in save f(self, obj) # Call unbound method with explicit self File "/root/miniconda2/lib/python2.7/multiprocessing/forking.py", line 67, in dispatcher self.save_reduce(obj=obj, *rv) File "/root/miniconda2/lib/python2.7/pickle.py", line 401, in save_reduce save(args) File "/root/miniconda2/lib/python2.7/pickle.py", line 286, in save f(self, obj) # Call unbound method with explicit self File "/root/miniconda2/lib/python2.7/pickle.py", line 554, in save_tuple save(element) File "/root/miniconda2/lib/python2.7/pickle.py", line 286, in save f(self, obj) # Call unbound method with explicit self File "/root/miniconda2/lib/python2.7/multiprocessing/forking.py", line 66, in dispatcher rv = reduce(obj) File "/root/miniconda2/lib/python2.7/site-packages/torch/multiprocessing/reductions.py", line 113, in reduce_storage fd, size = storage.share_fd() RuntimeError: unable to write to file </torch_29225_1654046705> at /py/conda-bld/pytorch_1493669264383/work/torch/lib/TH/THAllocator.c:267

When switch to the xentropy loss with python examples/resnet.py -d viper -b 64 -j 1 --loss xentropy --logs-dir logs/resnet-viper-xentropy The following error occured:

Exception in thread Thread-1: Traceback (most recent call last): File "/root/miniconda2/lib/python2.7/threading.py", line 801, in __bootstrap_inner self.run() File "/root/miniconda2/lib/python2.7/threading.py", line 754, in run self.__target(*self.__args, **self.__kwargs) File "/root/miniconda2/lib/python2.7/site-packages/torch/utils/data/dataloader.py", line 51, in _pin_memory_loop r = in_queue.get() File "/root/miniconda2/lib/python2.7/multiprocessing/queues.py", line 378, in get return recv() File "/root/miniconda2/lib/python2.7/site-packages/torch/multiprocessing/queue.py", line 22, in recv return pickle.loads(buf) File "/root/miniconda2/lib/python2.7/pickle.py", line 1388, in loads return Unpickler(file).load() File "/root/miniconda2/lib/python2.7/pickle.py", line 864, in load dispatchkey File "/root/miniconda2/lib/python2.7/pickle.py", line 1139, in load_reduce value = func(*args) File "/root/miniconda2/lib/python2.7/site-packages/torch/multiprocessing/reductions.py", line 68, in rebuild_storage_fd fd = multiprocessing.reduction.rebuild_handle(df) File "/root/miniconda2/lib/python2.7/multiprocessing/reduction.py", line 155, in rebuild_handle conn = Client(address, authkey=current_process().authkey) File "/root/miniconda2/lib/python2.7/multiprocessing/connection.py", line 169, in Client c = SocketClient(address) File "/root/miniconda2/lib/python2.7/multiprocessing/connection.py", line 308, in SocketClient s.connect(address) File "/root/miniconda2/lib/python2.7/socket.py", line 228, in meth return getattr(self._sock,name)(*args) error: [Errno 111] Connection refused

In both situations, the terminal is frozen after these errors prompt. I have to kill the corresponding Python process in order to exit. Any suggestions to solve this?

Hi there~

I see that the OIM Loss is a little different to that described in the paper. This implementation omitted the Circular Queue(CQ) part. May I ask what's the consideration behind this change?

Thank you.

hi，i have trained a resnet50 on my dataset.and i want to extract the feature that on pool5，how to use the parameters that on function of outputs = extract_cnn_feature(model, imgs, ?? ) . what should the third parameters look like ?

I ran the triplet_loss.py on market1501 as told in benchmark, but I got worse result that every item is about 5% lower than presented in bemchmark. BTW, I didn't change code. I do not understand why.Can you help me? Thanks!

Hello there. When I run the code: python examples/softmax_loss.py -d market1501 -b 64 -j 2 -a inception --logs-dir logs/softmax-loss/market1501-inception I get the following error. File "build/bdist.linux-x86_64/egg/reid/trainers.py", line 33, in train IndexError: invalid index of a 0-dim tensor. Use tensor.item() to convert a 0-dim tensor to a Python number

The full traceback is as follows: Market1501 dataset loaded subset | # ids | # images

train | 651 | 11387 val | 100 | 1549 trainval | 751 | 12936 query | 750 | 16483 gallery | 751 | 19281 /usr/local/lib/python2.7/dist-packages/open_reid-0.2.0-py2.7.egg/reid/models/inception.py:130: UserWarning: nn.init.kaiming_normal is now deprecated in favor of nn.init.kaiming_normal_. /usr/local/lib/python2.7/dist-packages/open_reid-0.2.0-py2.7.egg/reid/models/inception.py:134: UserWarning: nn.init.constant is now deprecated in favor of nn.init.constant_. /usr/local/lib/python2.7/dist-packages/open_reid-0.2.0-py2.7.egg/reid/models/inception.py:135: UserWarning: nn.init.constant is now deprecated in favor of nn.init.constant_. /usr/local/lib/python2.7/dist-packages/open_reid-0.2.0-py2.7.egg/reid/models/inception.py:137: UserWarning: nn.init.normal is now deprecated in favor of nn.init.normal_. /usr/local/lib/python2.7/dist-packages/open_reid-0.2.0-py2.7.egg/reid/models/inception.py:139: UserWarning: nn.init.constant is now deprecated in favor of nn.init.constant_. Traceback (most recent call last): File "examples/softmax_loss.py", line 217, in main(parser.parse_args()) File "examples/softmax_loss.py", line 147, in main trainer.train(epoch, train_loader, optimizer) File "build/bdist.linux-x86_64/egg/reid/trainers.py", line 33, in train IndexError: invalid index of a 0-dim tensor. Use tensor.item() to convert a 0-dim tensor to a Python number Any help will be appreciated

Hi, recently, the pytorch has updated and the code does not run smoothly in the new pytorch version. The following error occurs, could you please update a new version of the code ?

Traceback (most recent call last): File "examples/triplet_loss.py", line 217, in main(parser.parse_args()) File "examples/triplet_loss.py", line 145, in main trainer.train(epoch, train_loader, optimizer) File "build/bdist.linux-x86_64/egg/reid/trainers.py", line 31, in train File "build/bdist.linux-x86_64/egg/reid/trainers.py", line 80, in _forward File "/usr/local/lib/python2.7/dist-packages/torch/nn/modules/module.py", line 224, in call result = self.forward(*input, **kwargs) File "build/bdist.linux-x86_64/egg/reid/loss/triplet.py", line 26, in forward File "/usr/local/lib/python2.7/dist-packages/torch/autograd/variable.py", line 826, in rsub return SubConstant.apply(other, self) File "/usr/local/lib/python2.7/dist-packages/torch/autograd/functions/basic_ops.py", line 130, in forward return tensor.neg().add(constant) AttributeError: 'torch.cuda.ByteTensor' object has no attribute 'neg'

When I run the file examine.softmax_loss.py, I met the folowing problem:

from sklearn.utils.extmath import pinvh ImportError: cannot import name 'pinvh'

So how to solve it? Thanks

My dataset doesn't have camera lable. In order to satisfy the format of dataset, I assume all of them are from same camera (camera 0). And no problem happened during training. But error happened at evaluating, File "examples/softmax_loss.py", line 150, in main top1 = evaluator.evaluate(val_loader, dataset.val, dataset.val) File "/workspace/gjc/anaconda2/envs/py35/lib/python3.5/site-packages/open_reid-0.2.0-py3.5.egg/reid/evaluators.py", line 120, in evaluate File "/workspace/gjc/anaconda2/envs/py35/lib/python3.5/site-packages/open_reid-0.2.0-py3.5.egg/reid/evaluators.py", line 98, in evaluate_all File "/workspace/gjc/anaconda2/envs/py35/lib/python3.5/site-packages/open_reid-0.2.0-py3.5.egg/reid/evaluators.py", line 98, in <dictcomp> File "/workspace/gjc/anaconda2/envs/py35/lib/python3.5/site-packages/open_reid-0.2.0-py3.5.egg/reid/evaluation_metrics/ranking.py", line 78, in cmc RuntimeError: No valid query To simplify, I have deleted code about mAP and cuhk03.(Same error happened when I haven't modified code) Since I don't have camera lable, how should I evaluate the model in such situation? May anyone help me? @Cysu @zydou

Hi,

I have 3 questions here

1. Using the code provided here, I tested the market 1501 dataset using the following batch sizes (My 8Gb GPU card does not allow higher batch size)

batch size ( Rank1 ) 64 ( 77.4% ) 96 ( 79.5% ) 108 ( 79.8% )

Is this the expected results? Running a batch size of 256 will get me ~84% rank 1?

2. Also, there seems no different in the result whether image normalization is used or not. True?

3. I have memory leakage problem. Maximum epoch run before OS terminate the program is about 18-20, depending on batch size. Not sure if this is a problem of running Pytorch under Windows 10, or python code issues?

Thanks

I cloned the latest version open-reid (latest commit is a1df21b). First, I run the example code:

python examples/softmax_loss.py -d viper -b 64 -j 2 -a resnet50 --logs-dir logs/softmax-loss/viper-resnet50

The result is:

Mean AP: 15.5%
CMC Scores    allshots      cuhk03  market1501
  top-1           7.1%       12.2%        7.1%
  top-5          23.6%       35.6%       23.6%
  top-10         32.9%       47.3%       32.9%

Then, I run the same code again on the same machine:

python examples/softmax_loss.py -d viper -b 64 -j 2 -a resnet50 --logs-dir logs/softmax-loss/viper-resnet50

The result is:

Mean AP: 15.6%
CMC Scores    allshots      cuhk03  market1501
  top-1           7.9%       13.0%        7.9%
  top-5          20.9%       32.8%       20.9%
  top-10         30.9%       44.8%       30.9%

It's weird that they are different. It seems that these two lines are not work: https://github.com/Cysu/open-reid/blob/a1df21b00f9d3ecfce1329fef55af11f406c16a8/examples/softmax_loss.py#L71-L72 In Dataloader, train_transformer use RandomSizedRectCrop and RandomHorizontalFlip: https://github.com/Cysu/open-reid/blob/a1df21b00f9d3ecfce1329fef55af11f406c16a8/examples/softmax_loss.py#L36-L41 But RandomSizedRectCrop and RandomHorizontalFlip use python built-in random module other than numpy.random. https://github.com/Cysu/open-reid/blob/a1df21b00f9d3ecfce1329fef55af11f406c16a8/reid/utils/data/transforms.py#L19-L42

class RandomHorizontalFlip(object):
    """Horizontally flip the given PIL.Image randomly with a probability of 0.5."""

    def __call__(self, img):
        """
        Args:
            img (PIL.Image): Image to be flipped.
        Returns:
            PIL.Image: Randomly flipped image.
        """
        if random.random() < 0.5:
            return img.transpose(Image.FLIP_LEFT_RIGHT)
        return img

(Note: RandomHorizontalFlip source code at here)

So in examples/softmax_loss.py , I import random and change:

def main(args):
    np.random.seed(args.seed)
    torch.manual_seed(args.seed)

to:

def main(args):
    random.seed(args.seed)
    np.random.seed(args.seed)
    torch.manual_seed(args.seed)

Then I run the same example code twice. The results are still different. Then, in reid/utils/data/transforms.py, I change: https://github.com/Cysu/open-reid/blob/a1df21b00f9d3ecfce1329fef55af11f406c16a8/reid/utils/data/transforms.py#L26-L29 to

for attempt in range(10):
    area = img.size[0] * img.size[1]
    target_area = random.uniform(0.64, 1.0) * area
    print(target_area)
    aspect_ratio = random.uniform(2, 3)

Then run the example code twice. The target_area differ in first run and second run, indicating that random.seed(args.seed) is not work. So I rewrite the reid/utils/data/transforms.py with numpy.random. The final reid/utils/data/transforms.py is:

from __future__ import absolute_import

from torchvision.transforms import *
import numpy as np


class RandomHorizontalFlip(object):
    """Horizontally flip the given PIL.Image randomly with a probability of 0.5."""

    def __call__(self, img):
        """
        Args:
            img (PIL.Image): Image to be flipped.
        Returns:
            PIL.Image: Randomly flipped image.
        """
        if np.random.random() < 0.5:
            return img.transpose(Image.FLIP_LEFT_RIGHT)
        return img


class RectScale(object):
    def __init__(self, height, width, interpolation=Image.BILINEAR):
        self.height = height
        self.width = width
        self.interpolation = interpolation

    def __call__(self, img):
        w, h = img.size
        if h == self.height and w == self.width:
            return img
        return img.resize((self.width, self.height), self.interpolation)


class RandomSizedRectCrop(object):
    def __init__(self, height, width, interpolation=Image.BILINEAR):
        self.height = height
        self.width = width
        self.interpolation = interpolation

    def __call__(self, img):
        for attempt in range(10):
            area = img.size[0] * img.size[1]
            target_area = np.random.uniform(0.64, 1.0) * area
            print(target_area)
            aspect_ratio = np.random.uniform(2, 3)

            h = int(round(math.sqrt(target_area * aspect_ratio)))
            w = int(round(math.sqrt(target_area / aspect_ratio)))

            if w <= img.size[0] and h <= img.size[1]:
                x1 = np.random.randint(0, img.size[0] - w + 1)
                y1 = np.random.randint(0, img.size[1] - h + 1)

                img = img.crop((x1, y1, x1 + w, y1 + h))
                assert(img.size == (w, h))

                return img.resize((self.width, self.height), self.interpolation)

        # Fallback
        scale = RectScale(self.height, self.width,
                          interpolation=self.interpolation)
        return scale(img)

Then run the example code twice. The target_area is the same between first run and second run. But the final results (mAP, CMC) are still different. I'm wondering what's wrong with the code. Could you check the code and answer my quesion?

Hi, I'm trying to do a reid project with a new dataset. But the size of the images are different from each other. So what should I do to train with such dataset? Just convert it to the unified format? What should I do to split them into "trainval,gallery,query"? I only have one dataset with person id label. and camera id wasn't provided. And when I run the example in readme on a server with gpu, error happened. TypeError: sum received an invalid combination of arguments -got(dim=int, keepdim=bool) but expected one of no arguments *(int dim) It happened at reid/evaluation_metrics/classification.py line 17. I haven't solved it. But I have tried cpu version on my laptop by modifying some part of the code(delete cuda() and let pin_memory=False), then I got train data of epoch[0] like CMC score. Although I stopped training due to the bad performance of laptop, could the data printed verify that I can run with cpu? Will the change that I made in the code have influence on the training result? After training epoch[0],I got Mean AP:4.5% CMC Scores allshots cuhk03 market1501 top-1 0.5% 3.5% 0.5% top-5 5.0% 10.0% 5.0% top-10 10.5% 21.0% 10.5% It's quite low. My pc can't support continuing training. Will the result be better by training rest data? Or the bad result is due to my change to the code? Could someone tell me? Another problem is that how can I get the probability ranking of each image in the gallery if I use certain image as query?

Hi there,

I was going through the setup process (running 'setup.py'), and I came up with the following error:

Processing dependencies for open-reid==0.2.0 error: numpy 1.17.4 is installed but numpy<1.26.0,>=1.18.5 is required by {'scipy'}

I've checked my numpy version, and in fact, it didn't meet the requirements. So, I uninstalled numpy with pip and installed the 1.20.3 version.

When I try to run the 'setup.py 'again, the error persists.

Any ideas?

Thanks in advance.

When I use Oim Loss to train the model, In the fisrt several iterations, loss performs normally and reduce with the epoch increasing. But the loss suddenly became to 'NAN' and in the next iterations it was always 'NAN'. I check the data input and there is not 'nan' input. So, is this problem related to your Oim Loss method? Does anybody else meet this problem? I have run other two loss methods and they performed normally.

Hi since im running code on cpu windows 10 and disabled cuda() but still getting cuda error. here is the trace, please help me to resolve.

Traceback (most recent call last): File "examples/triplet_loss.py", line 220, in main(parser.parse_args()) File "examples/triplet_loss.py", line 144, in main trainer.train(epoch, train_loader, optimizer) File "C:\Users\MAHE\anaconda3\envs\open\lib\site-packages\open_reid-0.2.0-py3.6.egg\reid\trainers.py", line 30, in train File "C:\Users\MAHE\anaconda3\envs\open\lib\site-packages\open_reid-0.2.0-py3.6.egg\reid\trainers.py", line 66, in parse_data File "C:\Users\MAHE\anaconda3\envs\open\lib\site-packages\torch\cuda_init.py", line 161, in _lazy_init check_driver() File "C:\Users\MAHE\anaconda3\envs\open\lib\site-packages\torch\cuda_init.py", line 75, in _check_driver raise AssertionError("Torch not compiled with CUDA enabled") AssertionError: Torch not compiled with CUDA enabled

a = torch.tensor([[0.9, 0.01],[0.8,0.02],[-2.3, -2.4], [-2.4, -1.5]]) #gallery print(a.size()) b = torch.tensor([[2.1, 0.4],[-2.1,-3.2],[-2.2,-0.3]]) #query print(b.size()) dismat = torch.cdist(b, a) print(dismat) m,n=dismat.shape

Result torch.Size([4, 2]) torch.Size([3, 2]) tensor([[1.2618, 1.3544, 5.2154, 4.8847], [4.3936, 4.3334, 0.8246, 1.7263], [3.1155, 3.0170, 2.1024, 1.2166]]) torch.Size([3, 4])

import numpy as np gallery_ids= np.array([1,1,2,2]) #gallery query_ids = np.array([1,2,2]) #query

print(n[:, np.newaxis]) indices=np.argsort(dismat,axis=1) print(indices) print(n[indices]) matches = (gallery_ids[indices] == query_ids[:, np.newaxis]) print(matches)

Result [[1] [2] [2]] tensor([[0, 1, 3, 2], [2, 3, 1, 0], [3, 2, 1, 0]]) [[1 1 2 2] [2 2 1 1] [2 2 1 1]] [[ True True False False] [ True True False False] [ True True False False]]

It's strange when your algorithm detect that true for index [1][0],[1][1] and index [2][0],[2][1]? because the euclidan said that minimum distance is [1][2],[1][3] and index [2][2],[2][3] like my index that I made i just want to have an idea about your code in

Sort and find correct matches

indices = np.argsort(distmat, axis=1)
matches = (gallery_ids[indices] == query_ids[:, np.newaxis]) 

I test the examples/oim_loss.py with python examples/oim_loss.py -d cuhk01 -b 2 -j 0 -a resnet50 --logs-dir logs/softmax-loss/cuhk01-resnet50. And I got the model.module.num_features 2048 ,but when I run the resnet.py in your code with

model = create('resnet50', num_features=0, norm=True,dropout=0,num_classes=4) x = torch.rand(2, 3, 256,128) out = model(x) print("out.shape",out.shape) loss = OIMLoss(2048, 4) target = torch.empty(2, dtype=torch.long).random_(4) print(target) los, output = loss(out, target)

But got

File "F:/python_workspace/open-reid-master/reid/loss/oim.py", line 16, in forward outputs = inputs.mm(self.lut.t()) RuntimeError: size mismatch, m1: [2 x 4], m2: [2048 x 4]

How to deal with the problem?