In the last step, is it necessary to artificially refine the gt grasp data set for generation? If so, what tools are used to quickly mark up positive and negative samples?

What means by grasp_height and grasp_width?

I read the source code models/graspnet.py:79,butI don't understand what grasp_height and grasp_depth mean.

I understand other parameters. For example, grasp_width is the gripper's gripping width.

thanks

Hi everyone. I am using your work in order to build a vision-based grasping system for my master thesis. By running the demo.py and by observing the grasp score, there are grasps that have a score greater than 1, but from your paper, I would have expected a max grasp score of 1. So, why are there grasps scores greater than 1? Thanks.

您好，我在尝试使用多显卡训练的时候，设置参数如下： os.environ["CUDA_VISIBLE_DEVICES"] = '0,3' net = nn.DataParallel(net.cuda()) 但是模型加载数据的时候出现错误： RuntimeError: Caught RuntimeError in replica 0 on device 0. 请问作者在多显卡训练的时候是否遇到过这个问题？

您好，我电脑上有两张显卡，一张2080Ti，一张1080，驱动为NVIDIA-SMI 450.66，CUDA10.1

我想在1080上跑测试，于是我将command_test.sh中的CUDA_VISIBLE_DEVICES=0改为了CUDA_VISIBLE_DEVICES=1

但当我运行sh command_test.sh时，会报如下错误：

CUDA kernel failed : no kernel image is available for execution on the device void furthest_point_sampling_kernel_wrapper(int, int, int, const float*, float*, int*) at L:233 in /home/agent/grasp/graspnet-baseline/pointnet2/_ext_src/src/sampling_gpu.cu

该错误发生的位置大概是inference()中的

for batch_idx, batch_data in enumerate(TEST_DATALOADER)

请问是什么原因导致的呢？

I tried to use sh command_test.sh to test a trained model, but it reported an error in loading dexnet model, as shown in the following. I would like to inquire if you have any ideas to solve this problem. Thanks very much.

Loading data path and collision labels...: 100%|████████████████████████| 90/90 [00:00<00:00, 219.96it/s]
23040
23040
-> loaded checkpoint logs/log_rs/checkpoint.tar (epoch: 12)
Loading data path...: 100%|█████████████████████████████████████████████| 90/90 [00:00<00:00, 205.90it/s]
multiprocessing.pool.RemoteTraceback: 
"""
Traceback (most recent call last):
  File "/home/amax/miniconda3/envs/py37/lib/python3.7/multiprocessing/pool.py", line 121, in worker
    result = (True, func(*args, **kwds))
  File "/home/amax/zzhaoao/Grasp/graspnetAPI/graspnetAPI/graspnet_eval.py", line 121, in eval_scene
    model_list, dexmodel_list, _ = self.get_scene_models(scene_id, ann_id=0)
  File "/home/amax/zzhaoao/Grasp/graspnetAPI/graspnetAPI/graspnet_eval.py", line 52, in get_scene_models
    dexmodel = pickle.load(f)
AttributeError: 'ColorVisuals' object has no attribute 'crc'
"""

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "test.py", line 119, in <module>
    evaluate()
  File "test.py", line 113, in evaluate
    res, ap = ge.eval_all(cfgs.dump_dir, proc=cfgs.num_workers)
  File "/home/amax/zzhaoao/Grasp/graspnetAPI/graspnetAPI/graspnet_eval.py", line 305, in eval_all
    res = np.array(self.parallel_eval_scenes(scene_ids = list(range(100, 190)), dump_folder = dump_folder, proc = proc))
  File "/home/amax/zzhaoao/Grasp/graspnetAPI/graspnetAPI/graspnet_eval.py", line 231, in parallel_eval_scenes
    scene_acc_list.append(res.get())
  File "/home/amax/miniconda3/envs/py37/lib/python3.7/multiprocessing/pool.py", line 657, in get
    raise self._value
AttributeError: 'ColorVisuals' object has no attribute 'crc'

@Fang-Haoshu @chenxi-wang Thanks for this publication as well as the open source code.

I have a question reagarding the resulting grasps. Using the pretrained weights and the demo code, the network will sometimes not return any grasps on the object: Do you have any idea how to make it work in this scenario?

Additionally, I do not understand how to use the grasp depth and height. I looked at this issue, but still do not get what this information represents.

Using a parallel gripper, shouldn't the translation and rotation be enough to determine where to grasp the object? I have implemented your code on a Universal Robots manipulator mounted with Realsense d435 and OnRobot parallel gripper, and when sending the TCP to the grasp translation and rotation, it always seem to be a few cm short of gripping the object correct. Could this be related to not using the depth and height information?

Best regards, Emil Holst

Great article! I managed to get the code up and running and I am considering using it for my bachelor thesis in robotics. My problem is that i cannot get nearly as good point-clouds as you have in your demo data. Did you use any filters or fused multi-view point clouds etc.? E.g. this Turns into this:

Hi, thanks for sharing your work. It really facilitates robotic grasping. I am using your code recently in the gazebo. I found that the generated grasp points are on the edge of the table or inserts into the body of the can as in the figure below. The elevation of the camera is set to 45 degrees in my case. Could I ask if the graspnet is more suitable for top-down view( nearly 90 degrees to be perpendicular to the table) or top-down grasping?

Hi, I encountered an error as follows:

import numpy as np
data_root = '/media/wind/Share/DataSet/GraspNet1Billion'
collision_label = np.load(os.path.join(data_root, 'collision_label/scene_0110/collision_labels.npz'))
print(collision_label.files)
print(collision_label['arr_8'].shape)

Output: ['arr_0', 'arr_4', 'arr_5', 'arr_2', 'arr_3', 'arr_7', 'arr_1', 'arr_8', 'arr_6'] OSError Traceback (most recent call last) in () 3 collision_label = np.load(os.path.join(data_root, 'collision_label/scene_0110/collision_labels.npz')) 4 print(collision_label.files) ----> 5 print(collision_label['arr_8'].shape)

/home/wind/anaconda3/envs/pytorch160/lib/python3.7/site-packages/numpy/lib/npyio.py in getitem(self, key) 253 return format.read_array(bytes, 254 allow_pickle=self.allow_pickle, --> 255 pickle_kwargs=self.pickle_kwargs) 256 else: 257 return self.zip.read(key)

/home/wind/anaconda3/envs/pytorch160/lib/python3.7/site-packages/numpy/lib/format.py in read_array(fp, allow_pickle, pickle_kwargs) 761 read_count = min(max_read_count, count - i) 762 read_size = int(read_count * dtype.itemsize) --> 763 data = _read_bytes(fp, read_size, "array data") 764 array[i:i+read_count] = numpy.frombuffer(data, dtype=dtype, 765 count=read_count)

/home/wind/anaconda3/envs/pytorch160/lib/python3.7/site-packages/numpy/lib/format.py in _read_bytes(fp, size, error_template) 890 # done about that. note that regular files can't be non-blocking 891 try: --> 892 r = fp.read(size - len(data)) 893 data += r 894 if len(r) == 0 or len(data) == size:

/home/wind/anaconda3/envs/pytorch160/lib/python3.7/zipfile.py in read(self, n) 897 self._offset = 0 898 while n > 0 and not self._eof: --> 899 data = self._read1(n) 900 if n < len(data): 901 self._readbuffer = data

/home/wind/anaconda3/envs/pytorch160/lib/python3.7/zipfile.py in _read1(self, n) 967 data += self._read2(n - len(data)) 968 else: --> 969 data = self._read2(n) 970 971 if self._compress_type == ZIP_STORED:

/home/wind/anaconda3/envs/pytorch160/lib/python3.7/zipfile.py in _read2(self, n) 997 n = min(n, self._compress_left) 998 --> 999 data = self._fileobj.read(n) 1000 self._compress_left -= len(data) 1001 if not data:

/home/wind/anaconda3/envs/pytorch160/lib/python3.7/zipfile.py in read(self, n) 740 "Close the writing handle before trying to read.") 741 self._file.seek(self._pos) --> 742 data = self._file.read(n) 743 self._pos = self._file.tell() 744 return data

OSError: [Errno 5] Input/output error

Hello @Fang-Haoshu.

Would you mind give me a quick explain about the meaning of each field inside your meta.mat file.

{'__header__': b'MATLAB 5.0 MAT-file Platform: posix, Created on: Tue Mar 17 14:51:33 2020',
 '__version__': '1.0',
 '__globals__': [],
 'poses': array([[[-0.9756758 ,  0.20997444,  0.71809477, -0.10539012,
           0.28258908,  0.9570219 , -0.6063459 ,  0.6341392 ,
          -0.95235777],
         [-0.05635871,  0.7769375 , -0.6932231 , -0.9939491 ,
          -0.95306253, -0.1432863 , -0.11288764,  0.7661504 ,
          -0.22686528],
         [ 0.21185002, -0.59353083, -0.06149539, -0.0309534 ,
          -0.10869765, -0.25214702,  0.7871474 ,  0.10431238,
          -0.20383038],
         [ 0.1117    , -0.0714    , -0.0837    , -0.1092    ,
           0.0027    , -0.2043    , -0.0069    , -0.1581    ,
           0.0391    ]],
 
        [[ 0.21893798,  0.14432675, -0.64941216,  0.9167016 ,
           0.8435303 , -0.24861187,  0.7650126 , -0.66297054,
          -0.28011078],
         [-0.2993843 , -0.6250445 , -0.6992277 , -0.08504111,
           0.30085424,  0.04236081, -0.3529656 ,  0.6081769 ,
           0.915004  ],
         [ 0.9286739 , -0.76713043,  0.2989055 , -0.39041796,
          -0.44490823, -0.96767646,  0.53867525, -0.4365672 ,
           0.29035428],
         [ 0.0326    ,  0.0548    , -0.0858    ,  0.135     ,
          -0.0408    ,  0.0057    , -0.1437    , -0.0485    ,
           0.0349    ]],
 
        [[ 0.0110857 , -0.9669956 , -0.25020745,  0.38542327,
           0.45672753,  0.14933594,  0.21702617, -0.39791653,
           0.1206343 ],
         [ 0.95246667,  0.07541542, -0.17470661, -0.06952123,
           0.03403645,  0.9887743 ,  0.9288012 ,  0.20768833,
           0.33361623],
         [ 0.30444106, -0.24337623, -0.95229924,  0.9201172 ,
           0.8889553 ,  0.00491755,  0.30037972,  0.8936039 ,
          -0.9349586 ],
         [ 0.4197    ,  0.473     ,  0.4442    ,  0.5089    ,
           0.4402    ,  0.471     ,  0.3932    ,  0.4459    ,
           0.4803    ]]], dtype=float32),
 'cls_indexes': array([[ 1,  3,  6,  9, 23, 27, 52, 62, 39]]),
 'intrinsic_matrix': array([[631.54864502,   0.        , 638.43517329],
        [  0.        , 631.20751953, 366.49904066],
        [  0.        ,   0.        ,   1.        ]]),
 'factor_depth': array([[1000.]])}

Thanks

Thanks for your great job.Your annotation method is Brilliant.

I noticed your annotation method including two main parts:

1. 6D pose annotation
2. Grasp pose annotation

May I ask could you please provide detailed tools for annotating the second part: Grasp pose ?Cause we are helpless for this part annotation. Thanks very much.