Thank you for the impressive large-scale indoor dataset collecting work. It's a significant dataset with many possibilities for high-level application scenarios, and I like it.

It is thoughtful to generate ground truth high-resolution depth maps by discarding geometry which a direct line-of-sight from the novel viewpoint cannot be guaranteed.

But I think if we can get access to raw Faro high-resolution XYZRGB point cloud, it will grant the dataset more potential, such as point cloud completion task.

Will it be possible for us to get access to much more raw data collected in your well-designed data collecting process? We can explore more meaningful settings to explore 3D understanding in such a large-scale real indoor dataset.

I am trying to extract frames from the Raw dataset and running into trouble/confusion related to the orientation of the images which vary between Portrait and Landscape modes from video to video.

Here are a few questions I have on this topic.

1. Is there any annotation or way of determining what the correct orientation is for the Raw images/annotations/intrinsic? Most seem to be rotated by -90 degrees but not all as far as I can tell. It seems that videos in Landscape mode are mostly not rotated but could occasionally be upside down in my tests.

2. Are the Raw videos always in the "correct" orientation? They seem to be at a glance, so I have assumed this for now.

3. Is it known whether the camera operators switch between Landscape and Portrait modes in the middle of a video? If it's not known, then was it an intention?

Thank you and sorry if this is covered somewhere in the code that I missed.

Thank you for the large-scale indoor dataset. I noticed that only 1/6 images have pose parameters in 'lowres_wide.traj' file. Are there more posed images? Does there exist 'vga_wide.traj' files?

Dear ARKitScenes developers,

Thanks for the excellent work! I am particularly interested in using high-resolution depth maps in the raw dataset. The ones in the upsampling subset are too sparse for my use case, unfortunately.

However, after I downloaded one of the sequences in the raw dataset using the command below: python3 download_data.py raw --split Training --video_id 41048190 --download_dir /tmp/Datasets_ssd/ I couldn't find highres_depth. Could you please advise if I missed anything?

On a separate note, I also wonder if there are any plans to release the point clouds captured by the Faro laser scanner and how you computed the confidence for depth maps if it could be disclosed.

Hi, thanks for the great work again! I am trying to download the mesh file of each scan produced by ARKit (the low-resolution one). Unfortunately, I encounter 403 forbidden when downloading these files (limit the download file list to mesh only), and I worry about data loss. The following image shows some of my logs.

Hi,

Thanks for releasing this dataset. Since the highres depth map is too sparse to me, I'm trying to re-rendering highres depth map using the given camera parameters and mesh. I suppose the '.traj' contains extrsincs of each frames. Just wondering what is the orientation of raotation and translation, does that represent x,y,z or other coordinate system? which axis face the front of the camera? What is the near and far distance of each depth map?

Best,

Hi, thanks for your great work! I'm working on a data collection APP with ARKit. I notice that you estimate the ground truth poses instead of using ARCamera poses (ARCamera.transform). Why? Is it because the ARCamera poses are inaccurate?

Hello, this is really a nice work, However, as far as I know, the lidar equipped in Apple device can only acquire 9X64 points at a time. So I wonder how can you acquire the depth map in real-time? Is it generated by fusing the depth information from the lidar sensor and other information(such as RGB and IMU) through "sceneDepth" API?

Thanks for releasing this fruitful dataset. I am trying to train a 2D object detector. How can I get the 2D BBOX? What is "urc" coordinate that you are using in "data_prepare_online.py"?

Hi, @PeterZheFu

i download the upsampling dataset, there are 4 types assets, confidence, lowres_depth, highres_depth and wide_rgb.

The highres_depth is the ground-truth or the result of upsampling network(MSG/MSPF) ?

If i want to train the network , how to generate the metadata.csv file ? what does this file mean ?

Hi, when i try to download mesh files

python3 download_data.py raw --split Validation --video_id 48458667 --download_dir . --raw_dataset_assets mesh

i always got errors like that

<?xml version="1.0" encoding="UTF-8"?>
<Error><Code>AccessDenied</Code><Message>Access Denied</Message><RequestId>S35X8MRZXWAHDNCD</RequestId><HostId>RUdvpESfdl/jZ20rR9qMA67MmnhqACL1JTE6CXWbyn1qXnqby7vU6nYra1HBRIUX2ypgy2mxLfM=</HostId></Error>

Hi,

Thank you for sharing the dataset. In section 3.2 of the paper, you have metioned that watertight textured mesh could be generated by stereographic projection. Cloud you share these meshes or related code please?

• wide and wide_intrinsics are missing from the default assets list, hence they are not downloaded when downloading the raw datasets. Added these to the list

Since I found missing objects in the object annotation files, I suggest to start a thread were we collect such issues

In the sets 40776203 and 40776204 (Training) the bed in the corner is not labeled. It has the following transformation:

• in 40776203: center=[ 3.087, -1.471, -1.220 ], dimension=[ 2.320, 1.680, 0.461], rot=[-0.527, 0, 0]
• in 40776204: center=[ 0.585, -1.064, -1.141 ], dimension=[ 2.320, 1.680, 0.461], rot=[ 1.043, 0, 0]

Hello,

It looks like the 3D bounding boxes can only be used on the ARKit reconstructions. Could you provide the transformation between this mesh and the point clouds, so that the bounding boxes can be used on the point cloud as well? Could you also explain the difference between data['segments']['obbAligned'] and data['segments']['obb'] in the annotations file?

Thanks!

Hi team,

Thanks for releasing the Faro point clouds. It looks like some scenes such as 421006 have 2 copies of the point cloud, each containing 4 scans -

• 169952, 169953, 169955, 169962
• 170029, 170030, 170034, 170036

A visualization -

Additionally some scenes like 422013 have 2 rooms that are far apart -

Can you let us know how to handle these?

Hello, Thank you for your sharing! Could you please provide some details about this dataset, such as :

1. Whether the lowres_wide.traj can be used as the extrinsic of the images in vga_wide and ultra_wide dictionary...
2. The total size of raw data using the script download_data.py, it seemed over 7TB...