Hi! Thanks for your amazing work. I try to achieve the same performance as the paper shown, but now have some problems.

1. In leaderboard/data/scenarios/town05_all_scenarios.json, I see that the last one has "scenario_type": "Scenario10". But in the evaluation log, I get these:

"meta": {
                "exceptions": [
                    [
                        "RouteScenario_16",
                        0,
                        "Failed - Agent got blocked"
                    ],
                    [
                        "RouteScenario_17",
                        1,
                        "Failed - Agent got blocked"
                    ],
                    [
                        "RouteScenario_18",
                        2,
                        "Failed - Agent got blocked"
                    ],
                    [
                        "RouteScenario_20",
                        4,
                        "Failed - Agent got blocked"
                    ],
                    [
                        "RouteScenario_21",
                        5,
                        "Failed - Agent got blocked"
                    ],
                    [
                        "RouteScenario_22",
                        6,
                        "Failed - Agent got blocked"
                    ],
                    [
                        "RouteScenario_23",
                        7,
                        "Failed - Agent got blocked"
                    ],
                    [
                        "RouteScenario_24",
                        8,
                        "Failed - Agent got blocked"
                    ],
                    [
                        "RouteScenario_25",
                        9,
                        "Failed - Agent got blocked"
                    ]
                ]
            },

I am wondering that what's the relationship between Scenario and RouteScenario?

2. I evaluated the pre-trained model and a model trained by myself without any modification on your code. The results are:

Result of model_ckpt/geometric_fusion (pretrained)

Result of model_ckpt/transfuser (pretrained)

Result of transfuser (self-trained)

I think there is something wrong, because they are significantly poorer than the results in your paper. Could you find any possible mistakes?

Evaluation script used for pre-trained transfuser model:

#!/bin/bash

export CARLA_ROOT=carla
export CARLA_SERVER=${CARLA_ROOT}/CarlaUE4.sh
export PYTHONPATH=$PYTHONPATH:${CARLA_ROOT}/PythonAPI
export PYTHONPATH=$PYTHONPATH:${CARLA_ROOT}/PythonAPI/carla
export PYTHONPATH=$PYTHONPATH:$CARLA_ROOT/PythonAPI/carla/dist/carla-0.9.10-py3.7-linux-x86_64.egg
export PYTHONPATH=$PYTHONPATH:leaderboard
export PYTHONPATH=$PYTHONPATH:leaderboard/team_code
export PYTHONPATH=$PYTHONPATH:scenario_runner

export LEADERBOARD_ROOT=leaderboard
export CHALLENGE_TRACK_CODENAME=SENSORS
export PORT=2000 # same as the carla server port
export TM_PORT=8000 # port for traffic manager, required when spawning multiple servers/clients
export DEBUG_CHALLENGE=0
export REPETITIONS=1 # multiple evaluation runs
export ROUTES=leaderboard/data/evaluation_routes/routes_town05_long.xml
export TEAM_AGENT=leaderboard/team_code/transfuser_agent.py
export TEAM_CONFIG=model_ckpt/transfuser
export CHECKPOINT_ENDPOINT=results/transfuser_result_1203_V2.json
export SCENARIOS=leaderboard/data/scenarios/town05_all_scenarios.json
export SAVE_PATH=data/expert_TF1203_V2 # path for saving episodes while evaluating
export RESUME=True


python3 ${LEADERBOARD_ROOT}/leaderboard/leaderboard_evaluator.py \
--scenarios=${SCENARIOS}  \
--routes=${ROUTES} \
--repetitions=${REPETITIONS} \
--track=${CHALLENGE_TRACK_CODENAME} \
--checkpoint=${CHECKPOINT_ENDPOINT} \
--agent=${TEAM_AGENT} \
--agent-config=${TEAM_CONFIG} \
--debug=${DEBUG_CHALLENGE} \
--record=${RECORD_PATH} \
--resume=${RESUME} \
--port=${PORT} \
--trafficManagerPort=${TM_PORT}

Hi, I want to ask you for help. When I run the run_evaluation.sh to evaluate the method, I just stuck in the step 'Loading the world' for a long time. When it finally displayed the frames, the screen is very slow and the script will interrupt when the ego-agent into the bend. The following is the fault code.

`========= Preparing RouteScenario_16 (repetition 0) =========

Setting up the agent routes_town05_long_03_17_20_35_11 Loading the world Skipping scenario 'Scenario4' due to setup error: list index out of range Running the route leaderboard/team_code/aim_agent.py:169: UserWarning: The given NumPy array is not writeable, and PyTorch does not support non-writeable tensors. This means you can write to the underlying (supposedly non-writeable) NumPy array using the tensor. You may want to copy the array to protect its data or make it writeable before converting it to a tensor. This type of warning will be suppressed for the rest of this program. (Triggered internally at /pytorch/torch/csrc/utils/tensor_numpy.cpp:180.) rgb = torch.from_numpy(scale_and_crop_image(Image.fromarray(tick_data['rgb']), scale=self.config.scale, crop=self.config.input_resolution)).unsqueeze(0) /home/watson/anaconda3/envs/carla10/lib/python3.7/site-packages/torch/nn/functional.py:718: UserWarning: Named tensors and all their associated APIs are an experimental feature and subject to change. Please do not use them for anything important until they are released as stable. (Triggered internally at /pytorch/c10/core/TensorImpl.h:1156.) return torch.max_pool2d(input, kernel_size, stride, padding, dilation, ceil_mode) /home/watson/methods/transfuser_1/leaderboard/scripts/run_evaluation.sh: line 38: 22229 Segmentation fault (core dumped) python3 ${LEADERBOARD_ROOT}/leaderboard/leaderboard_evaluator.py --scenarios=${SCENARIOS} --routes=${ROUTES} --repetitions=${REPETITIONS} --track=${CHALLENGE_TRACK_CODENAME} --checkpoint=${CHECKPOINT_ENDPOINT} --agent=${TEAM_AGENT} --agent-config=${TEAM_CONFIG} --debug=${DEBUG_CHALLENGE} --record=${RECORD_PATH} --resume=${RESUME} --port=${PORT} --trafficManagerPort=${TM_PORT} `

And this is the run_evaluation.sh I changed.

#!/bin/bash

export CARLA_ROOT=carla/CARLA_0.9.10.1 export CARLA_SERVER=${CARLA_ROOT}/CarlaUE4.sh export PYTHONPATH=$PYTHONPATH:${CARLA_ROOT}/PythonAPI export PYTHONPATH=$PYTHONPATH:${CARLA_ROOT}/PythonAPI/carla export PYTHONPATH=$PYTHONPATH:$CARLA_ROOT/PythonAPI/carla/dist/carla-0.9.10-py3.7-linux-x86_64.egg export PYTHONPATH=$PYTHONPATH:leaderboard export PYTHONPATH=$PYTHONPATH:leaderboard/team_code export PYTHONPATH=$PYTHONPATH:scenario_runner

export LEADERBOARD_ROOT=/home/watson/methods/transfuser_1/leaderboard export CHALLENGE_TRACK_CODENAME=SENSORS export PORT=2000 # same as the carla server port export TM_PORT=8000 # port for traffic manager, required when spawning multiple servers/clients export DEBUG_CHALLENGE=0 export REPETITIONS=1 # multiple evaluation runs export ROUTES=leaderboard/data/evaluation_routes/routes_town05_long.xml export TEAM_AGENT=leaderboard/team_code/aim_agent.py # agent export TEAM_CONFIG=/home/watson/methods/transfuser_1/aim/log/aim # model checkpoint, not required for expert export CHECKPOINT_ENDPOINT=results/aim_result.json # results file export SCENARIOS=leaderboard/data/scenarios/town05_all_scenarios.json export SAVE_PATH=data/expert # path for saving episodes while evaluating export RESUME=True

python3 ${LEADERBOARD_ROOT}/leaderboard/leaderboard_evaluator.py
--scenarios=${SCENARIOS}
--routes=${ROUTES}
--repetitions=${REPETITIONS}
--track=${CHALLENGE_TRACK_CODENAME}
--checkpoint=${CHECKPOINT_ENDPOINT}
--agent=${TEAM_AGENT}
--agent-config=${TEAM_CONFIG}
--debug=${DEBUG_CHALLENGE}
--record=${RECORD_PATH}
--resume=${RESUME}
--port=${PORT}
--trafficManagerPort=${TM_PORT}

i'm following the steps described here , yet im getting this when i try to evaluate , would you please help me understand ?

/leaderboard/leaderboard/leaderboard_evaluator_local.py:89: DeprecationWarning: distutils Ver sion classes are deprecated. Use packaging.version instead.
if LooseVersion(dist.version) < LooseVersion('0.9.10'):

Registering the global statistics

and then it exit execution

when i use ./local_evaluation.sh,it stops automatically. `89: DeprecationWarning: distutils Version classes are deprecated. Use packaging.version instead. if LooseVersion(dist.version) < LooseVersion('0.9.10'):

Registering the global statistics` and then it stops.

Hi! First of all I love the work you've with the TransFuser project, it's really quite interesting and I've had fun diving into autonomous driving research with CARLA thanks to you and your team, so thank you for that! Currently I am trying to replicate your results and also tune the hyperparameters of the transfuser model so I have a few questions with respect to that.

1. I see that in all of the train.py scripts for all of the models, the --epochs parameter is set to a default value of 101 except for CILRS which is set to 201, however in the supplementary material you mention that all of the models were trained for 101 epochs. Was the CILRS model really trained at 201 epochs or why is it the only model set to that default value?

2. In the README for each model you give an example of how to train that model, for example, CUDA_VISIBLE_DEVICES=<gpu_id> python3 train.py --id transfuser --batch_size 56 and CUDA_VISIBLE_DEVICES=<gpu_id> python3 train.py --id late_fusion --batch_size 128 I see that for different models you provide different values for --batch_size in those examples, however the default values are all set to 24 if I'm not mistaken. What batch size did you use for producing the results found in your main paper? Did you use the same values for all of the models or, like in the examples provided, use different values for different models?

3. During the training process, the model validates itself every 5 epochs, is this so that it can autostop if there's no improvement after a certain number of epochs or what is the reason for this validation? If this is the case, then there wouldn't be much of a need to tune the epochs parameters, correct?

4. From what I can tell, the main tunable parameters when calling the train.py script are --epochs, --lr (learning rate), and --batch_size. Do you have any suggestions/recommendations as to how I can go about trying to tune these? Also, are there any other parameters that I didn't mention that could also be fun to experiment with to try to optimize the transfuser model?

Why do I use the dataset （210G） you provided to train the model by myself, during test phase on the Carla , the ego car can't go straight on the road but just make a turn and run out of the road? In the training phase, we try to train 'only_wp'=1 and set --setting with both 'all' and '02_05_withheld'. I use one GPU 3090 to train the model and set epoch for 41, batch_size=32, other settings are default. I am very confused with this phenomenon and appreciate if you could provide some idea for me.

Hello! I have some problems about the data generation. My operation process is：

1. Running CARLA Server
2. ./leaderboard/scripts/run_evaluation.sh This seems that data generation is not successful. I would like to ask if the above operation is correct, and what should I do if there is a problem. And I understand that some images should be generated in the end, where will they be stored? I would be very grateful if you could reply.

I ran this set of codes and found that the lidar point cloud would flicker during the process of displaying the point cloud input. After careful exploration, it seems that the slow rotation frequency of the lidar sensor causes the input of the point cloud to alternate between one frame before and one frame after. I think this is wrong. In order to verify it, I tried without laser point cloud input. The effect of the network model does not seem to be affected, which is very unreasonable.I doubled the rotation frequency and this phenomenon will be corrected. Can you explain it to me? Did I make a mistake somewhere?

Hi When I trying to generate the data with different scenerios, I have some questions about the dataset.

1. I found the 14_weathers_minimal_data is divide by routes in differenct Town and one route includes kinds of scenarios. Could I divide the dataset by scenario? I mean that there is just one kind of scenario occur in one route. Does it will affect the result of training?
2. There are 3 types of routes: long/short/tiny. I would like to know how the training result will happen without 'long' routes. Or even just have 'tiny' routes, will it make a different to the result?

Looking forward to your help.

We report the mean and standard deviation over 9 runs of each method (3 training seeds, each trained model evaluated 3 times) on Route Completion (RC) and Driving Score (DS) in Town05 Short and Town05 Long settings with scenario and without scenarios. LF: Late Fusion, GF: Geometric Fusion, TF: TransFuser

Hello, I want to know what ‘3 training seeds’ mean, can you talk about it in detail?

Thanks for your great work. I have some problems about the auxiliary task. I don't find the auxiliary loss in your train.py of the transfuser. Could you point how to use the auxiliary task?

Hello

Thank you for your great repository. I was wondering which weathers did you use to train your model in the paper? Is it only clearNoon weather similar to validation dataset?

Thank You

I was trying to submit the transfuser model to the CARLA Leaderboard, and the submission says it was pending as it uses computes and never progresses. I tested the image on a local docker container and found that run_evaluation.sh times out as the client cannot connect to the server. Normally, I run the server on a different terminal. When I try to do this on the container, make_docker.sh does not copy the ./CarlaUE4.sh to the image, so we don't have a usual way to run the CARLA server on the container. Can someone lend more details on how to connect/run the server on the docker container, or is if someone faced a similar issue?