Hey! I am trying to download the checkpoint for the trained lm (in the description you say it is under gs://piglet/checkpoints/language_model/model.ckpt-20000). However, this is not a valid path; I downloaded the contents under gs://piglet/checkpoints/language_model and I could only find the *.index, *.data-00000-of-000001 and *.meta files.

Please lmk if I am missing something or the files are provided somewhere else. Essentially I want to experiment with the predict_statechange part of the project and different ways to fine-tune in the NLU task, so I would prefer to avoid re-training the entire LM. I am also trying to run the finetune_sc.py file, would it be possible to point me to the yaml config file? I found two versions (flagship_configs and symbol2text_configs), but I am not sure about the paths or what some of the params mean ("train_time_symbolic_actions_prob", "fuse_action",). Thank you!

Hi Rowan, thanks for the great work! just wondering if you could provide a more detailed readme on the .tfrecord data used for training the dynamic model, i.e., what are the meanings of each field, for example where can I find the mapping from actions/action_id to action names; what does the ints in the meta means; etc. Below is one example instance in the .tfrecord file:

{   'actions/action_args': array([1, 0]),
    'actions/action_id': array([13]),
    'actions/action_success': array([1]),
    'agent_state': array([  1.        ,   0.        ,   0.90099925,  -0.75      ,
       180.        ,  30.000004  ], dtype=float32),
    'comparison_labels': array([1, 2]),
    'frames/encoded': array([255, 216, 255, ..., 143, 255, 217], dtype=uint8),
    'frames/format': array([106, 112, 101, 103], dtype=uint8),
    'frames/height': array([384]),
    'frames/num_frames': array([2]),
    'frames/width': array([640]),
    'meta': array([123,  34, 115,  99, 101, 110, 101,  95, 110,  97, 109, 101,  34,
        58,  32,  34,  70, 108, 111, 111, 114,  80, 108,  97, 110,  50,
        95, 112, 104, 121, 115, 105,  99, 115,  34,  44,  32,  34, 116,
       101, 120, 116,  34,  58,  32,  34,  70, 105, 108, 108,  32,  36,
        49,  32, 119, 105, 116, 104,  32, 108, 105, 113, 117, 105, 100,
        32, 117, 115, 105, 110, 103,  32,  36,  50,  44,  32, 116, 104,
       101, 110,  32, 101, 109, 112, 116, 121,  32, 105, 116,  46,  34,
        44,  32,  34, 109,  97, 105, 110,  95, 111,  98, 106, 101,  99,
       116,  95, 105, 100, 115,  34,  58,  32,  91,  34,  80, 111, 116,
       124,  43,  48,  48,  46,  56,  57, 124,  43,  48,  48,  46,  57,
        48, 124,  45,  48,  49,  46,  52,  49,  34,  44,  32,  34,  70,
        97, 117,  99, 101, 116, 124,  45,  48,  48,  46,  48,  50, 124,
        43,  48,  49,  46,  49,  52, 124,  45,  48,  49,  46,  54,  49,
        34,  44,  32,  34,  83, 105, 110, 107, 124,  43,  48,  48,  46,
        48,  48, 124,  43,  48,  48,  46,  56,  57, 124,  45,  48,  49,
        46,  52,  52, 124,  83, 105, 110, 107,  66,  97, 115, 105, 110,
        34,  93,  44,  32,  34, 119, 105, 100, 116, 104,  34,  58,  32,
        54,  52,  48,  44,  32,  34, 104, 101, 105, 103, 104, 116,  34,
        58,  32,  51,  56,  52,  44,  32,  34, 116,  97, 115, 107,  95,
       110,  97, 109, 101,  34,  58,  32,  34,  95, 102, 105, 108, 108,
        95, 111,  98, 106, 101,  99, 116, 120,  95, 119, 105, 116, 104,
        95, 108, 105, 113, 117, 105, 100,  34,  44,  32,  34, 102, 110,
        34,  58,  32,  34,  47, 104, 111, 109, 101,  47, 114, 111, 119,
        97, 110,  47, 100,  97, 116,  97, 115, 101, 116, 115,  51,  47,
       105, 112, 107,  45, 118,  49,  47, 102, 105, 108, 108,  95, 111,
        98, 106, 101,  99, 116, 120,  95, 119, 105, 116, 104,  95, 108,
       105, 113, 117, 105, 100,  47, 100,  56, 111,  57, 112, 104,  88,
        48,  53,  68, 117, 100,  46, 104,  53,  34, 125], dtype=uint8),
    'objects/ObjectTemperature': array([1, 1, 1, 1]),
    'objects/breakable': array([0, 0, 0, 0]),
    'objects/canBeUsedUp': array([0, 0, 0, 0]),
    'objects/canFillWithLiquid': array([1, 1, 1, 1]),
    'objects/cookable': array([0, 0, 0, 0]),
    'objects/dirtyable': array([1, 1, 0, 0]),
    'objects/distance': array([2, 1, 2, 2]),
    'objects/isBroken': array([0, 0, 0, 0]),
    'objects/isCooked': array([0, 0, 0, 0]),
    'objects/isDirty': array([0, 0, 0, 0]),
    'objects/isFilledWithLiquid': array([1, 1, 0, 0]),
    'objects/isOpen': array([0, 0, 0, 0]),
    'objects/isPickedUp': array([0, 1, 0, 0]),
    'objects/isSliced': array([0, 0, 0, 0]),
    'objects/isToggled': array([0, 0, 0, 0]),
    'objects/isUsedUp': array([0, 0, 0, 0]),
    'objects/mass': array([4, 4, 0, 0]),
    'objects/moveable': array([0, 0, 0, 0]),
    'objects/object_types': array([79, 79, 94, 94]),
    'objects/openable': array([0, 0, 0, 0]),
    'objects/parentReceptacles': array([94,  0,  0,  0]),
    'objects/pickupable': array([1, 1, 0, 0]),
    'objects/receptacle': array([1, 1, 1, 1]),
    'objects/receptacleObjectIds': array([ 0,  0, 79,  0]),
    'objects/salientMaterials_Ceramic': array([0, 0, 0, 0]),
    'objects/salientMaterials_Fabric': array([0, 0, 0, 0]),
    'objects/salientMaterials_Food': array([0, 0, 0, 0]),
    'objects/salientMaterials_Glass': array([0, 0, 0, 0]),
    'objects/salientMaterials_Leather': array([0, 0, 0, 0]),
    'objects/salientMaterials_Metal': array([1, 1, 0, 0]),
    'objects/salientMaterials_Organic': array([0, 0, 0, 0]),
    'objects/salientMaterials_Paper': array([0, 0, 0, 0]),
    'objects/salientMaterials_Plastic': array([0, 0, 0, 0]),
    'objects/salientMaterials_Rubber': array([0, 0, 0, 0]),
    'objects/salientMaterials_Soap': array([0, 0, 0, 0]),
    'objects/salientMaterials_Sponge': array([0, 0, 0, 0]),
    'objects/salientMaterials_Stone': array([0, 0, 0, 0]),
    'objects/salientMaterials_Wax': array([0, 0, 0, 0]),
    'objects/salientMaterials_Wood': array([0, 0, 0, 0]),
    'objects/size': array([4, 4, 5, 5]),
    'objects/sliceable': array([0, 0, 0, 0]),
    'objects/toggleable': array([0, 0, 0, 0])}

Thanks!

Hello, thank you for the amazing work!

So, if I want to download tfrecords, do I have to pay some money for them? Or is it free, but the reason why 'requester pays' is on is because otherwise google cloud would not allow to upload these files into their system?

Hello, thank you for the great work!

I have downloaded the annotations.jsonl and physical-interaction-tfrecords. However it is not clear to me how can I find the mapping between the examples in annotations.jsonl and the tfrecords in physical-interaction-tfrecords.

Specifically, what I need is to find the corresponding image frames (pre-condition and post-condition) for the each example in the annotation file. Could you help me with this?

Thank you!