Hi,

Thanks for open-sourcing the project. I was trying this on a non gpu windows 10 machine (conda environment, python 3.7.9, pytorch 1.5) I was able to run Preprocess dataset, but got the bellow error while running Inference

(envs) Lenovo-PC MINGW64 /d/NLP/NL_to_SQL/gap-text2sql/rat-sql-gap (main) $ python run.py eval experiments/spider-configs/gap-run.jsonnet WARNING <class 'seq2struct.models.enc_dec.EncDecModel.Preproc'>: superfluous {'name': 'EncDec'} WARNING <class 'seq2struct.models.enc_dec.EncDecModel'>: superfluous {'decoder_preproc': {'grammar': {'clause_order': None, 'end_with_from': True, 'factorize_sketch': 2, 'include_literals': False, 'infer_from_conditions': True, 'name': 'spider', 'output_from': True, 'use_table_pointer': True}, 'save_path': 'data/spider-bart/nl2code-1115,output_from=true,fs=2,emb=bart,cvlink', 'use_seq_elem_rules': True}, 'encoder_preproc': {'bart_version': 'facebook/bart-large', 'compute_cv_link': True, 'compute_sc_link': True, 'db_path': 'data/spider-bart/database', 'fix_issue_16_primary_keys': True, 'include_table_name_in_column': False, 'save_path': 'data/spider-bart/nl2code-1115,output_from=true,fs=2,emb=bart,cvlink'}} Parameter containing: tensor([[-0.0370, 0.1117, 0.1829, ..., 0.2054, 0.0578, -0.0750], [ 0.0055, -0.0049, -0.0069, ..., -0.0030, 0.0038, 0.0087], [-0.0448, 0.4604, -0.0604, ..., 0.1073, 0.0310, 0.0477], ..., [-0.0138, 0.0278, -0.0467, ..., 0.0455, -0.0265, 0.0125], [-0.0043, 0.0153, -0.0567, ..., 0.0496, 0.0108, -0.0099], [ 0.0053, 0.0324, -0.0179, ..., -0.0085, 0.0223, -0.0020]], requires_grad=True) Updated the model with ./pretrained_checkpoint\pytorch_model.bin Parameter containing: tensor([[-0.0383, 0.1205, 0.1776, ..., 0.1973, 0.0594, -0.0699], [ 0.0046, -0.0023, -0.0084, ..., -0.0036, 0.0047, 0.0084], [-0.0460, 0.4671, -0.0650, ..., 0.1027, 0.0256, 0.0475], ..., [ 0.0086, 0.0037, 0.0363, ..., -0.0296, -0.0097, -0.0068], [-0.0160, 0.0123, 0.0015, ..., 0.0040, 0.0185, 0.0038], [-0.0049, -0.0121, -0.0235, ..., 0.0200, 0.0148, -0.0020]], requires_grad=True) Loading model from logdir/bart_run_1\bs=12,lr=1.0e-04,bert_lr=1.0e-05,end_lr=0e0,att=1\model_checkpoint-00041000 Traceback (most recent call last): File "run.py", line 104, in main() File "run.py", line 83, in main infer.main(infer_config) File "D:\NLP\NL_to_SQL\gap-text2sql\rat-sql-gap\seq2struct\commands\infer.py", line 215, in main model = inferer.load_model(args.logdir, args.step) File "D:\NLP\NL_to_SQL\gap-text2sql\rat-sql-gap\seq2struct\commands\infer.py", line 48, in load_model last_step = saver.restore(logdir, step=step, map_location=self.device, item_keys=["model"]) File "D:\NLP\NL_to_SQL\gap-text2sql\rat-sql-gap\seq2struct\utils\saver.py", line 122, in restore items2restore, model_dir, map_location, step) File "D:\NLP\NL_to_SQL\gap-text2sql\rat-sql-gap\seq2struct\utils\saver.py", line 40, in load_checkpoint item_dict[item_name].load_state_dict(checkpoint[item_name]) File "D:\NLP\NL_to_SQL\gap-text2sql\envs\lib\site-packages\torch\nn\modules\module.py", line 847, in load_state_dict self.class.name, "\n\t".join(error_msgs))) RuntimeError: Error(s) in loading state_dict for EncDecModel: size mismatch for decoder.rule_logits.2.weight: copying a param with shape torch.Size([97, 128]) from checkpoint, the shape in current model is torch.Size([94, 128]). size mismatch for decoder.rule_logits.2.bias: copying a param with shape torch.Size([97]) from checkpoint, the shape in current model is torch.Size([94]). size mismatch for decoder.rule_embedding.weight: copying a param with shape torch.Size([97, 128]) from checkpoint, the shape in current model is torch.Size([94, 128]). (envs) Lenovo-PC MINGW64 /d/NLP/NL_to_SQL/gap-text2sql/rat-sql-gap (main) $

Can you guide me, where I need to make changes.

Hi, I am trying to learn and test your work and strictly following your instructions, but I cannot quite access model checkpoint from aws s3 bucket; is your project closed now, or checkpoint bucket is not public, or am I having some other connectivity issues? Running the following command: aws s3 cp s3://gap-text2sql-public/checkpoint-artifacts/gap-finetuned-checkpoint logdir/bart_run_1/bs\=12\,lr\=1.0e-04\,bert_lr\=1.0e-05\,end_lr\=0e0\,att\=1/model_checkpoint-00041000 Getting this error: fatal error: An error occurred (403) when calling the HeadObject operation: Forbidden Would be grateful for feedback, Regards

Hey @Impavidity @TheurgicDuke771 , I'm facing the same issue, I think there is no issue with data( Spider) I'm using.

Loading model from logdir/bart_run_1/bs=12,lr=1.0e-04,bert_lr=1.0e-05,end_lr=0e0,att=1/model_checkpoint-00041000

---------------------------------------------------------------------------

KeyError                                  Traceback (most recent call last)

<ipython-input-45-2534e992a832> in <module>()
----> 1 model = inferer.load_model(model_dir, checkpoint_step)

6 frames

/content/gap-text2sql/rat-sql-gap/seq2struct/models/variational_lstm.py in _hook_remove_dropout_masks_from_state_dict(cls, instance, state_dict, prefix, local_metadata)
     75     @classmethod
     76     def _hook_remove_dropout_masks_from_state_dict(cls, instance, state_dict, prefix, local_metadata):
---> 77         del state_dict[prefix + '_input_dropout_mask']
     78         del state_dict[prefix + '_h_dropout_mask']
     79 

KeyError: 'decoder.state_update._input_dropout_mask'

We can see the folder structure below,

Please guide me to tackle this issue. Thanks in advance.

I've tried to retrain your model and managed to get the same scores as you report. What I don't understand is:

• there is not early stopping
• the model used is the last saved model checkpoint, when the model is with huge overfit on train (very close to 0 loss)

However, I've tried to get scores on another model checkpoint with a lower loss in validation and it yielded worse EXACT matches. Therefore, why does it yield better scores if we overfit the training set?

Hello,

I am going through your setup instructions in the README, and for downloading the dataset there's this command: bash data/spider-20190205/generate.sh ./spider, which is meant to be run from the rat-sql-gap dir, I believe, however there is no directory containing this generation script, and I cannot find this script anywhere.

Please let me know if I am missing something obvious.

Thanks!

Hi. @Impavidity @pnpnpn, I'm referring to your code for my text-to-SQL study. To modify the GAP encoder part, I tuned relogic/tabart-pretraining.py. In order to execute the code, AverageSpanExtractor module is needed. But I can't find this module from relogic.logickit.modules.span_extractors.average_span_extractor. Is the module missing from github?

Best regard!

For every model I train, including the pretrained model, EXEC scores are zero. does this mean that none of the generated queries matches the gold SQL query when runned to produce an output table?

Resolve the issue #5 on supporting to run inference on own databases and queries.

• Add related dependency for supporting the functionality.
• Create an easy-to-use notebook.
• Add a database example.
• Update readme.

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

Resolve the issue #1 on missing the data preprocess script.

• Add data preprocess script and its dependency.
• Clean up unused dependencies and configuration file.

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

Hi,

I am facing some issue on identity the "ON clause" from output,

Question: List the pilots are from london Result: SELECT pilot.Name FROM pilot JOIN match WHERE match.Location = 'terminal'

Is there any solution to find the on clauses?

Thanks in advance ​

I am new to this, I would like to test Text 2 SQL with my own database.

Could anyone share me the steps to convert new database and execute the questions?

Thanks in advance.

While executing a natural language query "Birth year of Gina Rinehart", we got the following sql query output. The output sql query is okay, but whenever we are calling any variable data(in this case, our variable is 'Gina Rinehart'), we are getting "terminal"

Natural language query from spider's singer database: Birth year of Gina Rinehart. Corresponding output SQL query: SELECT singer.Birth_Year FROM singer WHERE singer.Name = 'terminal'

I find that the number of the grammar rules is 94 when I follow the instructions to preprocess the data (with the hyper-parameter fs=2). But the size of rule_embedding in the pre-trained model is 97.

Traceback (most recent call last): File "run.py", line 104, in main() File "run.py", line 83, in main infer.main(infer_config) File "/home/yuhao/zhenwen/repair_model/gap-text2sql-main/rat-sql-gap/seq2struct/commands/infer.py", line 239, in main model = inferer.load_model(args.logdir, args.step) File "/home/yuhao/zhenwen/repair_model/gap-text2sql-main/rat-sql-gap/seq2struct/commands/infer.py", line 48, in load_model last_step = saver.restore(logdir, step=step, map_location=self.device, item_keys=["model"]) File "/home/yuhao/zhenwen/repair_model/gap-text2sql-main/rat-sql-gap/seq2struct/utils/saver.py", line 122, in restore items2restore, model_dir, map_location, step) File "/home/yuhao/zhenwen/repair_model/gap-text2sql-main/rat-sql-gap/seq2struct/utils/saver.py", line 40, in load_checkpoint item_dict[item_name].load_state_dict(checkpoint[item_name]) File "/home/yuhao/.conda/envs/zhenwen/lib/python3.7/site-packages/torch/nn/modules/module.py", line 1407, in load_state_dict self.class.name, "\n\t".join(error_msgs))) RuntimeError: Error(s) in loading state_dict for EncDecModel: Unexpected key(s) in state_dict: "decoder.state_update._input_dropout_mask", "decoder.state_update._h_dropout_mask". size mismatch for decoder.rule_logits.2.weight: copying a param with shape torch.Size([97, 128]) from checkpoint, the shape in current model is torch.Size([94, 128]). size mismatch for decoder.rule_logits.2.bias: copying a param with shape torch.Size([97]) from checkpoint, the shape in current model is torch.Size([94]). size mismatch for decoder.rule_embedding.weight: copying a param with shape torch.Size([97, 128]) from checkpoint, the shape in current model is torch.Size([94, 128])

Can you please let me know how to download the additional 30K data [(Utterance, Schema, SQL) triples] that was used for pretraining?

Thanks in Advanced!