Hi, Thanks for the updates. Still unable to run evaluation script.

Traceback (most recent call last):
  File "eval_math_gpt.py", line 373, in <module>
    run_eval(args)
  File "eval_math_gpt.py", line 174, in run_eval
    output_ids = model.generate(
  File "/home/sahmed9/anaconda3/envs/env2020/lib/python3.8/site-packages/torch/nn/modules/module.py", line 947, in __getattr__
    raise AttributeError("'{}' object has no attribute '{}'".format(
AttributeError: 'GPT2LMHeadModel' object has no attribute 'generate'

Hi, similar to previous :

Traceback (most recent call last):
  File "eval_math_gpt.py", line 37, in <module>
    from dataset.deepmind import DeepMindMathDataset
ModuleNotFoundError: No module named 'dataset.deepmind'

On commenting out the same next error :

Traceback (most recent call last):
  File "eval_math_gpt.py", line 360, in <module>
    parser.add_argument('--arch', default='gpt2', choices=transformers.GPT2_PRETRAINED_MODEL_ARCHIVE_LIST)
AttributeError: module 'transformers' has no attribute 'GPT2_PRETRAINED_MODEL_ARCHIVE_LIST'

I try to show the dataset in a pdf file, and most of the latex files are good to transfer. But there are still some latex typos in:

1. test/intermediate_algebra/44.json

A URL appears to be mistakenly included in a JSON file, like

    "solution": "... the diagram,\n\nhttp://aops-classroom.s3.amazonaws.com/Algebra3/Algebra3_Putnam1958_Morning1.png\n\n ..."

Is it possible to correct this into a parsable form?

2. test/prealgebra/1117.json

"solution": "A $60 coat with a 20$\\%$ discount

3. test/number_theory/407.json something like:

6 3 _ _ _ _

4. ./test/prealgebra/1645.json

times $24.50 per square yard, or $\boxed{735}$ dollars.

For pre-training on amps where are the files "no_steps_flist_relative.txt" and "with_steps_flist_relative.txt" .

Are they the concatenation of all *.txt in folder "data_file_lists" ?

There is a missing swap file "make_flists.py.swp" in the mathematical root folder. If possible please share the same?

for eg. is this what is expected :

filenames = ['no_steps_flist_relative_algebra.txt', 'no_steps_flist_relative_calculus.txt', 'no_steps_flist_relative_counting_and_statistics.txt', 'no_steps_flist_relative_geometry.txt', 'no_steps_flist_relative_linear_algebra.txt', 'no_steps_flist_relative_number_theory.txt']
with open('./no_steps_flist_relative.txt', 'w') as outfile:
    for fname in filenames:
        with open(fname) as infile:
            for line in infile:
                outfile.write(line)

Traceback (most recent call last):
  File "eval_math_gpt.py", line 36, in <module>
    from dataset.aops import AOPSMathDataset
ModuleNotFoundError: No module named 'dataset.aops'

Which file is this ??

I plan to submit a new metric to Hugging Face that will compute accuracy treating equivalent answers (as defined by is_equiv) as correct. Without this change, Hugging Face users would have to download the math directory and add it to their PYTHONPATH before using the new metric. Instead, they'll be able to install the package from github using pip.

As part of this change, I've also:

• Added a line to README with install instructions
• Renamed equivalent.py to math_equivalence.py (math_equivalence is better as a package name than equivalent)
• Prefixed the name of private functions with an underscore in math_equivalence.py
• Moved tests to their own file

Hi, sorry im new in this field.

!python tune_gpt.py --khan-dataroot /content/amps/khan/ --save-dir /content/drive/MyDrive/model/

when i using the above code on google colap, i got error

Traceback (most recent call last): File "tune_gpt.py", line 333, in main() File "tune_gpt.py", line 318, in main train_data = get_dataset(args) File "tune_gpt.py", line 239, in get_dataset len_multiplier, dirname = args.khan_dataroot.split("@") ValueError: not enough values to unpack (expected 2, got 1)

How to fix this?

What is the default value used for the tokenizer-merges-file ?

Do you use the default merges_gpt2.txt or the custom digits removed file merges_gpt2_single_digit_numbers.txt

My understanding is that the file merges.txt is build during the training of the BBPE (Byte Level BPE) tokenizer on the corpus: it gets a new entry (line) at each iteration of the tokenizer to find the byte pairs most frequent.

How di you verify for this design decision? I understand the need for the "clean" merges file, but using a new merges file with pre-trained weights, wouldn't that be an error ? Since now there are tokens missing as compared to what the gpt2 was trained with?

Or should one re-train the tokenizer itself, on the current dataset vocab ?