Thanks for the great work and making it open source. I was trying to set up the code and follow this example, however, while setting it up, I am encountering this error:

---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
<ipython-input-12-2ebbdaa05245> in <module>()
      1 # Loading substitute generator
      2 sg = SubstituteGenerator.from_config(
----> 3     str(CONFIGS_PATH / "subst_generators" / "lexsub" / "xlnet_embs.jsonnet")
      4 )

19 frames
/content/LexSubGen/lexsubgen/subst_generator.py in from_config(cls, config_path)
     96             object of the SubstituteGenerator class.
     97         """
---> 98         subst_generator, _ = build_from_config_path(config_path)
     99         return subst_generator
    100 

/content/LexSubGen/lexsubgen/utils/params.py in build_from_config_path(config_path, config)
    194     params = Params(config)
    195 
--> 196     return build_from_params(params), config
    197 
    198 

/content/LexSubGen/lexsubgen/utils/params.py in build_from_params(params)
    236                 item = build_from_params(item_params)
    237             elif isinstance(item_params, list):
--> 238                 item = [build_from_params(elem_params) for elem_params in item_params]
    239             else:
    240                 item = item_params

/content/LexSubGen/lexsubgen/utils/params.py in <listcomp>(.0)
    236                 item = build_from_params(item_params)
    237             elif isinstance(item_params, list):
--> 238                 item = [build_from_params(elem_params) for elem_params in item_params]
    239             else:
    240                 item = item_params

/content/LexSubGen/lexsubgen/utils/params.py in build_from_params(params)
    226     if "class_name" in params:
    227         cls_name = params.pop("class_name")
--> 228         cls = clsname2cls(cls_name)
    229 
    230         # init params acquisition

/content/LexSubGen/lexsubgen/utils/params.py in clsname2cls(clsname)
    203         import_path += "." + module_path
    204     # try:
--> 205     module = importlib.import_module(import_path)
    206     cls = getattr(module, clsname)
    207     # except Exception as e:

/usr/lib/python3.7/importlib/__init__.py in import_module(name, package)
    125                 break
    126             level += 1
--> 127     return _bootstrap._gcd_import(name[level:], package, level)
    128 
    129 

/usr/lib/python3.7/importlib/_bootstrap.py in _gcd_import(name, package, level)

/usr/lib/python3.7/importlib/_bootstrap.py in _find_and_load(name, import_)

/usr/lib/python3.7/importlib/_bootstrap.py in _find_and_load_unlocked(name, import_)

/usr/lib/python3.7/importlib/_bootstrap.py in _load_unlocked(spec)

/usr/lib/python3.7/importlib/_bootstrap_external.py in exec_module(self, module)

/usr/lib/python3.7/importlib/_bootstrap.py in _call_with_frames_removed(f, *args, **kwds)

/content/LexSubGen/lexsubgen/post_processors/target_excluder.py in <module>()
      8 
      9 
---> 10 class TargetExcluder(PostProcessor):
     11     def __init__(self, lemmatizer: Optional[str] = None, use_pos_tag: bool = True):
     12         """

/content/LexSubGen/lexsubgen/post_processors/target_excluder.py in TargetExcluder()
     30         target_words: List[str],
     31         target_pos: Optional[List[str]] = None,
---> 32     ) -> Tuple[np.ndarray, Dict[str, int]]:
     33         """
     34         Abstract method that transforms prob estimator predictions.

/usr/local/lib/python3.7/dist-packages/overrides/overrides.py in overrides(method, check_signature, check_at_runtime)
     86     """
     87     if method is not None:
---> 88         return _overrides(method, check_signature, check_at_runtime)
     89     else:
     90         return functools.partial(

/usr/local/lib/python3.7/dist-packages/overrides/overrides.py in _overrides(method, check_signature, check_at_runtime)
    112                 return wrapper  # type: ignore
    113             else:
--> 114                 _validate_method(method, super_class, check_signature)
    115                 return method
    116     raise TypeError(f"{method.__qualname__}: No super class method found")

/usr/local/lib/python3.7/dist-packages/overrides/overrides.py in _validate_method(method, super_class, check_signature)
    133         and not isinstance(super_method, property)
    134     ):
--> 135         ensure_signature_is_compatible(super_method, method, is_static)
    136 
    137 

/usr/local/lib/python3.7/dist-packages/overrides/signature.py in ensure_signature_is_compatible(super_callable, sub_callable, is_static)
     93         ensure_return_type_compatibility(super_type_hints, sub_type_hints, method_name)
     94         ensure_all_kwargs_defined_in_sub(
---> 95             super_sig, sub_sig, super_type_hints, sub_type_hints, is_static, method_name
     96         )
     97         ensure_all_positional_args_defined_in_sub(

/usr/local/lib/python3.7/dist-packages/overrides/signature.py in ensure_all_kwargs_defined_in_sub(super_sig, sub_sig, super_type_hints, sub_type_hints, check_first_parameter, method_name)
    154             ):
    155                 raise TypeError(
--> 156                     f"`{method_name}: {name} must be a supertype of `{super_param.annotation}` but is `{sub_param.annotation}`"
    157                 )
    158 

TypeError: `TargetExcluder.transform: target_words must be a supertype of `typing.Union[typing.List[str], NoneType]` but is `typing.List[str]`

I made sure that the environment and versions are the same as suggested in the repo. Any help would be appreciated. Thanks

In the example listed here, is it possible to use other models from the huggingface models hub to generate lexical substitutes? I am happy to contribute more models to the repo once I understand the pipeline of adding new models.

Also, which approach from the paper does this example correspond to? Is it the XLNet+embs approach listed in bold in this table from the paper?

Patching CVE-2007-4559

Hi, we are security researchers from the Advanced Research Center at Trellix. We have began a campaign to patch a widespread bug named CVE-2007-4559. CVE-2007-4559 is a 15 year old bug in the Python tarfile package. By using extract() or extractall() on a tarfile object without sanitizing input, a maliciously crafted .tar file could perform a directory path traversal attack. We found at least one unsantized extractall() in your codebase and are providing a patch for you via pull request. The patch essentially checks to see if all tarfile members will be extracted safely and throws an exception otherwise. We encourage you to use this patch or your own solution to secure against CVE-2007-4559. Further technical information about the vulnerability can be found in this blog.

If you have further questions you may contact us through this projects lead researcher Kasimir Schulz.

https://github.com/Samsung/LexSubGen/blob/d3d06b40dcc41475957bccf2dd6b4cd08373e181/lexsubgen/metrics/candidate_ranking_metrics.py#L113

Could you please explain why we don't need to sort values here to calculate GAP? I believe we need, but it assumed that we give an input dict with the right order of insertion and it works properly. I think it's better to mention this fact in the annotation or to sort values.

Anyway, thanks for the great results and the interesting article!