• Avoid removing white spaces from paragraph otherwise the answer will be changed
  • Avoid using strip method
  • Replace \n to (space) in paragraph
• Check test cases passed

I could successfully install chazutsu in python2.7.10. But there is a problem.

when I import chazutsu, I have a following issue:

>>> import chazutsu
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/Users/xxx/VirtualEnv/py2venv/lib/python2.7/site-packages/chazutsu/__init__.py", line 1, in <module>
    import chazutsu.datasets as datasets
  File "/Users/xxx/VirtualEnv/py2venv/lib/python2.7/site-packages/chazutsu/datasets/__init__.py", line 1, in <module>
    from .movie_review import MovieReview
  File "/Users/xxx/VirtualEnv/py2venv/lib/python2.7/site-packages/chazutsu/datasets/movie_review.py", line 5, in <module>
    from chazutsu.datasets.framework.dataset import Dataset
  File "/Users/xxx/VirtualEnv/py2venv/lib/python2.7/site-packages/chazutsu/datasets/framework/dataset.py", line 10, in <module>
    from urllib.parse import urlparse
ImportError: No module named parse

Maybe, it's due to the specification change of urllib between python2.x and python3.x.

CustomerReview class has class methods products5, additional9, more3.

But Datasets README.md shows CustomerReview has 5products, 9additional, 3more. https://github.com/chakki-works/chazutsu/blob/master/chazutsu/datasets/README.md#customer-review-datasets

I think it is better to match documents with code. How about?

The language model data is not a kind of the format of X and y. The data is sequencial and the label data is given by shifting it. So to handle the language model data, some trick is needed to change X, y to X_t, X_t+1.

Patching CVE-2007-4559

Hi, we are security researchers from the Advanced Research Center at Trellix. We have began a campaign to patch a widespread bug named CVE-2007-4559. CVE-2007-4559 is a 15 year old bug in the Python tarfile package. By using extract() or extractall() on a tarfile object without sanitizing input, a maliciously crafted .tar file could perform a directory path traversal attack. We found at least one unsantized extractall() in your codebase and are providing a patch for you via pull request. The patch essentially checks to see if all tarfile members will be extracted safely and throws an exception otherwise. We encourage you to use this patch or your own solution to secure against CVE-2007-4559. Further technical information about the vulnerability can be found in this blog.

If you have further questions you may contact us through this projects lead researcher Kasimir Schulz.

This PR aims to improve the reliability of the test test_movie_review.py by changing the encoding method in chazutsu/datasets/framework/resource.py, so that the reading function won't run into error when reading ’. The error can be reproduced when running for multiple times. E.g. touch run.py and with in this python file write: import os for i in range(100): os.system('pytest tests/test_movie_review.py >> repeat_100_result.log')