Upon running python lstm_server.py -dir data I get the error below. I'm running Python 2.7.15 in a virtualenv and I haven't altered any files or directories save for creating /data and downloading the two corpus packs. Any help would be greatly appreciated, thank you :)

  File "lstm_server.py", line 169, in <module>
    app.add_api('lstm_server.yaml')
  File "/Users/pkarpenko/Documents/Dev/python/LSTMVis/venv/lib/python2.7/site-packages/connexion/app.py", line 168, in add_api
    validator_map=self.validator_map)
  File "/Users/pkarpenko/Documents/Dev/python/LSTMVis/venv/lib/python2.7/site-packages/connexion/api.py", line 108, in __init__
    validate_spec(spec)
  File "/Users/pkarpenko/Documents/Dev/python/LSTMVis/venv/lib/python2.7/site-packages/swagger_spec_validator/validator20.py", line 97, in validate_spec
    validate_apis(apis, bound_deref)
  File "/Users/pkarpenko/Documents/Dev/python/LSTMVis/venv/lib/python2.7/site-packages/swagger_spec_validator/validator20.py", line 310, in validate_apis
    idx=idx,
  File "/Users/pkarpenko/Documents/Dev/python/LSTMVis/venv/lib/python2.7/site-packages/swagger_spec_validator/validator20.py", line 243, in validate_parameter
    validate_default_in_parameter(param, deref)
  File "/Users/pkarpenko/Documents/Dev/python/LSTMVis/venv/lib/python2.7/site-packages/swagger_spec_validator/validator20.py", line 172, in validate_default_in_parameter
    deref=deref,
  File "/Users/pkarpenko/Documents/Dev/python/LSTMVis/venv/lib/python2.7/site-packages/swagger_spec_validator/common.py", line 29, in wrapper
    sys.exc_info()[2])
  File "/Users/pkarpenko/Documents/Dev/python/LSTMVis/venv/lib/python2.7/site-packages/swagger_spec_validator/common.py", line 24, in wrapper
    return method(*args, **kwargs)
  File "/Users/pkarpenko/Documents/Dev/python/LSTMVis/venv/lib/python2.7/site-packages/swagger_spec_validator/validator20.py", line 155, in validate_value_type
    validate_schema_value(schema=deref(schema), value=value, swagger_resolver=swagger_resolver)
  File "/Users/pkarpenko/Documents/Dev/python/LSTMVis/venv/lib/python2.7/site-packages/swagger_spec_validator/ref_validators.py", line 101, in validate_schema_value
    create_dereffing_validator(swagger_resolver)(schema, resolver=swagger_resolver).validate(value)
  File "/Users/pkarpenko/Documents/Dev/python/LSTMVis/venv/lib/python2.7/site-packages/jsonschema/validators.py", line 130, in validate
    raise error
swagger_spec_validator.common.SwaggerValidationError: ('\'states,words\' is not of type \'array\'\n\nFailed validating \'type\' in schema:\n    {\'collectionFormat\': \'csv\',\n     \'default\': \'states,words\',\n     \'description\': "list of required data dimensions, like state values (\'states\'), token values (\'words\'),\\n  or meta information (\'meta_XYZ\')\\n",\n     \'in\': \'query\',\n     \'items\': {\'type\': \'string\'},\n     \'name\': \'dims\',\n     \'required\': False,\n     \'type\': \'array\'}\n\nOn instance:\n    \'states,words\'', <ValidationError: "'states,words' is not of type 'array'">)

Dear developers, I ran the "install" instructions from the main page and downloaded the parens data as a test. However, I get the following error, indicating a missing module:

Traceback (most recent call last): File "lstm_server.py", line 8, in <module> from lstmdata.data_handler import LSTMDataHandler File "/home/dnieuwenhuijse/Software/LSTMVis/lstmdata/data_handler.py", line 11, in <module> import helper_functions as hf ModuleNotFoundError: No module named 'helper_functions'

I couldn't find the module on your github repo either, so I am guessing that you have removed the module from your repo by accident?

Kind regards,

David

Getting the above error when I attempt to execute get_states.lua. This is on my own data. Here is what I do before executing get_states.lua First, I download the tiny-Shakespeare dataset in jcjohson's github repo, torch-rnn; and split it into 2 datasets: train_tiny-shakespeare.txt and validation_tiny-shakespeare.txt.

Then, I run preprocess like so: python model/preprocess.py data/tinyshakespeare/train_tiny-shakespeare.txt
data/tinyshakespeare/validation_tiny-shakespeare.txt 50 64
data/tinyshakespeare/convert/tiny-shakespeare

Then, I run main.lua to train on this data, like so: th model/main.lua -rnn_size 128 -word_vec_size 64-num_layers 2
-epochs 50 -data_file data/tinyshakespeare/convert/tiny-shakespeare.hdf5
-val_data_file data/tinyshakespeare/convert/tiny-shakespeareval.hdf5
-gpuid 0 -savefile cv/tinyshakespeare
| tee train-tinyshakespeare.log

Then, I run get_states.lua like so: th model/get_states.lua
-data_file data/tinyshakespeare/convert/tiny-shakespeare.hdf5
-checkpoint_file cv/tinyshakespeare_epoch20.00_420.16.t7
-output_file data/reads/tinyshakespeare_states.h5

This is where I get the error.

1. data in /LSTMVis/lstmdata/05childbook
2. starting server with.... root@b6d19b96355f:/LSTMVis# python server.py -dir /LSTMVis/lstmdata/05childbook File "server.py", line 267 print args ^ SyntaxError: Missing parentheses in call to 'print'

Any ideas what I'm missing? @HendrikStrobelt Thx!

Hi All,

Came across this project and it looks really helpful. I'm looking to use it to explore LSTM in a current project.

I'm a bit confused about the states.h5 input file. I imagine that the state vector from each timestep is the h_t output of the LSTM, for each t (as in the notation of this LSTM description). Is this correct?

If so, the states are dependent on the input sequence. So I can put any sequence into the LSTM and then get these hidden states, but wouldn't this tool also need the states from all of my other input sequences for some of the analysis? Or does it only look at one input at a time? Is there an easy way to swap between or compare activations across input sequences?

Thanks for the clarification, and for contributing this tool!

I have a tagging task where each token has a bunch of annotations associated with it (the most important being the predicted and actual class). It would be great if the annotations could be shown below the main visualization. This ways, I could more easily see, e.g., which neurons lead to a wrong decision.

I generated the three files required for running the lstm server. But the main thing which I don't find in the documentation of the tool is what exactly is the path mentioned in lstm yaml file. I'm constantly getting errors in the line path!

The image below shows my yaml file:

Running the server shows the following error:

My network only consists of 1 hidden layer. I'm not sure whether I understood that part of visualization right!

Later I commented out those two lines in states and ran the server again.

which shows:

Could someone help me out as I don't find enough documentation on what does path exactly meant to do here?

When I run this execution: python lstm_server.py -dir data

Getting this error below. Ideas on how to fix?

Traceback (most recent call last):
  File "lstm_server.py", line 8, in <module>
    from lstmdata.data_handler import LSTMDataHandler
  File "/home/user/Documents/GitHub/LSTMVis/lstmdata/data_handler.py", line 11, in <module>
    import helper_functions as hf
ModuleNotFoundError: No module named 'helper_functions'

If you do character-wise prediction and not word-wise prediction, you might want to predict whitespaces. But the way the words.dict file is parsed right now crashes on whitespaces. This pull request should fix this.

Hey guys, Fantastic project!

As part of my research, I am looking into extending your platform outside of the NLP domain. In other words, I would like to be able to explore the activation states:

• for n-dimensional vectors rather than word ids
• with respect to the predicted and target value
• of any RNN

Would you be interested in collaborating on something of the kind?

Small changes to allow moving to Python3.7 (Python2.7 is already depricated in most environments).

• Requirements adjusted to latest version, and to include "swagger-ui-bundle" per warning thrown at startup.
• The previous version of the project.config included nested dict_items structures which are not directly serializable in Python3 so a workaround was introduced.
• Calls to iteritems() were replaced by items() for Python3
• maps are no longer passed around due to differences between Python2/Python3. Instead we use list(map())
• Localhost should be "127.0.0.1" for WSL on PC. Not sure if this breaks running on a mac.

https://github.com/HendrikStrobelt/LSTMVis/issues/42 "Parens Dataset - 10k small - 0.03 GB" https://drive.google.com/file/d/0B3yX0IkfCkLvWUowazhOZHFuSms/view?usp=sharing This dataset is not accessible to my google drive account. When I click the file sharing link, it says that I have no access to this file.

However, "Children Book - Gutenberg" https://drive.google.com/file/d/0B542UFSlrvMjMHcxWHluNzh3clU/view?usp=sharing I have access to this file. The scale of Children Book is too large for a simple trying.

Bumps pyyaml from 5.3.1 to 5.4.

Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting @dependabot rebase.

Excuse me, i ran the command:pip install connexion and download this package. But when i use python2 to import connexion， i found that

File "lstm_server.py", line 2, in import connexion File "C:\Users\10259.conda\envs\python27\lib\site-packages\connexion_init_.py", line 5, in File "C:\Users\10259.conda\envs\python27\lib\site-packages\connexion\apis_init_.py", line 1, in from .abstract import AbstractAPI # NOQA File "C:\Users\10259.conda\envs\python27\lib\site-packages\connexion\apis\abstract.py", line 32 class AbstractAPI(metaclass=AbstractAPIMeta): ^ SyntaxError: invalid syntax

I really want to know how to solve this problem. Thanks!

Not sure if this is addressed in the doc or elsewhere; I'll close this as soon as possible if I found out...

I did some development recently with OpenNMT. Is there a way to use this visualization tool for OpenNMT models? Especially since both OpenNMT and LSTMVi are developed by the same lab?

Thanks in advance!