This PR adds support for custom x-axes to the TensorBoard scalar plot. The axis names are specified in the tensorboard output constructor.

class TensorBoardOutput(LogOutput):
    def __init__(self,
                 log_dir,
                 x_axes=None,
                 flush_secs=120,
                 histogram_samples=1e3): 

When x_axes is None, it falls back to use iterations as the x-axis. If any x_axis is not present in the scalar tabular. A warning will be logged to the console. If all x_axes are not present, it falls back to use iteration as the x-axis.

Screenshots of an experiment with Epoch and TotalEnvSteps as x-axes.

I will open a separate PR in garage repo to set TotalEnvSteps as the default x-axis.

The current issue #45 is caused by inconsistent keys in tabularInput with csvOutput headers. To fix this:

1. Read the existing log file
2. Update the csv Dictwriter with the new header using union

• Union of tabularInput's and csvOutput's header is used to make sure all keys from both instances are captured

3. Write the same log file with new key headers and old data. If the value of new key is missing, the cell is left blank.

The cons of this solution:

• The csv log file is read and write to update the inconsistent key headers. It may be time-consuming to handle a large number of csv log files because of the heavy I/O. To ensure the I/O is as efficient as possible, all file reading and writing is done in RAM and file is closed upon used.

A better approach can be:

• Expand the header on-the-go as a new key(s) is encountered. Write the log file with the new header exactly once.

rlworkgroup#45 @avnishn @zequnyu

Travis keeps several packages like numpy pre-installed to speed up builds but this sometimes leads to wrong version being picked up by pip.

This change would ignore the pre-installed versions

Unfortunately, fixing the tests involves monkey patching unittest, since it has bug. That bug is fixed in CPython PR #4800, which has gone unmerged for over a year.

This attempts to allow a more robust handling of dynamic fileds in module dowel. This is implemented by the following logic:

• The CSV headers are now ordered
• When a new field is encountered, it is appended to the ordered list of headers
• The output files gets flushed and re-read to allow header replacement
• The new records with new fields get written

In order to accompanish this, these following changes to internal data structures and logic are added:

• csv._fieldnames becomes list() instead of set()
• The output file is now opened in w+ mode to allow both read and write access

Unit tests were added for the following cases:

• Adding a new field at the end of the list of fields
• Adding a new field in the middle of the list of fields

Sincerely

Currently, CsvOutput emits a warning if the keys of a TabularInput change after the first call to logger.log(TabularInput). A new key not seen before will be ignored and an old key not presented will be left blank. In other words, CsvOutput conservatively handles dynamic fieldnames.

This behaviour of CsvOutput makes it tricky to log performance of Multi- and Meta- ML algorithms, where there are usually per-task fields but not every task is presented in every iteration, resulting in missing of logs for some tasks.

The desired behaviour to handle inconsistent keys should be

• When a new key is encountered
  • Expand header with the new key.
  • Expand old rows with empty cells for the new key.
• If the value of any key is missing, leave the cell blank.

The previous automatic versioning script was flawed. It produced the correct package version for documentation builds and building PyPI distributions, but produced an incorrect version when you run setup.py from the downloaded package. Unfortunately, Python environment managers (e.g. Pipenv, conda) resolve package version by evaluating setup.py, not using the PyPI version.

This PR makes version generation simpler by reading the version string from a simple file. Automatic versioning from tags is handled by clobbering the version file from within the CI, rather than looking for a CI environment variable on every usage.

This was accidentally included during the import from rlworkgroup/garage.

The Snapshotter is not part of the Logger API, so it doesn't really belong in this package.

This PR moves unit tests modules to tests/dowel. This makes it so that the unit tests paths for a module are easy to predict. For instance, the tests for the module dowel.csv_output will now live at tests.dowel.test_csv_output rather than tests.test_csv_output.

Inconsistent header keys are handled. When a new key is introduced, the previous data is augmented line by line. In the logger, if data is a TabularInput instance, then it has its values emptied, with its keys remaining so that there is no data bleed when a key is omitted in the future. Tensorboard incompatibility is an issue as tensorboard does not accept the empty character (or string) as a legal numpy scalar.

Nine tests have been provided to cover the various circumstances:

1. No change in keys
2. Single increase in keys with consistent future usage
3. Multiple increase in keys with consistent future usage
4. Single increase in keys with inconsistent future usage
5. Multiple increase in keys with inconsistent future usage
6. Overlapping increase in keys with immediate inconsistency
7. Static keys - tensorboard incompatibility test
8. Dynamic keys - tensorboard incompatibility test
9. Empty tabulation test

Note: I have left the comment structure to be consistent with the preexisting code.

Fix for #45

Used DictReader to read in old file values and created new DictWriter object to rewrite all records. Old records without values for these new keys will be null as required.

Also, apologies for the unnecessary mentions to this issue earlier!

@avnishn @zequnyu

Introduction

Dowel is a tool that the garage Team uses for logging results from our various Reinforcement learning experiments.

Dowel can be used to log different types of data such as floats or strings. The logs can be logged to stdout (the console), CSV files, and Tensorboard.

You can check out an example of how Dowel is used here. In fact, almost all parts of the Dowel API are used in this example.

The problem

After statistics such as loss have been logged, and a call to logger.dump_all() is made for the first time, new tabular data can’t be written to a CSV output. This is because currently data cannot be inconsistently logged to CSV, meaning that on every single call to dump_all, the same logger keys must appear. Data that is inconsistently logged will not appear in the CSV output. This is a design flaw that we have been able to work around but affects our workflows.

Your goal is to solve the problem as well as introduce tests into our testing framework in order to verify your solution.

Some General Instructions

1. Fork Dowel and install all necessary dependencies.
2. Take a look at this toy example which when run exposes the bug and the accompanying issue mentioned above.
3. When you have finished writing your solution and tests, upload a PR onto your fork, not onto the upstream repository.
4. When you are done email us back with the link to your pull request.
5. Follow the rules of the contributing.md.

If you have any questions, open an issue in your fork, and tag @avnishn and @haydenshively. Our preferred mode of communication on any questions that you have is through github issues and pull requests, as this is how the Garage team communicates generally. For this reason, we won’t respond to any direct emails with regards to help with your project. We will however respond to any other questions that you have via email (interview scheduling, etc).

Best of luck, and let us know if there are any issues as early on as possible

When I tried to run the following commands from the "Git recipes" in CONTRIBUTING.md, I got error messages:

git remote add rlworkgroup [email protected]:rlworkgroup/dowel.git

git reset --hard master rlworkgroup/master

However, the following would work:

git remote add rlworkgroup https://github.com/rlworkgroup/dowel.git

git checkout master
git fetch rlworkgroup
git reset --hard rlworkgroup/master

Should CONTRIBUTING.md be updated?

Hi,

Thank you for this nice a simple tool for logging machine learning research. I often encounter situations where I would like to save multi-dimensional Numpy arrays. For example, the observation at each time-step in a reinforcement learning experiment.

It would be nice to have an output logger that supports Numpy arrays, Pytorch Tensors and Tensorflow Tensors.

I have written a simple output logger, NpzOutput, that writes Numpy arrays to a .npz file using Numpys savez functions. It is not optimal (no incremental saving), but thought I share it in case somebody is interested.

This is because it attempts to close the underlying TensorboardX writer in TensorboardOutput.__del__. However, global teardown of the python interpreter has already closed the thread used by TensorboardX.

Tables with keys of varying lengths are hard to read, since some of the key names end up far from their values. This change adds a sequence of dots on all odd lines, so that lines are easier to match up with their keys.