After doing import gtn I get the following import error:

File "<stdin>", line 1, in <module>

  File "/home/meow/.local/lib/python3.9/site-packages/gtn/__init__.py", line 13, in <module>
    from ._graph import *
ImportError: /home/meow/miniconda3/bin/../lib/libstdc++.so.6: version `GLIBCXX_3.4.29' not found (required by /home/meow/.local/lib/python3.9/site-packages/gtn/_graph.cpython-39-x86_64-linux-gnu.so)

Hi,

I'm failing the installation in both approaches. I have g++ version 7.5.0 and CMake version 3.23.0

pip install gtn gives:

Collecting gtn
  Using cached gtn-0.0.1.tar.gz (14 kB)
Building wheels for collected packages: gtn
  Building wheel for gtn (setup.py) ... error
  ERROR: Command errored out with exit status 1:
   command: /home/pyp/miniconda3/bin/python -u -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/tmp/pip-install-p97r2dzi/gtn/setup.py'"'"'; __file__='"'"'/tmp/pip-install-p97r2dzi/gtn/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' bdist_wheel -d /tmp/pip-wheel-7wzo73o6
       cwd: /tmp/pip-install-p97r2dzi/gtn/
  Complete output (49 lines):
  running bdist_wheel
  running build
  running build_py
  creating build
  creating build/lib.linux-x86_64-3.8
  creating build/lib.linux-x86_64-3.8/gtn
  copying src/bindings/python/gtn/__init__.py -> build/lib.linux-x86_64-3.8/gtn
  creating build/lib.linux-x86_64-3.8/gtn/criterion
  copying src/bindings/python/gtn/criterion/__init__.py -> build/lib.linux-x86_64-3.8/gtn/criterion
  running build_ext
  CMake Warning:
    Ignoring extra path from command line:
  
     "/tmp/pip-install-p97r2dzi/gtn/src"
  
  
  CMake Error: The source directory "/tmp/pip-install-p97r2dzi/gtn/src" does not appear to contain CMakeLists.txt.
  Specify --help for usage, or press the help button on the CMake GUI.
  Traceback (most recent call last):
    File "<string>", line 1, in <module>
    File "/tmp/pip-install-p97r2dzi/gtn/setup.py", line 100, in <module>
      setup(
    File "/home/pyp/miniconda3/lib/python3.8/site-packages/setuptools/__init__.py", line 153, in setup
      return distutils.core.setup(**attrs)
    File "/home/pyp/miniconda3/lib/python3.8/distutils/core.py", line 148, in setup
      dist.run_commands()
    File "/home/pyp/miniconda3/lib/python3.8/distutils/dist.py", line 966, in run_commands
      self.run_command(cmd)
    File "/home/pyp/miniconda3/lib/python3.8/distutils/dist.py", line 985, in run_command
      cmd_obj.run()
    File "/home/pyp/miniconda3/lib/python3.8/site-packages/wheel/bdist_wheel.py", line 290, in run
      self.run_command('build')
    File "/home/pyp/miniconda3/lib/python3.8/distutils/cmd.py", line 313, in run_command
      self.distribution.run_command(command)
    File "/home/pyp/miniconda3/lib/python3.8/distutils/dist.py", line 985, in run_command
      cmd_obj.run()
    File "/home/pyp/miniconda3/lib/python3.8/distutils/command/build.py", line 135, in run
      self.run_command(cmd_name)
    File "/home/pyp/miniconda3/lib/python3.8/distutils/cmd.py", line 313, in run_command
      self.distribution.run_command(command)
    File "/home/pyp/miniconda3/lib/python3.8/distutils/dist.py", line 985, in run_command
      cmd_obj.run()
    File "/tmp/pip-install-p97r2dzi/gtn/setup.py", line 53, in run
      self.build_extension(ext)
    File "/tmp/pip-install-p97r2dzi/gtn/setup.py", line 92, in build_extension
      subprocess.check_call(
    File "/home/pyp/miniconda3/lib/python3.8/subprocess.py", line 364, in check_call
      raise CalledProcessError(retcode, cmd)
  subprocess.CalledProcessError: Command '['cmake', '/tmp/pip-install-p97r2dzi/gtn/src', '-DCMAKE_LIBRARY_OUTPUT_DIRECTORY=/tmp/pip-install-p97r2dzi/gtn/build/lib.linux-x86_64-3.8/gtn/', '-DPYTHON_EXECUTABLE=/home/pyp/miniconda3/bin/python', '-DPROJECT_SOURCE_DIR=/tmp/pip-install-p97r2dzi/gtn/src', '-DGTN_BUILD_PYTHON_BINDINGS=ON', '-DGTN_BUILD_EXAMPLES=OFF', '-DGTN_BUILD_BENCHMARKS=OFF', '-DGTN_BUILD_TESTS=OFF', '-DCMAKE_BUILD_TYPE=Release']' returned non-zero exit status 1.
  ----------------------------------------
  ERROR: Failed building wheel for gtn
  Running setup.py clean for gtn
Failed to build gtn
Installing collected packages: gtn
    Running setup.py install for gtn ... error
    ERROR: Command errored out with exit status 1:
     command: /home/pyp/miniconda3/bin/python -u -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/tmp/pip-install-p97r2dzi/gtn/setup.py'"'"'; __file__='"'"'/tmp/pip-install-p97r2dzi/gtn/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' install --record /tmp/pip-record-3jz66vin/install-record.txt --single-version-externally-managed --compile --install-headers /home/pyp/miniconda3/include/python3.8/gtn
         cwd: /tmp/pip-install-p97r2dzi/gtn/
    Complete output (51 lines):
    running install
    running build
    running build_py
    creating build
    creating build/lib.linux-x86_64-3.8
    creating build/lib.linux-x86_64-3.8/gtn
    copying src/bindings/python/gtn/__init__.py -> build/lib.linux-x86_64-3.8/gtn
    creating build/lib.linux-x86_64-3.8/gtn/criterion
    copying src/bindings/python/gtn/criterion/__init__.py -> build/lib.linux-x86_64-3.8/gtn/criterion
    running build_ext
    CMake Warning:
      Ignoring extra path from command line:
    
       "/tmp/pip-install-p97r2dzi/gtn/src"
    
    
    CMake Error: The source directory "/tmp/pip-install-p97r2dzi/gtn/src" does not appear to contain CMakeLists.txt.
    Specify --help for usage, or press the help button on the CMake GUI.
    Traceback (most recent call last):
      File "<string>", line 1, in <module>
      File "/tmp/pip-install-p97r2dzi/gtn/setup.py", line 100, in <module>
        setup(
      File "/home/pyp/miniconda3/lib/python3.8/site-packages/setuptools/__init__.py", line 153, in setup
        return distutils.core.setup(**attrs)
      File "/home/pyp/miniconda3/lib/python3.8/distutils/core.py", line 148, in setup
        dist.run_commands()
      File "/home/pyp/miniconda3/lib/python3.8/distutils/dist.py", line 966, in run_commands
        self.run_command(cmd)
      File "/home/pyp/miniconda3/lib/python3.8/distutils/dist.py", line 985, in run_command
        cmd_obj.run()
      File "/home/pyp/miniconda3/lib/python3.8/site-packages/setuptools/command/install.py", line 61, in run
        return orig.install.run(self)
      File "/home/pyp/miniconda3/lib/python3.8/distutils/command/install.py", line 545, in run
        self.run_command('build')
      File "/home/pyp/miniconda3/lib/python3.8/distutils/cmd.py", line 313, in run_command
        self.distribution.run_command(command)
      File "/home/pyp/miniconda3/lib/python3.8/distutils/dist.py", line 985, in run_command
        cmd_obj.run()
      File "/home/pyp/miniconda3/lib/python3.8/distutils/command/build.py", line 135, in run
        self.run_command(cmd_name)
      File "/home/pyp/miniconda3/lib/python3.8/distutils/cmd.py", line 313, in run_command
        self.distribution.run_command(command)
      File "/home/pyp/miniconda3/lib/python3.8/distutils/dist.py", line 985, in run_command
        cmd_obj.run()
      File "/tmp/pip-install-p97r2dzi/gtn/setup.py", line 53, in run
        self.build_extension(ext)
      File "/tmp/pip-install-p97r2dzi/gtn/setup.py", line 92, in build_extension
        subprocess.check_call(
      File "/home/pyp/miniconda3/lib/python3.8/subprocess.py", line 364, in check_call
        raise CalledProcessError(retcode, cmd)
    subprocess.CalledProcessError: Command '['cmake', '/tmp/pip-install-p97r2dzi/gtn/src', '-DCMAKE_LIBRARY_OUTPUT_DIRECTORY=/tmp/pip-install-p97r2dzi/gtn/build/lib.linux-x86_64-3.8/gtn/', '-DPYTHON_EXECUTABLE=/home/pyp/miniconda3/bin/python', '-DPROJECT_SOURCE_DIR=/tmp/pip-install-p97r2dzi/gtn/src', '-DGTN_BUILD_PYTHON_BINDINGS=ON', '-DGTN_BUILD_EXAMPLES=OFF', '-DGTN_BUILD_BENCHMARKS=OFF', '-DGTN_BUILD_TESTS=OFF', '-DCMAKE_BUILD_TYPE=Release']' returned non-zero exit status 1.
    ----------------------------------------
ERROR: Command errored out with exit status 1: /home/pyp/miniconda3/bin/python -u -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/tmp/pip-install-p97r2dzi/gtn/setup.py'"'"'; __file__='"'"'/tmp/pip-install-p97r2dzi/gtn/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' install --record /tmp/pip-record-3jz66vin/install-record.txt --single-version-externally-managed --compile --install-headers /home/pyp/miniconda3/include/python3.8/gtn Check the logs for full command output.

When build from source, running make -j 8 gives:

[  1%] Building CXX object CMakeFiles/gtn.dir/gtn/autograd.cpp.o
[  3%] Building CXX object CMakeFiles/gtn.dir/gtn/cpu/creations.cpp.o
[  5%] Building CXX object CMakeFiles/gtn.dir/gtn/cpu/compose.cpp.o
[  6%] Building CXX object CMakeFiles/gtn.dir/gtn/cpu/functions.cpp.o
[  8%] Building CXX object CMakeFiles/gtn.dir/gtn/cpu/shortest.cpp.o
[ 10%] Building CXX object CMakeFiles/gtn.dir/gtn/creations.cpp.o
[ 11%] Building CXX object CMakeFiles/gtn.dir/gtn/device.cpp.o
[ 13%] Building CXX object CMakeFiles/gtn.dir/gtn/criterions.cpp.o
[ 15%] Building CXX object CMakeFiles/gtn.dir/gtn/functions.cpp.o
[ 16%] Building CXX object CMakeFiles/gtn.dir/gtn/graph.cpp.o
[ 18%] Building CXX object CMakeFiles/gtn.dir/gtn/parallel/parallel_map.cpp.o
[ 20%] Building CXX object CMakeFiles/gtn.dir/gtn/rand.cpp.o
[ 22%] Building CXX object CMakeFiles/gtn.dir/gtn/utils.cpp.o
[ 23%] Building CUDA object CMakeFiles/gtn.dir/gtn/cuda/creations.cu.o
[ 25%] Building CUDA object CMakeFiles/gtn.dir/gtn/cuda/compose.cu.o
[ 27%] Building CUDA object CMakeFiles/gtn.dir/gtn/cuda/cuda.cu.o
[ 28%] Building CXX object CMakeFiles/gtn.dir/gtn/cuda/functions.cpp.o
[ 30%] Building CUDA object CMakeFiles/gtn.dir/gtn/cuda/shortest.cu.o
/home/pyp/gtn/gtn/cuda/cuda.cu(94): error: identifier "cudaMallocAsync" is undefined

/home/pyp/gtn/gtn/cuda/cuda.cu(99): error: identifier "cudaFreeAsync" is undefined

/home/pyp/gtn/gtn/cuda/shortest.cu(19): error: A __constant__ variable cannot be marked constexpr

/home/pyp/gtn/gtn/cuda/shortest.cu(20): error: A __constant__ variable cannot be marked constexpr

2 errors detected in the compilation of "/home/pyp/gtn/gtn/cuda/cuda.cu".
CMakeFiles/gtn.dir/build.make:285: recipe for target 'CMakeFiles/gtn.dir/gtn/cuda/cuda.cu.o' failed
make[2]: *** [CMakeFiles/gtn.dir/gtn/cuda/cuda.cu.o] Error 1
make[2]: *** Waiting for unfinished jobs....
2 errors detected in the compilation of "/home/pyp/gtn/gtn/cuda/shortest.cu".
CMakeFiles/gtn.dir/build.make:313: recipe for target 'CMakeFiles/gtn.dir/gtn/cuda/shortest.cu.o' failed
make[2]: *** [CMakeFiles/gtn.dir/gtn/cuda/shortest.cu.o] Error 1
CMakeFiles/Makefile2:173: recipe for target 'CMakeFiles/gtn.dir/all' failed
make[1]: *** [CMakeFiles/gtn.dir/all] Error 2
Makefile:145: recipe for target 'all' failed
make: *** [all] Error 2

Thank you so much for your time!

Hi @awni,

I encountered some scenarios where the forward_score of a graph that is too large underflows. I'm wondering whether there's a way to work in a higher precision in GTN or whether I should change my methods to avoid such scenarios.

I reproduced one example here: https://colab.research.google.com/drive/1lRkhxJrPG4tBiHTbASl1iJX84Ao8JU8m?usp=sharing

g_emissions here comes from a CTC model, but instead of the normal g_criterion I use a modified version which marginalizes over many subword decompositions (similar to 4.2 in your paper https://arxiv.org/pdf/2010.01003.pdf). This is why my graphs are so large - do you have any suggestions?

I tried to implement a simple parallelFor using the underlying threadpool.

template <typename FuncType>
auto parallelFor(FuncType&& function, size_t size) {
  // Skipping bits similar to parallelMap
  ...
  for (size_t i = 0; i < size; ++i) {
    futures[i] = threadPool.get().enqueue(
      [size, i, &function, &eMutex, &eQueue]() -> OutType {
        try {
          return function(i);
        } catch (...) {
          .....
        }
      }
    );
  }
  ....
  }
  return out.value();
}

When using this method to perform the various for loops which according to the comments, should be independent on the iteration, I found that the code tends to fail. So far, my guess as to the source of the error is in the computeNodeAndArcPair function, although not sure.

An easy way to reproduce this error is to reverse the order in which the arcs are being processed. For instance on line 668

    for (int tid = totalArcs - 1; tid > 0; --tid) {

Again the same on line 491 and line 372

I am trying to compile GTN with CUDA support. I am using CUDA 11.2 and gcc 9.4.0.

I ran: cmake -DCMAKE_CUDA_COMPILER=/usr/local/cuda/11.2/bin/nvcc -DCMAKE_CUDA_ARCHITECTURES=80 .. which succeeds.

However, cmake -j 8 results in:

$ make -j 8

Scanning dependencies of target gtn
[  5%] Building CXX object CMakeFiles/gtn.dir/gtn/creations.cpp.o
[  6%] Building CXX object CMakeFiles/gtn.dir/gtn/device.cpp.o
[ 11%] Building CXX object CMakeFiles/gtn.dir/gtn/cpu/shortest.cpp.o
[  3%] Building CXX object CMakeFiles/gtn.dir/gtn/cpu/creations.cpp.o
[ 13%] Building CXX object CMakeFiles/gtn.dir/gtn/cpu/compose.cpp.o
[ 13%] Building CXX object CMakeFiles/gtn.dir/gtn/cpu/functions.cpp.o
[ 13%] Building CXX object CMakeFiles/gtn.dir/gtn/autograd.cpp.o
[ 13%] Building CXX object CMakeFiles/gtn.dir/gtn/criterions.cpp.o
[ 15%] Building CXX object CMakeFiles/gtn.dir/gtn/functions.cpp.o
[ 16%] Building CXX object CMakeFiles/gtn.dir/gtn/graph.cpp.o
[ 18%] Building CXX object CMakeFiles/gtn.dir/gtn/parallel/parallel_map.cpp.o
[ 20%] Building CXX object CMakeFiles/gtn.dir/gtn/rand.cpp.o
[ 22%] Building CXX object CMakeFiles/gtn.dir/gtn/utils.cpp.o
[ 23%] Building CUDA object CMakeFiles/gtn.dir/gtn/cuda/creations.cu.o
[ 25%] Building CUDA object CMakeFiles/gtn.dir/gtn/cuda/compose.cu.o
[ 27%] Building CUDA object CMakeFiles/gtn.dir/gtn/cuda/cuda.cu.o
[ 28%] Building CXX object CMakeFiles/gtn.dir/gtn/cuda/functions.cpp.o
[ 30%] Building CUDA object CMakeFiles/gtn.dir/gtn/cuda/shortest.cu.o
/gpfs/mariana/home/tanel.alumae/tools/gtn/gtn/cuda/shortest.cu(19): error: A __constant__ variable cannot be marked constexpr

/gpfs/mariana/home/tanel.alumae/tools/gtn/gtn/cuda/shortest.cu(20): error: A __constant__ variable cannot be marked constexpr

2 errors detected in the compilation of "/gpfs/mariana/home/tanel.alumae/tools/gtn/gtn/cuda/shortest.cu".
make[2]: *** [CMakeFiles/gtn.dir/build.make:303: CMakeFiles/gtn.dir/gtn/cuda/shortest.cu.o] Error 1
make[2]: *** Waiting for unfinished jobs....
make[1]: *** [CMakeFiles/Makefile2:198: CMakeFiles/gtn.dir/all] Error 2
make: *** [Makefile:160: all] Error 2

Any ideas?

Compare CTC performance with torch. Default values in the script correspond to a LibriSpeech example with stride 8 AM and batchsize=32 and 10k Word Pieces.

Here are some initial numbers (on Nvidia P100)

python bindings/python/benchmarks/ctc.py -N 10

| Summary:

• CUDA linking made simpler
• Keep parallel_compose_test.cpp in test/experimental folder

| Test Plan:

cmake .. -DCMAKE_BUILD_TYPE=RelWithDebInfo -DCMAKE_INSTALL_PREFIX=~/bin/ && make -j80 install make test

Also, was able to link gtn to a new repossity using cmake .. -Dgtn_DIR=~/bin/share/gtn/cmake/

TImings on my laptop using ./benchmarks/benchmark_functions

After Timing forwardScoreLinearForward (cpu) ... 9.1351 msec Timing forwardScoreLinearBackward (cpu) ... 9.085 msec Timing forwardScoreRandDAGForward (cpu) ... 5.7167 msec Timing forwardScoreRandDAGBackward (cpu) ... 6.4635 msec Timing viterbiPathLinearForward (cpu) ... 1.7872 msec Timing viterbiPathLinearBackward (cpu) ... 1.7198 msec Timing viterbiPathRandDAGForward (cpu) ... 2.0626 msec Timing viterbiPathRandDAGBackward (cpu) ... 0.1356 msec

Before Timing forwardScoreLinearForward (cpu) ... 14.57 msec Timing forwardScoreLinearBackward (cpu) ... 16.383 msec Timing forwardScoreRandDAGForward (cpu) ... 5.8892 msec Timing forwardScoreRandDAGBackward (cpu) ... 11.59 msec Timing viterbiPathLinearForward (cpu) ... 4.2249 msec Timing viterbiPathLinearBackward (cpu) ... 1.7191 msec Timing viterbiPathRandDAGForward (cpu) ... 1.7667 msec Timing viterbiPathRandDAGBackward (cpu) ... 0.087886 msec

1. Some optimizations for the CPU SOA data structure

• Use a pointer for the arc indices instead of making a vector
• Speed up linear construction
• Speed up clone

2. CUDA functionality:

• Graph::item
• clone, projectInput, projectOutput
• equal
• linearCreation, scalarCreation

3. Added a Device class to make device management cleaner (very similar to torch.device).

4. Refactored cpp testing to use a single main. Speeds up build and makes it easy to run all the tests.

Fix bug with epsilon composition.

Tests

Running tests... Test project /Users/awni/repos/gtn/build Start 1: autograd_test 1/8 Test #1: autograd_test .................... Passed 0.87 sec Start 2: creations_test 2/8 Test #2: creations_test ................... Passed 0.37 sec Start 3: criterion_test 3/8 Test #3: criterion_test ................... Passed 0.57 sec Start 4: functions_test 4/8 Test #4: functions_test ................... Passed 0.86 sec Start 5: parallel_test 5/8 Test #5: parallel_test .................... Passed 0.81 sec Start 6: graph_test 6/8 Test #6: graph_test ....................... Passed 0.44 sec Start 7: utils_test 7/8 Test #7: utils_test ....................... Passed 0.33 sec Start 8: rand_test 8/8 Test #8: rand_test ........................ Passed 0.50 sec

100% tests passed, 0 tests failed out of 8

Total Test time (real) = 4.74 sec

Benchmarks

TLDR not a huge impact, negligible for CTC.

Before fix

Timing composeForward ... 142.12 msec Timing composeBackward ... 0.79 msec Timing composeForwardSorted ... 9.53 msec Timing ctcLoss ... 124.37 msec Timing ctcGrad ... 21.44 msec Timing ngramCtcLoss ... 5.47 msec Timing ngramCtcGrad ... 0.61 msec Timing ctcBatched ... 210.01 msec

After fix

Timing composeForward ... 144.14 msec Timing composeBackward ... 0.8 msec Timing composeForwardSorted ... 10.19 msec Timing ctcLoss ... 131.27 msec Timing ctcGrad ... 21.73 msec Timing ngramCtcLoss ... 6.19 msec Timing ngramCtcGrad ... 0.61 msec Timing ctcBatched ... 218.7 msec

Hi! I noticed that the python example and docs for interfacing with pytorch didn't make use of the chain rule, which led to wrong gradients when you do things downstream with the loss.

Hi and thanks for a great library!

I noticed some of the README and examples import several Python libraries (e.g. numpy, torch etc.) that aren't listed anywhere as explicit dependencies. I feel like this could be helpful so that users can run the tests directly after installing the Python bindings:

python -m unittest discover bindings/python/test

Alternatively, the README could also instruct the user to install these manually, but that's a bit more hassle.

Happy to submit a PR for this. :-)

Hi,

Since I couldn't get gtn to compile with cuda (see our conversation in this issue), I turned off all CUDA related options. Specifically, I changed https://github.com/gtn-org/gtn/blob/master/CMakeLists.txt#L17 to OFF, and deleted https://github.com/gtn-org/gtn/blob/master/bindings/python/CMakeLists.txt#L28.

With the above changes, running

mkdir -p build && cd build
cmake ..
make -j $(nproc)

were successful

and running

cd bindings/python
conda install setuptools
python setup.py develop

were also successful,

However, when running

python -m unittest discover bindings/python/test

I also commented https://github.com/gtn-org/gtn/blob/master/bindings/python/gtn/init.py#L21, and delete ./bindings/python/test/test_cuda.py

I got output:

...F................................F...........
======================================================================
FAIL: test_forward_score_grad (test_autograd.AutogradTestCase)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/pyp/gtn/bindings/python/test/test_autograd.py", line 261, in test_forward_score_grad
    self.assertTrue(math.isnan(grad_weights[0]))
AssertionError: False is not true

======================================================================
FAIL: test_forward (test_functions.FunctionsTestCase)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/pyp/gtn/bindings/python/test/test_functions.py", line 217, in test_forward
    self.assertRaises(ValueError, gtn.forward_score, g)
AssertionError: ValueError not raised by forward_score

----------------------------------------------------------------------
Ran 48 tests in 0.158s

FAILED (failures=2)

Thanks for your time!

Hi! First of all, I think this is a cool project!

I was wondering if you have considered adding an optional parameter to forwardScore that invokes a check if the FSA/FST is unambiguous (i.e. at most one run per word)? If the FSA is ambiguous then the forward algorithm will fail to compute the weight of the language because some words are counted multiple times. I'm not sure if this is desired in some cases but I think there should be a way to get a warning at least.