I have three bundles which I have added to my itch.io library, resulting in about 4,000 games in my library. When using python -m itchiodl.downloader with my API key to download my library, the following occurs:

• The downloader says a number of "pages" are loaded
• The downloader starts going through the list of games, saying it is downloading them
• Publisher and game folders are created for these games on my hard drive in the CWD
• JSON files are also created for games, within publisher folders
• The downloader says games have been downloaded (x of 3918)
• Nothing is actually downloaded except for the JSON files
• Eventually the downloader simply stops, with the last item being that a game was downloaded (x of 3918)

Sometimes it makes it to about 1,000 games "downloaded", sometimes 1,500 before it quits. It leaves no error message or error text file with any other information. It doesn't seem to matter if I change the number of jobs (I've tried 4 jobs and 8 jobs), run with administrator privileges (using Python through Windows command prompt), or anything else. I'm not utilizing a VPN or anything else special. I'm running Python 3.9 64-bit on Windows 10.

Both scripts doesn't seem to work on Python 3.10:

itch-download:

Traceback (most recent call last):
  File "/home/user/.local/bin/itch-download", line 8, in <module>
    sys.exit(main())
  File "/home/user/.local/lib/python3.10/site-packages/itchiodl/downloader/__main__.py", line 21, in main
    l = itchiodl.LoginAPI(user, password)
  File "/home/user/.local/lib/python3.10/site-packages/itchiodl/login.py", line 29, in LoginAPI
    raise RuntimeError
RuntimeError

itch-load-bundle:

Traceback (most recent call last):
  File "/home/user/.local/bin/itch-load-bundle", line 8, in <module>
    sys.exit(main())
  File "/home/user/.local/lib/python3.10/site-packages/itchiodl/bundle_tool/__main__.py", line 12, in main
    b.load_games()
  File "/home/user/.local/lib/python3.10/site-packages/itchiodl/bundle.py", line 14, in load_games
    pages = int(s.select("span.pager_label a")[-1].text)
IndexError: list index out of range

Works fine on Python 3.9

Hey, great tool! I'm trying to download my library but it always crashes at a game that only has a dropbox link for download.

Traceback (most recent call last): File "C:\Users*\AppData\Local\Programs\Python\Python38\lib\runpy.py", line 192, in _run_module_as_main return _run_code(code, main_globals, None, File "C:\Users*\AppData\Local\Programs\Python\Python38\lib\runpy.py", line 85, in run_code exec(code, run_globals) File "C:\Users*\AppData\Local\Programs\Python\Python38\lib\site-packages\itchio\downloader_main.py", line 26, in lib.download_library() File "C:\Users*\AppData\Local\Programs\Python\Python38\lib\site-packages\itchio\library.py", line 32, in download_library game.download(self.login) File "C:\Users*\AppData\Local\Programs\Python\Python38\lib\site-packages\itchio\game.py", line 62, in download itchio.utils.download(url, path, self.name +" - "+file) File "C:\Users*\AppData\Local\Programs\Python\Python38\lib\site-packages\itchio\utils.py", line 11, in download filename = re.search(r'filename="(.+)"', cd).group(1) File "C:\Users*\AppData\Local\Programs\Python\Python38\lib\re.py", line 199, in search return _compile(pattern, flags).search(string) TypeError: expected string or bytes-like object

Since there's no way to remove games from the library, could you please add a way to ignore games with dropbox links?

The only workaround I've found is to create a .json file with the name of the game so that the downloader skips it.

The download script consistently crash on this file.

Downloading Calico - Calico.Template.zip
Traceback (most recent call last):
  File "/home/user/.local/bin/itch-download", line 8, in <module>
    sys.exit(main())
  File "/home/user/.local/lib/python3.10/site-packages/itchiodl/downloader/__main__.py", line 31, in main
    lib.download_library()
  File "/home/user/.local/lib/python3.10/site-packages/itchiodl/library.py", line 35, in download_library
    game.download(self.login)
  File "/home/user/.local/lib/python3.10/site-packages/itchiodl/game.py", line 68, in download
    itchiodl.utils.download(url, path, self.name + " - " + file)
  File "/home/user/.local/lib/python3.10/site-packages/itchiodl/utils.py", line 21, in download
    filename = re.search(r'filename="(.+)"', cd).group(1)
AttributeError: 'NoneType' object has no attribute 'group'

Downloading Troika! Numinous Edition - edit
Traceback (most recent call last):
  File "c:\Python310\lib\runpy.py", line 196, in _run_module_as_main
    return _run_code(code, main_globals, None,
  File "c:\Python310\lib\runpy.py", line 86, in _run_code
    exec(code, run_globals)
  File "c:\Python310\lib\site-packages\itchio\downloader\__main__.py", line 26, in <module>
    lib.download_library()
  File "c:\Python310\lib\site-packages\itchio\library.py", line 32, in download_library
    game.download(self.login)
  File "c:\Python310\lib\site-packages\itchio\game.py", line 62, in download
    itchio.utils.download(url, path, self.name +" - "+file)
  File "c:\Python310\lib\site-packages\itchio\utils.py", line 11, in download
    filename = re.search(r'filename="(.+)"', cd).group(1)
  File "c:\Python310\lib\re.py", line 200, in search
    return _compile(pattern, flags).search(string)
TypeError: expected string or bytes-like object

The game in question is: https://melsonian-arts-council.itch.io/troika-numinous-edition/ purchased from the Ukraine bundle.

The type of cd is none, thus the header doesn't exist, if I understand it correctly.

Maybe add the ability to input a link to a game page, like https://etra-etra-games.itch.io/vortex-an-fps-where-you-cant-damage-enemies for example, so that you can download games that aren’t able to be added to your library.

Is there any way to differentiate between Windows, Linux and MacOS versions? For example if I want to download all Windows versions of all games (since I don't use Linux or MacOS)?

Currently as I see it really downloads everything, even if I don't need Linux or MacOS.

On windows, at least.

Too lazy to submit a PR, but this works: Utils.py:

def download_url(url, output_path, desc):
    with DownloadProgressBar(unit='B', unit_scale=True,
                             miniters=1, desc=desc) as t:
        output_path = re.sub(r'[\:\*\?\"\<\>\|]', ' ', output_path)
        urllib.request.urlretrieve(url, filename=output_path, reporthook=t.update_to)

hey all

I know this is gaining some traction at the moment, so I thought I'd add some multithreading to try and make it run faster / more efficiently

I'm not going to merge this in until I've had several people test this, so I'd appreciate feedback in the comments

Installing:

# clone the repo
git clone https://github.com/Emersont1/itchio

# checkout to multithreading
cd itchio
git checkout multithreading

# create and load a venv
python -m venv venv
source venv/bin/activate

# install
pip install .

# profit?
python -m itchiodl.downloader -k INSERT_API_KEY_HERE

C:\Users\Username\Desktop\ItchioGames>python -m itchiodl.bundle_tool
Username: removed
Password:
Bundle URL: removed
Traceback (most recent call last):
  File "C:\Users\Username\AppData\Local\Programs\Python\Python310\lib\runpy.py", line 196, in _run_module_as_main
    return _run_code(code, main_globals, None,
  File "C:\Users\Username\AppData\Local\Programs\Python\Python310\lib\runpy.py", line 86, in _run_code
    exec(code, run_globals)
  File "C:\Users\Username\AppData\Local\Programs\Python\Python310\lib\site-packages\itchiodl\bundle_tool\__main__.py", line 17, in <module>
    main()
  File "C:\Users\Username\AppData\Local\Programs\Python\Python310\lib\site-packages\itchiodl\bundle_tool\__main__.py", line 13, in main
    b.load_games()
  File "C:\Users\Username\AppData\Local\Programs\Python\Python310\lib\site-packages\itchiodl\bundle.py", line 15, in load_games
    pages = int(s.select("span.pager_label a")[-1].text)
IndexError: list index out of range

Extra info: I am not using 2FA, just username and password

How can I fix this?

Hopefully this will fix the issues in #28

Can you test it on the one it wasn't working on before with these fixes please

# clone the repo
git clone https://github.com/Emersont1/itchio

# checkout to filename-fixes
cd itchio
git checkout filename-fixes

# create and load a venv
python -m venv venv
source venv/bin/activate

# install
pip install .

# profit?
python -m itchiodl.downloader -k INSERT_API_KEY_HERE

Currently, itchiodl will download every new file as if the older one don't exist:

• if a file with the same name exist, it will overwrite it
• if a file with a different name exist, it will just download the new file and "ignore" the existing file

This is a bit of a problem, in the sense that everytime a game update, you will more than likely keep the older version.

lgogdownloader (the main GOG.com downloader) "solves" that with --check-orphan function that will check and reports (only reports, it won't do anything to them) the files that exist that aren't available to download anymore (which is, almost all the time: older versions).

itchiodl definitely needs something like that considering how many "thing" someone can have with the bundles.

Something that can delete those older files would be even better, but a "detector" would be a good start.

PS: thanks (again) for itchiodl!

I can't seem to get the downloader working, I got the bundle tool to add all my games to my library. When I start a download I get this window, it looks like it is downloading but no files or folders are made anywhere on my computer. I have tried Python 3.8 as I saw it was stable in a Reddit post and I have tried 3.10 as well. I even tried a different computer in case it was an issue with Windows 11. I have also tried using the API key as well. Is there something I am not doing?

I feel like I've got these changes in a relatively clean spot for review, to summarize:

• added the -vf argument for Verbose Folders, created folder structure uses display names for game titles and publishers
• added the -v argument for toggling verify step functionality
• added the -sas argument for specifying that you wish to skip downloading files above a size threshold (for disk saving)
• various small improvements like using one call to os.makedirs() vs multiple os.mkdir() calls
• patched an issue where download keys weren't being inherited properly by game objects as they were being appended to the library list, preventing downloads

root@ca48796d249d:/# itch-load-bundle
Username: [email protected]
Password:
Bundle URL: http://itch.io/bundle/download/4T80AvOSA70gkpyUhMNhaaghCbEY7J0DAJjWaWZd
Traceback (most recent call last):
  File "/usr/local/bin/itch-load-bundle", line 8, in <module>
    sys.exit(main())
  File "/usr/local/lib/python3.10/site-packages/itchiodl/bundle_tool/__main__.py", line 13, in main
    b.load_games()
  File "/usr/local/lib/python3.10/site-packages/itchiodl/bundle.py", line 17, in load_games
    pages = int(s.select("span.pager_label a")[-1].text)
IndexError: list index out of range
root@ca48796d249d:/#

Did I use the Program as intended, or is this actually an issue?