I'm trying to use --inventory since my bucket have around ~22800 objects and doing just regular --s3 --bucket name --inventory --output name.html only generate up to 1000 objects (can this be overridden by the way? Since aws s3api list-object-versions can show all of the objects), but upon issuing ./dir_sizer.py --s3 --bucket 90poe-athens-go-mod-proxy --inventory --output ~/s3-cost/90poe-athens-go-mod-proxy.html I'm greeted with:

Using S3 Inventory report "test-inventory" generated 2022-08-25 03:00:00...
Scanning, gathered 1000 totaling 175.90 MiB...Traceback (most recent call last):
  File "/Users/alexk/src/dir_sizer/./dir_sizer.py", line 264, in <module>
    main()
  File "/Users/alexk/src/dir_sizer/./dir_sizer.py", line 190, in main
    for filename, size in load_files(opts, abstraction):
  File "/Users/alexk/src/dir_sizer/./dir_sizer.py", line 256, in load_files
    for filename, size in abstraction.scan_folder(opts):
  File "/Users/alexk/src/dir_sizer/s3_abstraction.py", line 310, in scan_folder
    for i, cur in enumerate(s3_list_objects(msg, opts, s3)):
  File "/Users/alexk/src/dir_sizer/s3_abstraction.py", line 274, in s3_list_objects
    'Size': int(row['Size']),
ValueError: invalid literal for int() with base 10: ''

What am I doing wrong here?

Thanks for the awesome project otherwise!

Hello, thank you for an awesome project. It was working for me quite fine few months ago, but tested yesterday and received following stack trace:

Scanning...Traceback (most recent call last):
  File "/Users/alexk/src/dir_sizer/dir_sizer.py", line 264, in <module>
    main()
  File "/Users/alexk/src/dir_sizer/dir_sizer.py", line 190, in main
    for filename, size in load_files(opts, abstraction):
  File "/Users/alexk/src/dir_sizer/dir_sizer.py", line 256, in load_files
    for filename, size in abstraction.scan_folder(opts):
  File "/Users/alexk/src/dir_sizer/s3_abstraction.py", line 313, in scan_folder
    for i, cur in enumerate(s3_list_objects(msg, opts, s3)):
  File "/Users/alexk/src/dir_sizer/s3_abstraction.py", line 272, in s3_list_objects
    for row in get_bucket_inventory(msg, s3, opts['s3_bucket'], required_fields=required_fields, prefix=prefix):
  File "/Users/alexk/src/dir_sizer/s3_abstraction.py", line 206, in get_bucket_inventory
    inv_prefix = config['Destination']['S3BucketDestination']['Prefix']
KeyError: 'Prefix'

Environment: Python 3.9.13, macos, fresh virtual env with boto3 and Pillow installed. Inventory file: 94990aad-1014-4013-809c-82a266e40f14.csv.gz s3 inventory config:

I've additionally tested inside fresh linux vm, as well as docker container (with minimal dockerfile), but got same error.

Should I still define default prefix for scanning ? Please let me know if I should provide any additional information, and once again thanks for the help and awesome tool!