Downloads .ksy files and their dependencies straight from the official kaitai-struct format gallery.

Overview

ksy-dl

Downloads .ksy files and their dependencies straight from the official kaitai-struct format gallery.

This tool will:

  • Fetch any of the official specifications
  • Fetch only their dependencies
  • Organize the specifications into separate directories based on the category of the specification
  • Rewrite the import paths of the dependencies so that they use relative paths

This makes it easy to include, modify, and share official and community created kaitai-struct specifications in individual projects.

Requirements

$ git clone https://github.com/tins2831/ksy-dl.git
$ cd ksy-dl
$ python3 -m pip install --user -r requirements.txt

Usage

usage: python3 ksy-dl.py QUERY OUTPUT_DIR
...

Examples:
$ python3 ksy-dl.py "network/pcap" .
$ python3 ksy-dl.py "pcap" ../kaitai # will create non-existing directories too
$ python3 ksy-dl.py "/network/pcap" network/
$ python3 ksy-dl.py "pcap.ksy" .
$ python3 ksy-dl.py "google_protobuf" .

Auto-updating the database

First, clone the kaitai format gallery from github:

$ git clone https://github.com/kaitai-io/kaitai_struct_formats.git

Then adjust and append this to your crontab:

# Update ksy-dl format db.
# Runs every day at midnight.
00 00 * * * (KAITAI_FMT_REPO=
    
    
     /tools/db-update.sh)

    
   
You might also like...
Downloads photos you saved from a specific profile.

instagram-download-saved A python script that downloads photos you saved from a specific profile. The only dependency is instaloader, an open-source p

Python script to automate youtube-dl downloads

Automated Download Tool !! Project status I am writing a new version of this program, which will solve several errors. The new version only supports G

Tool To Get Downloads up to 4k from Paramount+
Tool To Get Downloads up to 4k from Paramount+

Paramount 4K Downloader Tool To Get Downloads up to 4k from Paramount+ 😄 Hello Fellow Developers/ ! Hi! My name is WVDUMP. I am Leaking the script

A python scripts that downloads doujin from nhentai without having an account

nhentai-downloader a python scripts that downloads doujin from https://nhentai.net without having an account. Usage Needs Python 3^ Linux pip3 install

Downloads yiffer.xyz comics as images

yiffer-dl Downloads comics as images from yiffer.xyz.

ComicDownloader - Downloads Comics from readcomiconline.li

ComicDownloader Downloads Comics from readcomiconline.li To use this script from

Simple tool downloads public PoC (refer from nomi-sec)

PoC Collection This is the little script to collect the proof-of-concept which is refered from nomi-sec. The repository now is only develop for linux-

Downloads state flags from wikipedia for states/regions from all countries

world-state-flags Downloads state flags from wikipedia for states/regions from all countries This data is NOT curated Uses https://github.com/dr5hn/co

Downloads separate (specified) file to a randomly generated folder in /TEMP then executes it.

PyTemp-1 A Python3 file downloader. What you do with this code / project / idea is non of my buisness or concern, and this was made for **educational*

Comments
  • Optionally extract all the types in the specification(s)

    Optionally extract all the types in the specification(s)

    There exist many ksy file specifications that have a mess of nested types. It gets ugly real fast:

    meta:
      id: food
    types:
      potato:
        types:
          bun:
            types:
              steak:
                types:
                  chicken:
                    enums:
                      cook_method:
                        1: fried
                        2: roasted
                    types:
                      cereal:
                        seq:
                          - id: flakes
                            type: u1
                            enum: cook_method
      rice:
        types:
          corn:
            seq:
              - id: kernel
                type: u1
      cabbage:
        seq:
          - id: leaf
            type: u1
    
    from kaitai_local import food
    
    ...
    if mycereal.flakes.value == food.potato.bun.steak.chicken.cook_method.fried.value:
        # annoyed
    

    I'm positive anyone would be annoyed by that. Users should be allowed to toggle a flag at the command-line interface to tell ksy-dl to extract all the types into a hierarchical structure of files:

    └── kaitai_local
        └── food
            ├── cabbage
            │   └── cabbage.ksy
            ├── food.ksy
            ├── potato
            │   └── bun
            │       └── steak
            │           └── chicken
            │               ├── cereal
            │               │   └── cereal.ksy
            │               └── chicken.ksy
            └── rice
                └── corn
                    └── corn.ksy
    

    Not only does this make a specification more easier to sift through and use in code, it also becomes modular now:

    from kaitai_local.food import food
    from kaitai_local.food.potato.bun.steak.chicken.cereal import cereal
    from kaitai_local.food.potato.bun.steak.chicken import chicken
    from kaitai_local.food.rice.corn import corn
    # from kaitai_local.food.cabbage import cabbage <- can omit this now instead of importing it
    
    ...
    if mycereal.flakes.value == chicken.cook_method.roasted.value:
        # not annoyed
    

    For this to work though, I'd have to:

    • Adjust the imports for each type
    • Correct the references to subtypes
    • (possibly more)
    enhancement 
    opened by tins2831 1
  • Please add some FAQ

    Please add some FAQ

    Hello. Thanks for the tool. But what is the rationale for such a tool? Isn't it easier to download from the official git repo instead, as I do in my https://github.com/kaitaiStructCompile/kaitaiStructCompile.repo ?

    The only really valuable use case I can imagine is downloading pre-transpiled sources. But it is not yet implemented. I'd create a separate bug for it.

    enhancement 
    opened by KOLANICH 2
Owner
null
Python module to download all media from a GoFile gallery.

GoFile Downloader Setup First of all, clone this repository : ~$ git clone https://github.com/quatrecentquatre-404/gofile-downloader Second, oh wait..

Quatrecentquatre 61 Jan 1, 2023
FireDM is a python open source (Internet Download Manager) with multi-connections, high speed engine, it downloads general files and videos from youtube and tons of other streaming websites .

python open source (Internet Download Manager) with multi-connections, high speed engine, based on python, LibCurl, and youtube_dl https://github.com/firedm/FireDM

null 1.6k Apr 12, 2022
PyDownloader - Downloads files and folders at high speed (based on your interent speed).

PyDownloader - Downloads files and folders at high speed (based on your interent speed).

Armen._.G 4 Feb 24, 2022
Downloads files and folders

PyDownloader Downloads files and folders at high speed (based on your interent speed). This is very useful to transfer big files from one computer to

ArmenG 4 Feb 24, 2022
Download Apple Music Cover Artwork in the best Quality by providing an Apple Music Link. It downloads the jpg, png and webp version since they often differ from another.

amogus.py - Version 0.0.5 amogus - Apple Music Hi-Res Artwork Fetcher this is my first real python tool so sorry if its bad amogus is a Python script

reaper 46 Jan 9, 2023
Downloads and Updates GOG Galaxy 2.0 Plugins/Integrations

GOG Galaxy Plugins Downloader Summary This program downloads GOG Galaxy 2.0 Plugins and installs them to the proper location. You probably do not want

slashbunny 253 Dec 12, 2022
A program which takes an Anime name or URL and downloads the specified range of episodes.

super-anime-downloader A console application written in Python3.x (GUI will be added soon) which takes a Anime Name/URL as input and downloads the ran

Sayyid Ali Sajjad Rizavi 26 Jul 18, 2022
Python Program that downloads gaming required packages based on your Linux Distribution.

LibreGaming Python Program that downloads gaming required packages based on your Linux Distribution. Table of contents Distributions Prerequisites Dep

Ahmed Al Balochi 195 Jan 1, 2023
Organize your downloads easily with DownloadOrganizer

DownloadOrganizer Organize your downloads organize your downloads easily with DownloadOrganizer Instilation how to install DownloadOrganizer Method 1:

null 1 Dec 2, 2021
A script that downloads YouTube videos/audio

YouTube-Downloader A script that downloads YouTube videos/audio from youtube. Usage Download the script by executing the following in your terminal :

Debayan Sarkar 2 Jan 4, 2022