命令行版本的HLS/DASH流下载器,支持标准AES-128-CBC解密

Overview

XstreamDL-CLI

基于python 3.7.4+的,命令行版本的,HLS/DASH流下载器,支持标准AES-128-CBC解密

使用

首先安装必要的库

pip install -r requirements.txt
python -m XstreamDL_CLI.cli FILEPATH/STREAMURL

实例

  1. 通过python执行下载模块

    python -m XstreamDL_CLI.cli --b64key oKi/hwKVuLveo/hISX1PQw== --hexiv b3d5ca56926d49d8e96b70aa5c7b358e --name 第一节总论 https://hls.videocc.net/d06ae002cb/2/d06ae002cb4a0bed78fb912c874fdbb2_2.m3u8
  2. 通过exe直接下载

    XstreamDL-CLI_v1.0.0.exe --b64key oKi/hwKVuLveo/hISX1PQw== --hexiv b3d5ca56926d49d8e96b70aa5c7b358e --name 第一节总论 https://hls.videocc.net/d06ae002cb/2/d06ae002cb4a0bed78fb912c874fdbb2_2.m3u8
  3. 如果使用Windows Terminal,可以将下面的命令保存为bat文件

    chcp 65001
    wt new-tab -p "Command Prompt" -d "%cd%" cmd /k "XstreamDL-CLI_v1.0.0.exe --b64key oKi/hwKVuLveo/hISX1PQw== --hexiv b3d5ca56926d49d8e96b70aa5c7b358e --name 第一节总论 https://hls.videocc.net/d06ae002cb/2/d06ae002cb4a0bed78fb912c874fdbb2_2.m3u8"

合并需要先将ffmpeg置于环境变量

pyinstaller打包

pyinstaller -i logo.ico -n XstreamDL-CLI_v1.0.0 -F XstreamDL_CLI\__main__.py

示意

  • 普通m3u8下载

  • master m3u8下载

  • 特殊master m3u8下载

特性

  • 基于aiohttp

其他

  • 逐步完善中

参考

Issues
  • I cannot using command

    I cannot using command "DL" when using newest version!

    Hi,

    I just used your software recently and I see, in the latest version I cannot run the "DL" command when I have entered the key, kid, file name and stream link. When I click to "DL" button, I see that they will auto show popup but just 2 second and off, I have try again but it still that problem! But, on a previous version is v.1.2.7 is can run command "DL" but when I download video, I see it can download to 80-90% only they cannot download to 100%, just for video, audio I can download in 100% normally! I dont know why, can you help me ?

    I cannot download full 100%

    Thanks!

    opened by georgesaumen 43
  • Plis help :)

    Plis help :)

    Hi

    I used download and decrypt example: python -m XstreamDL_CLI.cli --select --disable-force-close --limit-per-host 100 --key f31203576a323d09d0c305d236a0c793:00fa546ee19f98fc0044237d2ceb820b "https://akamaicdn.hbogo.eu/a9626f47-b065-2a26-43f9-a3094fb7c4d3_hbo/COMP/29223422_hun_comp_d35eda69-a367-4b47-aa0c-a51032d94be2_3400000_v2.ism/manifest" And i have a bug, sorry i'm a beginner:

    image

    Thank you for your answer

    opened by Amb33r 10
  • 下载长视频容易中断

    下载长视频容易中断

    m3u8url = 'http://58.222.51.151/vipts.tc.qq.com/A53DH4JO0WA6HF-yGkWDAlJ3MZJQcasVauzM1iEIupBk/uwMROfz2r55goaQXGdGnC2de64gtX89GT746tcTJVnDpJgsD/svp_50112/GquHDAOkA7rx4Twr13V0_6Gm0UOcM1Y7NuPcpo6keTiLDvhC_0t-XR3YqFPdMQtE39dVgo3S0Ks2cuzatMW9RSA6G5md6O3_vRa-mi_xcJ-hxo0fByBUGEkrwUA8pkmeU7Aso-kDV4Dp8tXnOZI-78bm-Kktoo7qstxWToYb1ssRA6RthPusdw/gzc_1000102_0b53taaawaaaymaiekw2tbq4bggdbokaad2a.f321004.ts.m3u8?ver=4'

    例如这个链接,总是下载到62%就会断,但是接着下载又能完全下载完

    37_1 ██████████████████ 921.03/1007.86MB • 0.33MB/s • 153/244 • 62.70% 2022-01-10 21:22:30,044 XstreamDL stream.py 212 : ERROR 尝试合并 37_1 但是未下载完成

    opened by hecoter 9
  • aiohttp request meet 403 response

    aiohttp request meet 403 response

    在下载某厂商的m3u8时,返回403:

    <!DOCTYPE HTML PUBLIC "-//IETF//DTD HTML 2.0//EN">
    <html>
    <head><title>403 Forbidden</title></head>
    <body>
    <h1>403 Forbidden</h1>
    <p>You don't have permission to access the URL on this server.<hr/>Powered by Tengine</body>
    </html>
    

    改用urllib或curl后都可以正常下载,推测是aiohttp的问题。

    我看代码中均未用到aiohttp的异步特性,是否考虑替换为同步的HTTP请求库,或者直接调用curl(这样其实更方便调试)?

    opened by elxy 8
  • 直播无法下载

    直播无法下载

    https://playlist.fubo.tv/master.mpd?alternate-feed=0&callsign=KTNV&cdn_cache_key=S1ROVi1kZXNrdG9wLVVTQS04OTAzMi04OTEwMS0wLTAtREFTSC0wLTA%3D&cookie_supported_platform=1&device=ALL&hdnts=ip%3D66.187.4.21~st%3D1639621403~exp%3D1639707803~acl%3D%2F%2A~data%3D1639707803~hmac%3D2ef355cd8643174c4f123f9095c701fc54ed78d571b0a87325963f292f71b0c6&is_kvtdb=0&watchToken=eyJhbGciOiJIUzI1NiIsInRva2VuVHlwZSI6IlVzYUJhc2VXYXRjaFRva2VuIiwidHlwIjoiSldUIn0.eyJ3YXRjaFRva2VuIjp7ImNhbGxTaWduIjoiS1ROViIsImRldmljZSI6IkFMTCIsImJhbmR3aWR0aCI6Ik1BU1RFUiIsImNvdW50cnkiOiJVU0EiLCJ1c2VySWQiOiI2MTliNTg3MWJhZmY3MTAwMDFkMTQzNDIiLCJyZXF1ZXN0VHlwZSI6ImxpdmUiLCJyZWdpb24iOiJ3ZXN0In0sImhvbWVaaXAiOiI4OTAzMiIsImdlb1ppcCI6Ijg5MTAxIn0.j9AhzliF_tqiQD6GCwDkM_A2QN6VwzhU3-sdWA4C5AE

    opened by 362227 6
  • No track ID - problem

    No track ID - problem

    Hi. I have a question.

    I downloaded the same manifest file using yt-dlp and XstreamDL. XstreamDL is much faster and I like this app more. However, the output files are different.

    yt-dlp gives video.ismv and XstreamDL video.mp4

    mediainfo

    The problem is that both videos cannot be decrypted because they don't have init files. I can get init files but to mux them I need to know video track ID. In this case it is 9. But video downloaded by XstreamDL doesn't show any information.

    Can you explain it, please?

    opened by m0ck69 5
  • --live-duration has been set but still getting no attribute 'timestamp'

    --live-duration has been set but still getting no attribute 'timestamp'

    您好,不好意思打扰了 想请问 Key & Key ID 都放了 --live-duration 也放了可是还是提示 AttributeError: 'str' object has no attribute 'timestamp' [2588] Failed to execute script '__main__' due to unhandled exception! 具体如下:

    2022-02-14 18:08:25,626 XstreamDL log.py 41 : INFO  log file -> C:/Users/Sam Sam/Desktop/Widevine Livestream Test/logs/XstreamDL-2022-02-14_180825.log
    2022-02-14 18:08:25,628 XstreamDL default.py 19 : WARNING  headers is not exists, put your config file to C:/Users/Sam Sam/Desktop
    2022-02-14 18:08:25,629 XstreamDL cli.py 158 : INFO  use 1.4.0, set URI to ['https://rest-as.ott.kaltura.com/api_v3/service/assetFile/action/playManifest/partnerId/147/assetId/340313/assetType/media/assetFileId/11460225/contextType/PLAYBACK/isAltUrl/False/a.mpd']
    Traceback (most recent call last):
      File "XstreamDL_CLI\__main__.py", line 4, in <module>
      File "XstreamDL_CLI\cli.py", line 161, in main
      File "XstreamDL_CLI\daemon.py", line 30, in daemon
      File "XstreamDL_CLI\extractor.py", line 48, in fetch_metadata
      File "XstreamDL_CLI\extractor.py", line 95, in raw2streams
      File "XstreamDL_CLI\extractor.py", line 127, in parse_as_dash
      File "XstreamDL_CLI\extractors\dash\parser.py", line 133, in parse
      File "XstreamDL_CLI\extractors\dash\parser.py", line 150, in walk_period
      File "XstreamDL_CLI\extractors\dash\parser.py", line 178, in walk_adaptationset
      File "XstreamDL_CLI\extractors\dash\parser.py", line 242, in walk_representation
      File "XstreamDL_CLI\extractors\dash\parser.py", line 318, in walk_segmenttemplate
      File "XstreamDL_CLI\extractors\dash\parser.py", line 328, in walk_segmenttimeline
      File "XstreamDL_CLI\extractors\dash\parser.py", line 358, in walk_s
    AttributeError: 'str' object has no attribute 'timestamp'
    [2588] Failed to execute script '__main__' due to unhandled exception! ```
    
    是我的指令有问题吗?版本为 v1.4.0
    感谢
    opened by samleong123 4
  • Error message

    Error message "only one segment, please download it manually"

    Hi, I'm getting the error message "only one segment, please download it manually" when I try to download using mpd URL, this problem started from this commit, before this commit I could download without any problems.

    https://github.com/xhlove/XstreamDL-CLI/commit/fa6fcfcd3d4269465dbf89e0ea76b6210d146119

    55

    The mpd link expires very fast, but here is the mpd file

    717575237053-1617969874401-master.zip

    and here is the debug log

    XstreamDL-2021-12-18_053151.zip

    I used the master branch.

    https://codeload.github.com/xhlove/XstreamDL-CLI/zip/refs/heads/master

    opened by Essam23 4
  • 大大 能帮我看一下 像这种mpd为什么下不了吗

    大大 能帮我看一下 像这种mpd为什么下不了吗

    https://manifest.prod.boltdns.net/manifest/v1/dash/live-baseurl/bccenc/6182597161001/43760353-82c2-4dbe-9b42-e1f5ecd90157/6s/manifest.mpd?behavior_id=309ea3f0-8cb4-4b2a-b93a-0a2207360930&fastly_token=NjE2ZTAwYThfMjdhNmJjYTk5NjlhMmI5YWY5NmRhYTVkZThmOTM2N2Q5NjgwNTJhMDcxNTNkMWNiN2JjNTMwOGRjMWJmMDRiNQ%3D%3D

    搜狗截图20211019002356 一直是这样~

    opened by huliad2022 4
  • WINDOWS BINARIES?

    WINDOWS BINARIES?

    Hello, 'xhlove'!

    Would you mind, creating Windows binaries, occasionally (and also working, on Windows XP, 64 bit, system/PC, if possible)?

    Thus I can actively test your programme, because I often play different live streams (like "AES-128" encrypted, and "SAMPLE-AES" encrypted, too)!?

    Good luck!!!!!

    Peter

    opened by kokolino2 4
  • please help

    please help

    Hello, every time i tried to download the video i keep getting this error Screenshot_2 could please tell me where did i go wrong? i follow ever step but still this

    opened by snowywish 3
  • 是否可以通过 mp4decrypt 加密 DASH 分片后使用 FFMPEG 合成导出HLS

    是否可以通过 mp4decrypt 加密 DASH 分片后使用 FFMPEG 合成导出HLS

    您好 感谢大佬的这个工具,非常实用,可以通过提供Key的方式录制加密DASH. 看了看原理基本是先下载分片 - merge所有分片 - mp4decrypt解密 那是否可以通过 下载3个分片 - merge 3个分片 - mp4decrypt解密 [解密1]- 再下载3个分片 - merge 3个分片 - mp4decrypt解密 [解密2] - ffmpeg 导出HLS 虽然会有延迟,可是我貌似找不到可以直接通过 ffmpeg 解密的方法 还有请大佬研究研究,有需要我帮忙也可以(虽然我也不是很会😢) 感谢

    opened by samleong123 2
Releases(1.4.0)
  • 1.4.0(Feb 12, 2022)

    感谢各位XstreamDL-CLI用户的反馈,现发布1.4.0版本,相关累计更新信息如下

    • 修复mpd解析异常
      • https://github.com/xhlove/XstreamDL-CLI/commit/2c35af59cc867816a8ba552461293d600cb894cd
    • 修复m3u8解析时自动过滤广告分段引起的误伤,增加--ad-keyword,用户可以自定义过滤关键词
      • https://github.com/xhlove/XstreamDL-CLI/commit/9a44f9b72d0ec9cf916102a26c2be0f89bdebe73
    • 对含有SegmentBaee标签的mpd进行解析支持,默认输出为单个分段
      • https://github.com/xhlove/XstreamDL-CLI/commit/fe91ce3b5c20e4d38b3b65dfd4b388e0137b64b2
    • 一些mpd字幕元数据的链接是假的,加入判断过滤
      • https://github.com/xhlove/XstreamDL-CLI/commit/4bbb18f57921ef2c6a3a69e7ca64047e45a72c2e
    • 增加--name-from-url选项,使用该选项后文件分段命名将从链接中获取
      • https://github.com/xhlove/XstreamDL-CLI/commit/d6d193a519e7a3cb4638154d711199eab0c4fdc2
      • 该选项仅目前是实验性质的,可能存在问题,使用时请注意检查
    • 修复匹配mediaPresentationDuration失败的情况
      • https://github.com/xhlove/XstreamDL-CLI/commit/ce061db03102d20a74c235918346e88d6d56ed7e
    • 增加--speed-up--speed-up-left选项
      • --speed-up 用于加速下载即将完成时剩余的分段,该选项默认不开启,目前属于实验性质
      • --speed-up-left 用于指定剩余X个分段时进行加速(实际就是重下),该选项默认为10
      • https://github.com/xhlove/XstreamDL-CLI/commit/088bf7c8ea5fa1ff6b1e4a05c8e6448b87bdc2e2
    Source code(tar.gz)
    Source code(zip)
    XstreamDL-CLI_v1.4.0.exe(13.74 MB)
  • 1.3.9(Jan 21, 2022)

    感谢各位XstreamDL-CLI用户的反馈,现发布1.3.9版本,相关累计更新信息如下

    • 完善对availabilityStartTime的处理
      • https://github.com/xhlove/XstreamDL-CLI/commit/def6b3d2e46bf1343aa303dd5c014c1ed21cd097
      • https://github.com/xhlove/XstreamDL-CLI/commit/af1e077dc93566d1f78051b795a649098ff58f82
    • 针对多个S标签的优化处理,该情况应当使用--multi-s选项
      • https://github.com/xhlove/XstreamDL-CLI/commit/34939cd6c194f24fdb94c927a3b53870de498fef
    • 优化直播流404的重试机制,一定程度上减少日志记录的数据
      • https://github.com/xhlove/XstreamDL-CLI/commit/e4101c906ac01914a58356848254c022ddb5b661
    • 日志中显示版本号,便于联动排查bug
      • https://github.com/xhlove/XstreamDL-CLI/commit/22a8406a51626381c16ba5872a1309661ea4d845
    • 修复m3u8 aes key获取异常的问题(显然几乎没有人用来下常规aes加密m3u8...不然这个bug不会这么晚才发现TAT
      • https://github.com/xhlove/XstreamDL-CLI/commit/69ad8e823209bd3564b9ad65baf4f8e7b98b0340
    • 针对SegmentTemplate嵌套在Representation标签的情况进行了适配
      • https://github.com/xhlove/XstreamDL-CLI/commit/ef983956b1072aec33a5324a81004a5089c04b1a
    • 对某种类型的mpd进行了优化处理
      • https://github.com/xhlove/XstreamDL-CLI/commit/6885230d7516671a34aecdd2be99373132fb2658
    Source code(tar.gz)
    Source code(zip)
    XstreamDL-CLI_v1.3.9.exe(13.58 MB)
  • 1.3.8(Dec 18, 2021)

  • 1.3.7(Nov 16, 2021)

  • 1.3.6(Nov 15, 2021)

  • 1.3.5(Nov 14, 2021)

  • 1.3.4(Nov 7, 2021)

    • dash直播流出现404时,对进度条优化
    • 优化dash直播流录制时间偏移
    • 音频合并默认后缀修改为m4a
    • 增加--best-quality选项,用于自动选择最佳质量
    • 增加--resolution option,用于自动选择指定分辨率
    • 完善日志以及其他各类细节异常修复
    • 加密ism自动二进制合并
    Source code(tar.gz)
    Source code(zip)
    XstreamDL-CLI_v1.3.4_fix.exe(13.22 MB)
  • 1.3.3(Oct 31, 2021)

  • 1.3.2(Oct 30, 2021)

    本次更新重点:完善了说明,并对下载中各类信息进行详细记录,方便反馈。

    • 完善更新README
    • 增加日志记录
    • 修复实际下载完成但进度条不到100%的问题
    • ffmpeg和mp4decrypt等放置到binaries文件夹,没有的会进行提示,然后默认使用全局环境变量中的
    • $Time$ 计算bug修复
    • BaseURL拼接bug修复,不再需要--prefer-content-base-url
    • 单个分段不进行下载,并显示提示
    • 处理逻辑完善
    Source code(tar.gz)
    Source code(zip)
    XstreamDL-CLI_v1.3.2.exe(13.13 MB)
  • 1.3.1(Oct 7, 2021)

    • 去除click库依赖
    • 去除rich库依赖
    • 完善BaseURL拼接逻辑
    • 完善$Number$的格式化
    • 初步简单支持dash直播流
    • 跳过403和404两种类型状态码的文件合并
    • 增加--url-patch选项用于某些有鉴权的网站
    • 增加--live和--live-duration选项用于录制直播流(dash),后者可以设置录制时长
    • 优化对m3u8的MAP识别
    • 检查logs文件夹是否存在
    • dash解析优化
    • 增加--prefer-content-base-url选项,优先使用mpd内的BaseURL
    • 其他优化
    Source code(tar.gz)
    Source code(zip)
    binaries.7z(33.66 MB)
    XstreamDL-CLI_v1.3.1_fix.exe(13.13 MB)
  • 1.3.0(Sep 13, 2021)

    • 分段数命名异常修复
    • mpd解析异常修复
    • 修复linux下合并异常
    • 添加--parse-only选项
    • 添加--show-init选项 用于区别某些流属性完全一致但实际不一致的情况
    • 添加--add-index-to-name 用于避免出现上一条所述现象时下载文件不一致的情况
    • 优化识别base_url不以http开头的情况
    • 修复一处语法错误
    • 识别带有ContentProtection的流,自动使用二进制合并
    • 根据系统语言显示不同语种提示(中/英)
    • m3u8 map分段缺失修复(fix ver)
    Source code(tar.gz)
    Source code(zip)
    XstreamDL-CLI_v1.3.0_fix.exe(14.51 MB)
  • 1.2.9(Aug 7, 2021)

  • 1.2.8(Aug 7, 2021)

  • 1.2.7(Aug 3, 2021)

  • 1.2.6(Aug 1, 2021)

  • 1.2.5(May 31, 2021)

  • 1.2.4(May 19, 2021)

  • 1.2.3(May 15, 2021)

  • 1.2.2(May 3, 2021)

  • 1.2.0(Apr 22, 2021)

  • 1.1.0(Mar 7, 2021)

    v1.1.0正式发布

    • 增加自定义user-agent
    • 增加自定义referer
    • 增加自定义请求头
    • 增加安全关闭连接设定
    • 增加多轨道下载选择设定
    • 支持文件夹内m3u8批量下载
    • 下载速度和稳定性提升
    • 各类BUG修复和优化

    追求下载速度可开启--disable-force-close设定

    Source code(tar.gz)
    Source code(zip)
    XstreamDL-CLI_v1.1.0.exe(14.47 MB)
  • 1.0.0(Feb 4, 2021)

Owner
xhlove
xhlove
A Python module to encrypt and decrypt data with AES-128 CFB mode.

cryptocfb A Python module to encrypt and decrypt data with AES-128 CFB mode. This module supports 8/64/128-bit CFB mode. It can encrypt and decrypt la

Quan Lin 1 Feb 23, 2022