PyAV is a Pythonic binding for the FFmpeg libraries.

Related tags

Video python ffmpeg pyav
Overview

PyAV

GitHub Test Status
Gitter Chat Documentation
GitHub Python Package Index Conda Forge

PyAV is a Pythonic binding for the FFmpeg libraries. We aim to provide all of the power and control of the underlying library, but manage the gritty details as much as possible.

PyAV is for direct and precise access to your media via containers, streams, packets, codecs, and frames. It exposes a few transformations of that data, and helps you get your data to/from other packages (e.g. Numpy and Pillow).

This power does come with some responsibility as working with media is horrendously complicated and PyAV can't abstract it away or make all the best decisions for you. If the ffmpeg command does the job without you bending over backwards, PyAV is likely going to be more of a hindrance than a help.

But where you can't work without it, PyAV is a critical tool.

Installation

Due to the complexity of the dependencies, PyAV is not always the easiest Python package to install from source. Since release 8.0.0 binary wheels are provided on PyPI for Linux, Mac and Windows linked against a modern FFmpeg. You can install these wheels by running:

pip install av

If you want to use your existing FFmpeg/Libav, the C-source version of PyAV is on PyPI too:

pip install av --no-binary av

Alternative installation methods

Another way of installing PyAV is via conda-forge:

conda install av -c conda-forge

See the Conda quick install docs to get started with (mini)Conda.

And if you want to build from the absolute source (for development or testing):

git clone [email protected]:PyAV-Org/PyAV
cd PyAV
source scripts/activate.sh

# Either install the testing dependencies:
pip install --upgrade -r tests/requirements.txt
# or have it all, including FFmpeg, built/installed for you:
./scripts/build-deps

# Build PyAV.
make

Have fun, read the docs, come chat with us, and good luck!

Comments
  • Refcount/Memory issues

    Refcount/Memory issues

    I m not versed with Pyhton refcounting and memory management but I have experienced some problems with av.VideoFrame

    In my application I save numpy arrays to a video file using pyav. For this I call:

    frame = av.VideoFrame(input_frame.width,input_frame.height,'bgr24') to create a frame for each image.

    Since this is done inside a fn that is called continuously, I expect frame to go out of scope and be garbage collected. On MacOS this is the case and I don't have any trouble.

    However on Linux I quickly run out of memory (I can watch the python process growing in memory use until the system hangs.)

    A simple workaround is creating a single instance of frame attaching it to self. and re-using it by updating the planes.

    Any ideas why this happens when I make new frames?

    bug 
    opened by mkassner 52
  • `av==9.1.1` wheels seem to be be not packaged properly for Windows and Python 3.8/9

    `av==9.1.1` wheels seem to be be not packaged properly for Windows and Python 3.8/9

    Overview

    av==9.1.1 wheel released a couple of hours ago seems to be not packaged properly for Windows and Python 3.8 and 3.9.

    Expected behavior

    Import without issues.

    Actual behavior

    Traceback (most recent call last):
      File "C:\Users\circleci\project\test\datasets_utils.py", line 90, in _import
        module = importlib.import_module(package)
      File "C:\Users\circleci\project\env\lib\importlib\__init__.py", line 127, in import_module
        return _bootstrap._gcd_import(name[level:], package, level)
      File "<frozen importlib._bootstrap>", line 1014, in _gcd_import
      File "<frozen importlib._bootstrap>", line 991, in _find_and_load
      File "<frozen importlib._bootstrap>", line 975, in _find_and_load_unlocked
      File "<frozen importlib._bootstrap>", line 671, in _load_unlocked
      File "<frozen importlib._bootstrap_external>", line 843, in exec_module
      File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed
      File "C:\Users\circleci\project\env\lib\site-packages\av\__init__.py", line 44, in <module>
        from av._core import time_base, library_versions
    ImportError: DLL load failed while importing _core: The specified module could not be found.
    

    Investigation

    CI runs:

    Reproduction

    import av
    

    Versions

    • OS: Windows
    • PyAV runtime: 9.1.1
    bug help wanted 
    opened by pmeier 36
  • Decoding with multiple threads?

    Decoding with multiple threads?

    I was doing some decoding benchmarks with PyAV, and had a question about decoding in multiple threads. I'm trying to decode several mp4 files, each one in it's own thread. However, total CPU usage never exceeds 100%, it's almost as if PyAV has some sort of internal lock that doesn't allow more than one thread to decode at a time.

    Does anyone understand why decoding multiple files with PyAV doesn't scale with multiple threads? I've looked through the code, but am not seeing anything super obvious. This is with Python 3.6, CPython on MacOS. The decode loop I'm running in each thread is very simple, basically:

    def decode(input_file):
        container = av.open(input_file)
        for _ in container.decode(video=0):
            pass
    
    def main():
        for input_file in sys.argv[1:]:
            t = threading.Thread(target=decode, args=(input_file,))
            t.start()
    

    PS: I haven't tried experimenting with PyPy yet, but I wouldn't think CPython's GIL is a problem, since almost all the heavy lifting is being done in ffmpeg libraries, not Python code. Correct me if I'm wrong on this though...

    enhancement design 
    opened by jd20 30
  • Video Decoding from Custom Stream: Way to Set Buffer Size?

    Video Decoding from Custom Stream: Way to Set Buffer Size?

    Oh, am I impressed with how easy PyAV makes using ffmpeg! Thanks.

    I am decoding video from a custom stream, I am managing to decode data in RAM and get image frames out. But the decoder wants to buffer data before giving me the first frame out. I would like to get each frame out as promptly as possible, and as I feed more data in, get out new frames--again as promptly as possibly.

    Googling around there seems to be an ffmpeg option to control this: codec_ctx->delay (mentioned in http://ffmpeg.org/pipermail/libav-user/2014-December/007672.html).

    Is that what I am looking for? Is there a way to set it via an av.open() option?

    Thanks!

    -kb

    enhancement stale 
    opened by kentborg 29
  • Data (not file or network protocol) interface to libav

    Data (not file or network protocol) interface to libav

    Hi,

    We've been struggling (#445) with using libav to decode packets of compressed audio that come into our server via a dedicated websocket connection.

    It appears that the only way to send data to libav is though some file name (which may actually be a URL of some sort). So our solution is currently to use a unix named pipe, av.open(pipe) a container, then read packets, frames, from the stream in the container. This worked OK-ish, until we hit #445.

    ## consume reads data from a named pipe, and returns the data through framesq.
    ## It should be called from a separate process. 
    ## Decompressed data is returned in a Queue framesq.
    def consume(pipename, framesq, doneq, sr):
        try:
            container = av.open(pipename, "r", options={'max_delay': '10000'})
            audio = container.streams[0]
    
            resampler = av.AudioResampler(
                format = av.AudioFormat("s16p").packed,
                layout = int(1),
                rate = sr,
            )
    
            for packet in container.demux(audio):
                for frame in packet.decode():
                    frame.pts = None
                    for i, plane in enumerate(frame.planes):
                        frame = resampler.resample(frame)
                        bytes = frame.planes[0].to_bytes()
                        framesq.put(bytes)
    
        except Exception as e:
            logger.debug("Error in AV consumption: %r", e)
    
        doneq.put(True) ## signal end of data  
    

    In the light of solution #325, I've been experimenting with UDP and unix domain sockets to set up the transport of data, to circumvent the problem that ffmpeg options don't seem to make much of a difference. However I can't get any of these to work (a boilerplate example would be appreciated:-).

    But really, we would just want to send the actual compressed data (incl headers etc) to libav synchoneously, and receive the decompressed data out whenever libav has enough information to output another frame. But we can't find an entry to pyav / libav / ffmpeg do do such a thing, hence the workaround above.

    Does anyone know if it would be possible to get a chunk-by-chunk implementation of the decoding?

    user help 
    opened by davidavdav 27
  • WIP: Support Python 3.

    WIP: Support Python 3.

    This is a work in progress. The package builds and installs under py3, but it fails on import like so:

    In [1]: import av
    ---------------------------------------------------------------------------
    ImportError                               Traceback (most recent call last)
    <ipython-input-1-55f15964059f> in <module>()
    ----> 1 import av
    
    /Users/danielallan/Documents/Repos/PyAV/av/__init__.py in <module>()
          8 
          9 # For convenience, IMPORT ALL OF THE THINGS (that are constructable by the user).
    ---> 10 from .audio.fifo import AudioFifo
         11 from .audio.format import AudioFormat
         12 from .audio.frame import AudioFrame
    
    /Users/danielallan/container.pxd in init av.audio.fifo (/Users/danielallan/Documents/Repos/PyAV/src/av/audio/fifo.c:2743)()
    
    /Users/danielallan/frame.pxd in init av.container (/Users/danielallan/Documents/Repos/PyAV/src/av/container.c:5922)()
    
    /Users/danielallan/stream.pxd in init av.frame (/Users/danielallan/Documents/Repos/PyAV/src/av/frame.c:2108)()
    
    /Users/danielallan/frame.pxd in init av.stream (/Users/danielallan/Documents/Repos/PyAV/src/av/stream.c:4773)()
    
    /Users/danielallan/plane.pxd in init av.audio.frame (/Users/danielallan/Documents/Repos/PyAV/src/av/audio/frame.c:2564)()
    
    ImportError: dlopen(/Users/danielallan/Documents/Repos/PyAV/av/plane.so, 2): Symbol not found: _PyString_FromStringAndSize
      Referenced from: /Users/danielallan/Documents/Repos/PyAV/av/plane.so
      Expected in: dynamic lookup
    
    opened by danielballan 26
  • Set hwaccel ?

    Set hwaccel ?

    When using ffmpeg, we can use -hwaccel <hwtype> (for example with hwtype = cuvid) before -i to instruct ffmpeg to use the specified decoders, instead of the default one. In the case of cuvid, this would be h264_cuvid, hevc_cuvid, vp9_cuvid, etc.

    This is really useful when we don't know in advance what codec a container uses, but we still want to use the hardware-accelerated decoders.

    Is there a way to default to the hardware-accelerated decoders in PyAV?

    Thanks!

    enhancement research 
    opened by wegel 24
  • Add hardware acceleration to video decoding

    Add hardware acceleration to video decoding

    • Add optional dictionary parameter to input open allowing the user to pass setting for hardware acceleration
    • Add nvidia libraries to the ffmpeg build if available
    enhancement 
    opened by rvillalba-novetta 22
  • Expose encoding/decoding raw packets without streams

    Expose encoding/decoding raw packets without streams

    Streams are not actually necessary for encoding/decoding, only muxing/demuxing. Refactor encoding/decoding onto a CodecContext, and then have a {Video,Audio,Subtitle}CodecContext which implements the specifics functions.

    Do this for #154.

    enhancement 
    opened by mikeboers 22
  • PyAV does not build with ffmpeg 5.x

    PyAV does not build with ffmpeg 5.x

    Overview

    av/stream.c uses __pyx_v_c_stream->codec->codec_type, but it is deprecated ~4 years ago and will be removed in next ffmpeg version (libavformat>58).

    In https://trac.ffmpeg.org/wiki/Bump59

    ... libavformat - AVFormatContext.open_cb replaced by io_open/io_close - AVStream.codec removed; stream information is now exported through AVStream.codecpar; for decoding/encoding the users should allocate a separate codec context - HTTP and RTSP protocol's 'user-agent' option renamed to 'user_agent' - ...

    Expected behavior

    pip install av --no-binary av works with the latest FFmpeg version

    Actual behavior

    Build failed due to the above compile error

    Build report:

    ...
        src/av/stream.c: In function ‘__pyx_f_2av_6stream_wrap_stream’:
        src/av/stream.c:2257:27: error: ‘struct AVStream’ has no member named ‘codec’
         2257 |   switch (__pyx_v_c_stream->codec->codec_type) {
              |                           ^~
        src/av/stream.c: In function ‘__pyx_f_2av_6stream_6Stream__init’:
        src/av/stream.c:2875:29: error: ‘struct AVStream’ has no member named ‘codec’
         2875 |   __pyx_t_1 = __pyx_v_stream->codec;
              |                             ^~
        src/av/stream.c:2995:15: warning: assignment discards ‘const’ qualifier from pointer target type [-Wdiscarded-qualifiers]
         2995 |     __pyx_t_6 = __pyx_v_self->_codec_context->codec;
              |               ^
        src/av/stream.c: In function ‘__pyx_f_2av_6stream_6Stream__finalize_for_output’:
        src/av/stream.c:3688:130: error: ‘struct AVStream’ has no member named ‘codec’
         3688 |   __pyx_t_7 = __pyx_f_2av_5error_err_check(avcodec_parameters_from_context(__pyx_v_self->_stream->codecpar, __pyx_v_self->_stream->codec), 0, NULL); if (unlikely(__pyx_t_7 == ((int)-1))) __PYX_ERR(0, 144, __pyx_L1_error)
              |                                                                                                                                  ^~
        error: command '<here is my_gcc_path>' failed with exit code 1
     ...
    

    Reproduction

    build pyAV with latest ffmpeg library

    Versions

    • OS: Linux

    • FFmpeg:

    ffmpeg version N-102619-gebedd26eef Copyright (c) 2000-2021 the FFmpeg developers
    built with gcc 10.2.0 (GCC)
    configuration: --prefix=<my path> --extra-cflags='-fPIC -m64' --enable-shared --disable-static --enable-gpl --enable-nonfree --enable-libvpx --enable-libmp3lame --enable-libopus --enable-libwebp --enable-libass --enable-cuda-nvcc --enable-libnpp --enable-cuvid --enable-nvenc --nvccflags='-gencode arch=compute_60,code=sm_60 -O2' --enable-libx264 --enable-libx265 --enable-libfdk-aac --enable-libwebp --enable-openssl --enable-libsvtav1 --enable-libvorbis --enable-fontconfig --enable-pic
    libavutil      57.  0.100 / 57.  0.100
    libavcodec     59.  1.100 / 59.  1.100
    libavformat    59.  2.101 / 59.  2.101
    libavdevice    59.  0.100 / 59.  0.100
    libavfilter     8.  0.101 /  8.  0.101
    libswscale      6.  0.100 /  6.  0.100
    libswresample   4.  0.100 /  4.  0.100
    libpostproc    56.  0.100 / 56.  0.100
    

    Research

    I have done the following:

    build help wanted 
    opened by zzjjbb 21
  • Didn't find 'avformat_open_input' Ubuntu

    Didn't find 'avformat_open_input' Ubuntu

       We didn't find `avformat_open_input` in the libraries.
        We look for it only as a sanity check to make sure the build
        process is working as expected. It is not, so we must abort.
    
        Please open a ticket at https://github.com/mikeboers/PyAV/issues
        with the folowing information:
    
        PyAV: 0.3.1
        Python: 2.7.6 (default, Jun 22 2015, 17:58:13) \n[GCC 4.8.2]
        platform: Linux-3.19.0-59-generic-x86_64-with-Ubuntu-14.04-trusty
        extension_extra:
            libraries: ['X11', 'Xext', 'Xv', 'avcodec', 'avdevice', 'avfilter', 'avformat', 'avutil', 'bz2', 'm', 'swresample', 'swscale', 'xcb', 'xcb-render', 'xcb-shape', 'xcb-xfixes', 'z']
            library_dirs: ['/usr/local/lib']
            include_dirs: ['/usr/local/include', 'include', '/usr/include/python2.7']
        config_macros:
            PYAV_VERSION=0.3.1
            PYAV_VERSION_STR="0.3.1"
            PYAV_COMMIT_STR="unknown-commit"
            PYAV_HAVE_LIBSWRESAMPLE=1
            PYAV_HAVE_AV_CALLOC=1
            PYAV_HAVE_AV_FRAME_GET_BEST_EFFORT_TIMESTAMP=1
    
        ----------------------------------------
    Command "/usr/bin/python -u -c "import setuptools, tokenize;__file__='/tmp/pip-build-A8DzRj/av/setup.py';exec(compile(getattr(tokenize, 'open', open)(__file__).read().replace('\r\n', '\n'), __file__, 'exec'))" install --record /tmp/pip-txMrAL-record/install-record.txt --single-version-externally-managed --compile" failed with error code 1 in /tmp/pip-build-A8DzRj/av/
    

    EDIT by @mikeboers: down the page I wrote:

    I'm pretty sure there there is some sort of compilation error that arises during linking to the static library vs the dynamic ones. The "fix", I believe, is to discuss this in the error message, and expose a flag so that you can try again and print out all of the compiler errors.

    opened by katrinabrock 21
  • Segfault on container.close() with debugpy

    Segfault on container.close() with debugpy

    Overview

    Hi all. I'm experiencing some weird behavior when closing an InputContainer. When debugging with debugpy, I'll get a segmentation fault if I call container.close() or let the object go out of scope with a context manager. I've encountered this issue on different machines, OSes, vms, and it only seems to happen while debugging and after accessing data from a Stream object.

    Expected behavior

    InputContainer objects should be garbage collected with no segfault

    Actual behavior

    A SIGSEGV signal is emitted when an InputContainer is closed

    Traceback:

    ============================= test session starts ==============================
    platform linux -- Python 3.10.6, pytest-7.2.0, pluggy-1.0.0
    rootdir: /home/vm/Desktop/pyav-test
    collected 1 item
    
    test_av.py
    Fatal Python error: Segmentation fault
    
    Thread 0x00007ff86e5fb640 (most recent call first):
      File "/usr/lib/python3.10/threading.py", line 324 in wait
      File "/usr/lib/python3.10/threading.py", line 607 in wait
      File "/home/vm/.vscode/extensions/ms-python.python-2022.20.1/pythonFiles/lib/python/debugpy/_vendored/pydevd/pydevd.py", line 261 in _on_run
      File "/home/vm/.vscode/extensions/ms-python.python-2022.20.1/pythonFiles/lib/python/debugpy/_vendored/pydevd/_pydevd_bundle/pydevd_daemon_thread.py", line 49 in run
      File "/usr/lib/python3.10/threading.py", line 1016 in _bootstrap_inner
      File "/usr/lib/python3.10/threading.py", line 973 in _bootstrap
    
    Thread 0x00007ff86edfc640 (most recent call first):
      File "/usr/lib/python3.10/threading.py", line 324 in wait
      File "/usr/lib/python3.10/threading.py", line 607 in wait
      File "/home/vm/.vscode/extensions/ms-python.python-2022.20.1/pythonFiles/lib/python/debugpy/_vendored/pydevd/pydevd.py", line 215 in _on_run
      File "/home/vm/.vscode/extensions/ms-python.python-2022.20.1/pythonFiles/lib/python/debugpy/_vendored/pydevd/_pydevd_bundle/pydevd_daemon_thread.py", line 49 in run
      File "/usr/lib/python3.10/threading.py", line 1016 in _bootstrap_inner
      File "/usr/lib/python3.10/threading.py", line 973 in _bootstrap
    
    Thread 0x00007ff86f5fd640 (most recent call first):
      File "/usr/lib/python3.10/threading.py", line 320 in wait
      File "/usr/lib/python3.10/threading.py", line 607 in wait
      File "/home/vm/.vscode/extensions/ms-python.python-2022.20.1/pythonFiles/lib/python/debugpy/_vendored/pydevd/_pydevd_bundle/pydevd_timeout.py", line 43 in _on_run
      File "/home/vm/.vscode/extensions/ms-python.python-2022.20.1/pythonFiles/lib/python/debugpy/_vendored/pydevd/_pydevd_bundle/pydevd_daemon_thread.py", line 49 in run
      File "/usr/lib/python3.10/threading.py", line 1016 in _bootstrap_inner
      File "/usr/lib/python3.10/threading.py", line 973 in _bootstrap
    Thread 0x00007ff86fdfe640 (most recent call first):
      File "/home/vm/.vscode/extensions/ms-python.python-2022.20.1/pythonFiles/lib/python/debugpy/_vendored/pydevd/_pydevd_bundle/pydevd_comm.py", line 204 in _read_line
      File "/home/vm/.vscode/extensions/ms-python.python-2022.20.1/pythonFiles/lib/python/debugpy/_vendored/pydevd/_pydevd_bundle/pydevd_comm.py", line 222 in _on_run
      File "/home/vm/.vscode/extensions/ms-python.python-2022.20.1/pythonFiles/lib/python/debugpy/_vendored/pydevd/_pydevd_bundle/pydevd_daemon_thread.py", line 49 in run
      File "/usr/lib/python3.10/threading.py", line 1016 in _bootstrap_inner
      File "/usr/lib/python3.10/threading.py", line 973 in _bootstrap
    
    Thread 0x00007ff8705ff640 (most recent call first):
      File "/usr/lib/python3.10/threading.py", line 324 in wait
      File "/usr/lib/python3.10/queue.py", line 180 in get
      File "/home/vm/.vscode/extensions/ms-python.python-2022.20.1/pythonFiles/lib/python/debugpy/_vendored/pydevd/_pydevd_bundle/pydevd_comm.py", line 367 in _on_run
      File "/home/vm/.vscode/extensions/ms-python.python-2022.20.1/pythonFiles/lib/python/debugpy/_vendored/pydevd/_pydevd_bundle/pydevd_daemon_thread.py", line 49 in run
      File "/usr/lib/python3.10/threading.py", line 1016 in _bootstrap_inner
      File "/usr/lib/python3.10/threading.py", line 973 in _bootstrap
    
    Current thread 0x00007ff872668000 (most recent call first):
      File "/home/vm/.vscode/extensions/ms-python.python-2022.20.1/pythonFiles/lib/python/debugpy/_vendored/pydevd/_pydevd_bundle/pydevd_safe_repr.py", line 128 in _is_long_iter
      File "/home/vm/.vscode/extensions/ms-python.python-2022.20.1/pythonFiles/lib/python/debugpy/_vendored/pydevd/_pydevd_bundle/pydevd_safe_repr.py", line 112 in _repr
      File "/home/vm/.vscode/extensions/ms-python.python-2022.20.1/pythonFiles/lib/python/debugpy/_vendored/pydevd/_pydevd_bundle/pydevd_safe_repr.py", line 196 in _repr_iter
      File "/home/vm/.vscode/extensions/ms-python.python-2022.20.1/pythonFiles/lib/python/debugpy/_vendored/pydevd/_pydevd_bundle/pydevd_safe_repr.py", line 77 in __call__
      File "/home/vm/.vscode/extensions/ms-python.python-2022.20.1/pythonFiles/lib/python/debugpy/_vendored/pydevd/_pydevd_bundle/pydevd_xml.py", line 315 in get_variable_details
      File "/home/vm/.vscode/extensions/ms-python.python-2022.20.1/pythonFiles/lib/python/debugpy/_vendored/pydevd/_pydevd_bundle/pydevd_suspended_frames.py", line 51 in get_var_data
      File "/home/vm/.vscode/extensions/ms-python.python-2022.20.1/pythonFiles/lib/python/debugpy/_vendored/pydevd/_pydevd_bundle/pydevd_comm.py", line 753 in internal_get_variable_json
      File "/home/vm/.vscode/extensions/ms-python.python-2022.20.1/pythonFiles/lib/python/debugpy/_vendored/pydevd/_pydevd_bundle/pydevd_constants.py", line 475 in new_func
      File "/home/vm/.vscode/extensions/ms-python.python-2022.20.1/pythonFiles/lib/python/debugpy/_vendored/pydevd/_pydevd_bundle/pydevd_comm.py", line 527 in do_it
      File "/home/vm/.vscode/extensions/ms-python.python-2022.20.1/pythonFiles/lib/python/debugpy/_vendored/pydevd/pydevd.py", line 1772 in process_internal_commands
      File "/home/vm/.vscode/extensions/ms-python.python-2022.20.1/pythonFiles/lib/python/debugpy/_vendored/pydevd/pydevd.py", line 2097 in _do_wait_suspend
      File "/home/vm/.vscode/extensions/ms-python.python-2022.20.1/pythonFiles/lib/python/debugpy/_vendored/pydevd/pydevd.py", line 2062 in do_wait_suspend
      File "/home/vm/Desktop/pyav-test/test_av.py", line 11 in test_av
      File "/home/vm/Desktop/pyav-test/env/lib/python3.10/site-packages/_pytest/python.py", line 195 in pytest_pyfunc_call
      File "/home/vm/Desktop/pyav-test/env/lib/python3.10/site-packages/pluggy/_callers.py", line 39 in _multicall
      File "/home/vm/Desktop/pyav-test/env/lib/python3.10/site-packages/pluggy/_manager.py", line 80 in _hookexec
      File "/home/vm/Desktop/pyav-test/env/lib/python3.10/site-packages/pluggy/_hooks.py", line 265 in __call__
      File "/home/vm/Desktop/pyav-test/env/lib/python3.10/site-packages/_pytest/python.py", line 1789 in runtest
      File "/home/vm/Desktop/pyav-test/env/lib/python3.10/site-packages/_pytest/runner.py", line 167 in pytest_runtest_call
      File "/home/vm/Desktop/pyav-test/env/lib/python3.10/site-packages/pluggy/_callers.py", line 39 in _multicall
      File "/home/vm/Desktop/pyav-test/env/lib/python3.10/site-packages/pluggy/_manager.py", line 80 in _hookexec
      File "/home/vm/Desktop/pyav-test/env/lib/python3.10/site-packages/pluggy/_hooks.py", line 265 in __call__
      File "/home/vm/Desktop/pyav-test/env/lib/python3.10/site-packages/_pytest/runner.py", line 260 in <lambda>
      File "/home/vm/Desktop/pyav-test/env/lib/python3.10/site-packages/_pytest/runner.py", line 339 in from_call
      File "/home/vm/Desktop/pyav-test/env/lib/python3.10/site-packages/_pytest/runner.py", line 259 in call_runtest_hook
      File "/home/vm/Desktop/pyav-test/env/lib/python3.10/site-packages/_pytest/runner.py", line 220 in call_and_report
      File "/home/vm/Desktop/pyav-test/env/lib/python3.10/site-packages/_pytest/runner.py", line 131 in runtestprotocol
      File "/home/vm/Desktop/pyav-test/env/lib/python3.10/site-packages/_pytest/runner.py", line 112 in pytest_runtest_protocol
      File "/home/vm/Desktop/pyav-test/env/lib/python3.10/site-packages/pluggy/_callers.py", line 39 in _multicall
      File "/home/vm/Desktop/pyav-test/env/lib/python3.10/site-packages/pluggy/_manager.py", line 80 in _hookexec
      File "/home/vm/Desktop/pyav-test/env/lib/python3.10/site-packages/pluggy/_hooks.py", line 265 in __call__
      File "/home/vm/Desktop/pyav-test/env/lib/python3.10/site-packages/_pytest/main.py", line 349 in pytest_runtestloop
      File "/home/vm/Desktop/pyav-test/env/lib/python3.10/site-packages/pluggy/_callers.py", line 39 in _multicall
      File "/home/vm/Desktop/pyav-test/env/lib/python3.10/site-packages/pluggy/_manager.py", line 80 in _hookexec
      File "/home/vm/Desktop/pyav-test/env/lib/python3.10/site-packages/pluggy/_hooks.py", line 265 in __call__
      File "/home/vm/Desktop/pyav-test/env/lib/python3.10/site-packages/_pytest/main.py", line 324 in _main
      File "/home/vm/Desktop/pyav-test/env/lib/python3.10/site-packages/_pytest/main.py", line 270 in wrap_session
      File "/home/vm/Desktop/pyav-test/env/lib/python3.10/site-packages/_pytest/main.py", line 317 in pytest_cmdline_main
      File "/home/vm/Desktop/pyav-test/env/lib/python3.10/site-packages/pluggy/_callers.py", line 39 in _multicall
      File "/home/vm/Desktop/pyav-test/env/lib/python3.10/site-packages/pluggy/_manager.py", line 80 in _hookexec
      File "/home/vm/Desktop/pyav-test/env/lib/python3.10/site-packages/pluggy/_hooks.py", line 265 in __call__
      File "/home/vm/Desktop/pyav-test/env/lib/python3.10/site-packages/_pytest/config/__init__.py", line 167 in main
      File "/home/vm/.vscode/extensions/ms-python.python-2022.20.1/pythonFiles/testlauncher.py", line 36 in run
      File "/home/vm/.vscode/extensions/ms-python.python-2022.20.1/pythonFiles/testlauncher.py", line 44 in <module>
      File "/home/vm/.vscode/extensions/ms-python.python-2022.20.1/pythonFiles/lib/python/debugpy/_vendored/pydevd/_pydevd_bundle/pydevd_runpy.py", line 124 in _run_code
      File "/home/vm/.vscode/extensions/ms-python.python-2022.20.1/pythonFiles/lib/python/debugpy/_vendored/pydevd/_pydevd_bundle/pydevd_runpy.py", line 135 in _run_module_code
      File "/home/vm/.vscode/extensions/ms-python.python-2022.20.1/pythonFiles/lib/python/debugpy/_vendored/pydevd/_pydevd_bundle/pydevd_runpy.py", line 321 in run_path
      File "/home/vm/.vscode/extensions/ms-python.python-2022.20.1/pythonFiles/lib/python/debugpy/adapter/../../debugpy/launcher/../../debugpy/../debugpy/server/cli.py", line 284 in run_file
      File "/home/vm/.vscode/extensions/ms-python.python-2022.20.1/pythonFiles/lib/python/debugpy/adapter/../../debugpy/launcher/../../debugpy/../debugpy/server/cli.py", line 430 in main
      File "/home/vm/.vscode/extensions/ms-python.python-2022.20.1/pythonFiles/lib/python/debugpy/adapter/../../debugpy/launcher/../../debugpy/__main__.py", line 39 in <module>
      File "/usr/lib/python3.10/runpy.py", line 86 in _run_code
      File "/usr/lib/python3.10/runpy.py", line 196 in _run_module_as_main
    
    Extension modules: _pydevd_bundle.pydevd_cython, _pydevd_frame_eval.pydevd_frame_evaluator, av._core, av.logging, av.bytesource, av.buffer, av.audio.format, av.enum, av.error, av.utils, av.option, av.descriptor, av.container.pyio, av.dictionary, av.format, av.stream, av.container.streams, av.sidedata.motionvectors, av.sidedata.sidedata, av.packet, av.container.input, av.container.output, av.container.core, av.codec.context, av.video.format, av.video.reformatter, av.plane, av.video.plane, av.video.frame, av.video.stream, av.codec.codec, av.frame, av.audio.layout, av.audio.plane, av.audio.frame, av.audio.stream, av.audio.fifo, av.filter.pad, av.filter.link, av.filter.context, av.filter.graph, av.filter.filter, av.audio.resampler, av.video.codeccontext (total: 44)
    

    Investigation

    I've been able to reproduce this behavior on the following platforms

    • Windows 11 22H2
    • Ubuntu 20.04 (WSL2)
    • Ubuntu 22.04 (VM)
    • MacOS Ventura 13.0.1

    Each segfault happens after closing an InputContainer after accessing any of its Stream objects

    Reproduction

    The following code should reproduce the error

    import av
    import av.datasets
    
    content = av.datasets.curated("pexels/time-lapse-video-of-night-sky-857195.mp4")
    input_container = av.open(content)
    video_stream = input_container.streams.get(video=0)
    total_frames = video_stream.frames
    
    input_container.close() # <--- segfault happens here
    
    print(total_frames)
    
    

    or

    import av
    import av.datasets
    
    content = av.datasets.curated("pexels/time-lapse-video-of-night-sky-857195.mp4")
    with av.open(content) as input_container:
        video_stream = input_container.streams.get(video=0)
        total_frames = video_stream.frames
        # segfault happens as this goes out of scope
    
    print(total_frames)
    

    Versions

    • OS: Ubuntu 20.04 (WSL2)
    • PyAV runtime:
    PyAV v10.0.0
    library configuration: --disable-static --enable-shared --libdir=/tmp/vendor/lib --prefix=/tmp/vendor --disable-alsa --disable-doc --disable-mediafoundation --enable-fontconfig --enable-gmp --enable-gnutls --enable-gpl --enable-libaom --enable-libass --enable-libbluray --enable-libdav1d --enable-libfreetype --enable-libmp3lame --enable-libopencore-amrnb --enable-libopencore-amrwb --enable-libopenjpeg --enable-libopus --enable-libspeex --enable-libtheora --enable-libtwolame --enable-libvorbis --enable-libvpx --enable-libx264 --enable-libx265 --enable-libxcb --enable-libxml2 --enable-libxvid --enable-lzma --enable-version3 --enable-zlib
    library license: GPL version 3 or later
    libavcodec     59. 37.100
    libavdevice    59.  7.100
    libavfilter     8. 44.100
    libavformat    59. 27.100
    libavutil      57. 28.100
    libswresample   4.  7.100
    libswscale      6.  7.100
    
    • PyAV build:
    pip installed
    
    • FFmpeg:
    ffmpeg version 4.2.7-0ubuntu0.1 Copyright (c) 2000-2022 the FFmpeg developers
    built with gcc 9 (Ubuntu 9.4.0-1ubuntu1~20.04.1)
    configuration: --prefix=/usr --extra-version=0ubuntu0.1 --toolchain=hardened --libdir=/usr/lib/x86_64-linux-gnu --incdir=/usr/include/x86_64-linux-gnu --arch=amd64 --enable-gpl --disable-stripping --enable-avresample --disable-filter=resample --enable-avisynth --enable-gnutls --enable-ladspa --enable-libaom --enable-libass --enable-libbluray --enable-libbs2b --enable-libcaca --enable-libcdio --enable-libcodec2 --enable-libflite --enable-libfontconfig --enable-libfreetype --enable-libfribidi --enable-libgme --enable-libgsm --enable-libjack --enable-libmp3lame --enable-libmysofa --enable-libopenjpeg --enable-libopenmpt --enable-libopus --enable-libpulse --enable-librsvg --enable-librubberband --enable-libshine --enable-libsnappy --enable-libsoxr --enable-libspeex --enable-libssh --enable-libtheora --enable-libtwolame --enable-libvidstab --enable-libvorbis --enable-libvpx --enable-libwavpack --enable-libwebp --enable-libx265 --enable-libxml2 --enable-libxvid --enable-libzmq --enable-libzvbi --enable-lv2 --enable-omx --enable-openal --enable-opencl --enable-opengl --enable-sdl2 --enable-libdc1394 --enable-libdrm --enable-libiec61883 --enable-nvenc --enable-chromaprint --enable-frei0r --enable-libx264 --enable-shared
    libavutil      56. 31.100 / 56. 31.100
    libavcodec     58. 54.100 / 58. 54.100
    libavformat    58. 29.100 / 58. 29.100
    libavdevice    58.  8.100 / 58.  8.100
    libavfilter     7. 57.100 /  7. 57.100
    libavresample   4.  0.  0 /  4.  0.  0
    libswscale      5.  5.100 /  5.  5.100
    libswresample   3.  5.100 /  3.  5.100
    libpostproc    55.  5.100 / 55.  5.100
    

    Research

    I have done the following:

    Additional context

    Strangely enough, I've only encountered this issue while debugging. The code runs fine if no debugger is attached.

    bug 
    opened by king-cony 0
  • How do I load and store an mp4 video file?

    How do I load and store an mp4 video file?

    Overview

    I want to have two functions:

    • one to load a video array from an .mp4 file
    • to store a video array to an .mp4 file

    I tried implementing those as follows:

    
    from ctypes import Union
    from pathlib import Path
    from typing import BinaryIO
    
    import av
    import numpy as np
    
    
    def load(file_path, **kwargs) -> np.ndarray:
        frames = []
        with av.open(file_path, **kwargs) as container:
            for i, frame in enumerate(container.decode(video=0)):
                img = frame.to_image()
                frames.append(img)
    
        return np.moveaxis(np.stack(frames), -3, -2)
    
    
    # following this approach: https://github.com/PyAV-Org/PyAV/issues/498
    def save_to_file(
        np_tensor: np.ndarray,
        file_path: str,
        frame_rate: int = 30,
        codec: str = 'mpeg4',
    ) -> None:
    
        video_tensor = np.moveaxis(np.clip(np_tensor, 0, 255), -3, -2).astype('uint8')
    
        with av.open(file_path, mode='w') as container:
            if video_tensor.ndim == 3:
                video_tensor = np.expand_dims(video_tensor, axis=0)
    
            stream = container.add_stream(codec, rate=frame_rate)
            stream.width = video_tensor.shape[-2]
            stream.height = video_tensor.shape[-3]
            stream.pix_fmt = 'yuv420p'
    
            for vid in video_tensor:
                frame = av.VideoFrame.from_ndarray(vid, format="rgb24")
                frame.pts = None
    
                for packet in stream.encode(frame):
                    container.mux(packet)
    
            for packet in stream.encode(None):
                container.mux(packet)
    
    
    tmp_file = str(Path.cwd() / 'tmp.mp4')
    
    video_tensor = np.array(
        [[
            [[1, 2, 3], [4, 5, 6]],
            [[7, 8, 9], [10, 11, 12]],
            [[13, 14, 15], [16, 17, 18]],
            [[19, 20, 21], [22, 23, 24]],
        ]]
    )
    
    
    save_to_file(np_tensor=video_tensor, file_path=tmp_file)
    
    video_from_file = load(tmp_file)
    assert video_tensor.shape == video_from_file.shape
    print(f"tensor = {video_tensor}")
    print(f"video_from_file = {video_from_file}")
    assert np.allclose(video_tensor, video_from_file)
    
    
    

    Expected behavior

    I have an mp4 file, that I want to load to video_1. When storing the video_1 array to an mp4 file, and then load the video from that new file to a second array, video_2, I expect: np.allclose(video_1, video_2)

    Actual behavior

    I am getting an Assertion error:

    tensor = [[[[ 1  2  3]
       [ 4  5  6]]
    
      [[ 7  8  9]
       [10 11 12]]
    
      [[13 14 15]
       [16 17 18]]
    
      [[19 20 21]
       [22 23 24]]]]
    video_from_file = [[[[ 3  4  7]
       [ 3  4  7]]
    
      [[ 8  9 12]
       [ 8  9 12]]
    
      [[15 16 19]
       [15 16 19]]
    
      [[20 21 24]
       [20 21 24]]]]
    Traceback (most recent call last):
      File "/Users/charlottegerhaher/Library/Application Support/JetBrains/PyCharm2022.2/scratches/scratch_26.py", line 68, in <module>
        assert np.allclose(video_tensor, video_from_file)
    AssertionError
    
    
    

    Traceback:

    Traceback (most recent call last):
      File "/Users/charlottegerhaher/Library/Application Support/JetBrains/PyCharm2022.2/scratches/scratch_26.py", line 68, in <module>
        assert np.allclose(video_tensor, video_from_file)
    AssertionError
    

    Research

    I have done the following:

    user help 
    opened by anna-charlotte 0
  • Add stubs file to PyAV

    Add stubs file to PyAV

    Overview

    There's currently no .pyi files in the PyAV installation path, which brings much difficulty to the development, as the language server couldn't resolve any symbols and provides no completion or hints.

    Desired Behavior

    Add some basic stubs to PyAV, I'm not looking forward to a detailed type hint, just function signature and some documents, I'd heard that there's some tools like stubgen has the ability to finish the task automatically, but I'm not sure that it will work with Cython files and C module.

    Additional context

    I'll also try to write the stubs by myself, but I certainly need more help

    enhancement 
    opened by RuoXian233 0
  • How do I compute pts for additional frames of a video generated from an existing one

    How do I compute pts for additional frames of a video generated from an existing one

    Overview

    • I want to traverse a video and after frame 10 - add 10 more frames made from current 10-nth frame onto which I draw an overlay and then resume and remux the rest of the frames.
    • I don't know how to compute pts for new added frames and the rest

    I am using the example provided by @jlaine at https://github.com/PyAV-Org/PyAV/issues/473 with great help to know how to compute pts.

    I've came with this formula hardly digging

    def frame_to_pts(frame, time_base, frame_rate, start_time):
        return int((frame + start_time * time_base * frame_rate) / (time_base * frame_rate))
    

    then

    import av
    import cv2
    
    
    # open input and find video stream
    input_container = av.open('input.mp4', 'r')
    input_stream = input_container.streams.get(video=0)[0]
    
    # open output and add a video stream
    output_container = av.open('output.mp4', 'w')
    output_stream = output_container.add_stream('h264', rate=input_stream.rate)
    
    frame_current = 0
    
    for frame in input_container.decode(input_stream):
        # perform edge detection
        img = frame.to_ndarray(format='bgr24')
        img = cv2.cvtColor(cv2.Canny(img, 100, 200), cv2.COLOR_GRAY2BGR)
    
        if frame_current > 10 and frame_current <= 20:
            # add 10 more frames
            for i in range(10):
                # compute new pts
                pts_computed = frame_to_pts(frame_current, frame.time_base, input_stream.average_rate, input_stream.start_time)
                # rebuild a VideoFrame, preserving timing information
                new_frame = av.VideoFrame.from_ndarray(img, format='bgr24')
                new_frame.pts = pts_computed
                new_frame.time_base = frame.time_base
                # encode and mux
                for packet in output_stream.encode(new_frame):
                    output_container.mux(packet)
                # advance one frame
                frame_current += 1
        else:
            # compute new pts
            pts_computed = frame_to_pts(frame_current, frame.time_base, input_stream.average_rate, input_stream.start_time)
            # rebuild a VideoFrame, preserving timing information
            old_frame = av.VideoFrame.from_ndarray(img, format='bgr24')
            old_frame.pts = pts_computed
            old_frame.time_base = frame.time_base
            # encode and mux
            for packet in output_stream.encode(old_frame):
                output_container.mux(packet)
                # advance one frame
            frame_current += 1
    
    
    # flush and close output
    for packet in output_stream.encode(None):
        output_container.mux(packet)
    output_container.close()
    

    Expected behavior

    The formula above seems to work for h264 codec, but I have no confidence in it -

    Actual behavior

    Switching to h264_nvenc instead of h264 gets me pts < dts errors.

    Investigation

    Tried other codecs hevc_nvenc - seems to work

    Research

    I have done the following:

    user help 
    opened by iongion 0
  • [streams] populate added stream with codec param (fixes #1044)

    [streams] populate added stream with codec param (fixes #1044)

    fixes #1044

    avformat_new_stream(AVFormatContext *s, const AVCodec *c) does not use their second parameter [1] So avcodec_parameters_from_context() should do the job [2]

    opened by HanzCEO 1
Owner
PyAV
Pythonic bindings for FFmpeg's libraries.
PyAV
A Telegram bot to convert videos into x265/x264 format via ffmpeg.

Video Encoder Bot A Telegram bot to convert videos into x265/x264 format via ffmpeg. Configuration Add values in environment variables or add them in

Adnan Ahmad 82 Jan 3, 2023
Stream music with ffmpeg and python

youtube-stream Stream music with ffmpeg and python original Usage set the KEY in stream.sh run server.py run stream.sh (You can use Git bash or WSL in

Giyoung Ryu 14 Nov 17, 2021
A tool to fuck a video/audio quality using FFmpeg

Media quality fucker A tool to fuck a video/audio quality using FFmpeg How to use Download the source Download Python Extract FFmpeg Put what you want

Maizena 8 Jan 25, 2022
A simple Telegram bot to extract hard-coded subtitle from videos using FFmpeg & Tesseract.

Video Subtitle Extractor Bot A simple Telegram bot to extract hard-coded subtitle from videos using FFmpeg & Tesseract. Note that the accuracy of reco

null 14 Oct 28, 2022
A Telegram bot to convert videos into x265/x264 format via ffmpeg.

Video Encoder Bot A Telegram bot to convert videos into x265/x264 format via ffmpeg. Configuration Add values in environment variables or add them in

null 1 Mar 8, 2022
A wrapper around ffmpeg to make it work in a concurrent and memory-buffered fashion.

Media Fixer Have you ever had a film or TV show that your TV wasn't able to play its audio? Well this program is for you. Media Fixer is a program whi

Halit Şimşek 3 May 4, 2022
A GUI based glitch tool that uses FFMPEG to create motion interpolated glitches in your videos.

FF Dissolve Glitch This is a GUI based glitch tool that uses FFmpeg to create awesome and wierd motion interpolated glitches in videos. I call it FF d

Akash Bora 19 Nov 10, 2022
Python based script to operate FFMPEG.

FMPConvert Python based script to operate FFMPEG. Ver 1.0 -- 2022.02.08 Feature ✅ Maximum compatibility: Third-party dependency libraries unused ✅ Che

cybern000b 1 Feb 28, 2022
Pythonic bindings for FFmpeg's libraries.

PyAV PyAV is a Pythonic binding for the FFmpeg libraries. We aim to provide all of the power and control of the underlying library, but manage the gri

PyAV 1.8k Jan 3, 2023
Ffmpeg videostream - High speed video frame access in Python, using FFmpeg and FFshow

FFmpeg VideoStream High speed video frame access in Python, using FFmpeg and FFshow This script requires: Karl Kroening's 'ffmpeg-python' library. (ht

null 3 Sep 29, 2022
cross-library (GStreamer + Core Audio + MAD + FFmpeg) audio decoding for Python

audioread Decode audio files using whichever backend is available. The library currently supports: Gstreamer via PyGObject. Core Audio on Mac OS X via

beetbox 419 Dec 26, 2022
cross-library (GStreamer + Core Audio + MAD + FFmpeg) audio decoding for Python

audioread Decode audio files using whichever backend is available. The library currently supports: Gstreamer via PyGObject. Core Audio on Mac OS X via

beetbox 359 Feb 15, 2021
A Telegram bot to convert videos into x265/x264 format via ffmpeg.

Video Encoder Bot A Telegram bot to convert videos into x265/x264 format via ffmpeg. Configuration Add values in environment variables or add them in

Adnan Ahmad 82 Jan 3, 2023
Stream music with ffmpeg and python

youtube-stream Stream music with ffmpeg and python original Usage set the KEY in stream.sh run server.py run stream.sh (You can use Git bash or WSL in

Giyoung Ryu 14 Nov 17, 2021
A tool to fuck a video/audio quality using FFmpeg

Media quality fucker A tool to fuck a video/audio quality using FFmpeg How to use Download the source Download Python Extract FFmpeg Put what you want

Maizena 8 Jan 25, 2022
Datamoshing with FFmpeg

ffmosher Datamoshing with FFmpeg Drag and drop video onto mosh.bat to create a datamoshed video. To datamosh an image, please ensure the file is in a

null 18 Sep 11, 2022
Run an FFmpeg command and see the percentage progress and ETA.

Run an FFmpeg command and see the percentage progress and ETA.

null 25 Dec 22, 2022
CLI Utility to encode and recursively recreate directories with ffmpeg.

FFenmass CLI Utility to encode and recursively recreate directories with ffmpeg. Report Bug · Request Feature Table of Contents Getting Started Prereq

George Av. 8 May 6, 2022
A simple Telegram bot to extract hard-coded subtitle from videos using FFmpeg & Tesseract.

Video Subtitle Extractor Bot A simple Telegram bot to extract hard-coded subtitle from videos using FFmpeg & Tesseract. Note that the accuracy of reco

null 14 Oct 28, 2022