Python package for Near Duplicate Video Detection (Perceptual Video Hashing) - Get a 64-bit comparable hash-value for any video.

https://user-images.githubusercontent.com/64683866/148959826-9b83c1ad-f9ef-4054-83df-d420ca318de6.mp4

Translation of the Hindi langauge written in Latin alphabet/Roman alphabet into English is: The third wave of covid-19 is here Again* I will pass the tests without studying**.

* (exams were canceled in the past year) ** (because of online tests and cheating)

Please design a vector logo for this project. The logo must contain the project name 'videohash'.

You must release the logo under the MIT License(same license as the project).

I want the output in SVG and PNG formats. The logo must be transparent(the background) and should be professional.

Examples of logos of some real projects that I like. The new logo can be of a similar design but MUST NOT be exactly copied.

• https://commons.wikimedia.org/wiki/File:OpenCV_Logo_with_text_svg_version.svg
• https://commons.wikimedia.org/wiki/File:TensorFlowLogo.svg
• https://commons.wikimedia.org/wiki/File:NumPy_logo_2020.svg

Both the PNG and SVG formats should be inside the assets directory in the pull request. This issue is not gonna get assigned before you open a pull request but the one I like the most is gonna be selected. Please open a pull request only if you are good at making vector logos. Thank you!

Describe the bug The download fails on reddit.

To Reproduce less than or equal to v2.1.7

Python 3.9.0 (default, Oct 21 2021, 15:27:22) 
[GCC 10.3.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> url1 = "https://www.reddit.com/r/IndianDankMemes/comments/rn2yxa/ha_bhai_normi_hu_mai/"
>>> from videohash import VideoHash
>>> url1 = "https://www.reddit.com/r/IndianDankMemes/comments/rn2yxa/ha_bhai_normi_hu_mai/"
>>> url2 = "https://www.reddit.com/r/IndianDankMemes/comments/rmw1o9/i_am_happy_i_am_happy_i_am_happi_today/"
>>> videohash1 = VideoHash(url=url1)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/akamhy/projects/benchmark_videohash/venv/lib/python3.9/site-packages/videohash/videohash.py", line 85, in __init__
    self._copy_video_to_video_dir()
  File "/home/akamhy/projects/benchmark_videohash/venv/lib/python3.9/site-packages/videohash/videohash.py", line 288, in _copy_video_to_video_dir
    Download(
  File "/home/akamhy/projects/benchmark_videohash/venv/lib/python3.9/site-packages/videohash/downloader.py", line 51, in __init__
    self.download_video()
  File "/home/akamhy/projects/benchmark_videohash/venv/lib/python3.9/site-packages/videohash/downloader.py", line 85, in download_video
    raise DownloadFailed(
videohash.exceptions.DownloadFailed: '/home/akamhy/projects/benchmark_videohash/venv/bin/yt-dlp' failed to download the video at 'https://www.reddit.com/r/IndianDankMemes/comments/rn2yxa/ha_bhai_normi_hu_mai/'.
[Reddit] rn2yxa: Downloading JSON metadata
[Reddit] rn2yxa: Downloading m3u8 information
[Reddit] rn2yxa: Downloading MPD manifest

ERROR: [Reddit] k4nqp99cdc781: Requested format is not available

>>> videohash1 = VideoHash(url=url1, download_worst=False)
>>> videohash2 = VideoHash(url=url2, download_worst=False)
>>> videohash1 - videohash2
4
>>> 

Expected behavior Download the video without any extra arguments.

Please complete the following information:

• Operating system: NA
• Python Version: NA
• VideoHash version: NA

Additional context I don't use Reddit but a friend of mine was using videohash to search posts by templates. Both the URLs use the same template.

• https://www.reddit.com/r/IndianDankMemes/comments/rn2yxa/ha_bhai_normi_hu_mai/
• https://www.reddit.com/r/IndianDankMemes/comments/rmw1o9/i_am_happy_i_am_happy_i_am_happi_today/

If you are good at Python please write a script that would download the latest FFmpeg from https://www.gyan.dev/ffmpeg/builds/ffmpeg-git-full.7z

Uncompress the archive.

Copy the bin directory from the decompressed folder, and paste inside C:\Program Files\ffmpeg.

Add C:\Program Files\ffmpeg\bin\ to the Environment Variables.

See https://github.com/akamhy/videohash/wiki/Install-FFmpeg,-but-how%3F#install-ffmpeg-on-windows.

I've written a script for testing on windows, you may find it useful. Link : https://github.com/akamhy/videohash/blob/main/assets/windows_ffmpeg_downloader_at_cwd.py

I am a non-native speaker and I suck at formal grammar. If you are a native speaker or just good at writing doc strings/comments/copy editing please open a pull request. Language must be formal, don't add any jokes or slang.

Thank you!

Problem: When I try to get the hash of a long video (about 90 min), it turns out errors when making tile: "encoder error -2 when writing image file" and "Maximum supported image dimension is 65500 pixels".

My solution now: Instead of just changing the frame_interval to shorten the width, I found that it might be a limit of the jpeg format. So the solution I am using is to change the default output of the function "make_tile" in "tilemaker.py" to png format: save_tiles(tiles, prefix="tile", directory=tiles_dir, file_format="png")

Suggestion: Now, there is no error. However, I am not sure if this will affect the hash value (as generated with jpeg format), or if any confilct to any part of this lib. As I notice the default parameter of "file_format" in th function "save_tiles" is already png, I am confused why the jpeg format is explictly given in the function "make_tile". If there is no other problem, maybe using png as the default in "make_tile" is better considering some long videos?

Thank you!

Describe the bug Hash collision for some videos of same length

To Reproduce

v1 = VideoHash(url="https://canvaz.scdn.co/upload/artist/3PhoLpVuITZKcymswpck5b/video/5e966e9c01f147cdae93a02c61a4bf7c.cnvs.mp4")
v2 = VideoHash(url="https://canvaz.scdn.co/upload/licensor/7JGwF0zhX9oItt9901OvB5/video/dc047df48f774d1590b61fd38bc082e4.cnvs.mp4")
print(v1 == v2)

Expected behavior The hash should be different but is same.

Screenshots NA

Please complete the following information:

• Operating system: NA
• Python Version: NA
• VideoHash version: NA

Additional context The issue can probably be resolved by extracting more features such as brightness levels or maybe the most dominant colors of frames extracted at a specific FPS. Increase the number of hash bits to accommodate more data.

OR

why not use colorhash + whash and change the bit site to 128( twice of 64, the current size)? They generate hash is very different ways and collisions should be highly unlikely.

In a windows environment, if the encoding is not specified, ASCII is expected. This creates errors when running setup.py when the README (or other files) contain emojis or other non-ascii characters. This fixes this at least for the README, not sure if there are other instances of this.

Would be great to expose access to the frame interval. I quite often work with very long or very short videos and would be great to specifiy the frame interval to check. Alternatively, be able to select a total number of frames and have it randomly select that number of frames from across the video.

When using the from_path method of hashing a video, if the path to the video contains any number of spaces, it will break the ffmpeg commands given to subprocess.Popen. This is because:

1. The paths within the command are not encapsulated by quotation marks, causing ffmpeg to interpret only the part of the path prior to the first whitespace as the target path, and the rest of the given path as additional, invalid arguments
2. The command given to subprocess.Popen is split on spaces before being interpreted, again forcing the system to interpret different parts of the path as new arguments

This is easily fixed by inserting escaped quotation marks around any paths in the ffmpeg commands and dropping the .split() on operation in the command and setting shell=True in Popen.

I've taken the liberty of including the updated functions here:

def frames(input_file, output_prefix):
    """Extract the frames of the video.
    Export frames as images at output_prefix as a 7 digit padded jpeg file.
    """
    command = "ffmpeg -i \"{input_file}\" -r 1 \"{output_prefix}_%07d.jpeg\"".format(
        input_file=input_file, output_prefix=output_prefix
    )
    process = Popen(command, shell=True, stdout=DEVNULL, stderr=STDOUT)
    output, error = process.communicate()


def compressor(input_file, task_dir, task_uid):
    # APPLY : ffmpeg -i input.webm -s 64x64 -r 30  output.mp4

    output_file = join(task_dir, task_uid + "compressed.mp4")
    command = "ffmpeg -i \"{input_file}\" -s 64x64 -r 30 \"{output_file}\"".format(
        input_file=input_file, output_file=output_file
    )
    process = Popen(command, shell=True, stdout=DEVNULL, stderr=STDOUT)
    output, error = process.communicate()

    return output_file

Hope you find this useful! Thanks for the great module!

…n attr now in VideoHash class instances and video_duration func can also be accesed directly as videohash.video_duration also added tests for the new module

This is main to address the issue described here: https://github.com/akamhy/videohash/issues/98

It's mostly an issue with subprocess and ffmpeg, but videohash is susceptible to it. Let me know if you have any questions.

I also added a bit to allow me to save the ffmpeg logs (which I needed at first to diagnose what was going on). You can keep it at your leisure, but I left it as optional.

Thanks again for this library. It's a very good idea and I'm looking forward to experimenting more with the various results!

When running videohash as part of a program that has also used subprocess it seems to inherit the stdin and that can result in various failures for ffmpeg.

I have been documenting it here: https://github.com/digitalmethodsinitiative/4cat/pull/303#issue-1422452417 Essentially, I can use videohash alone, but not with additional subprocesses unless I edit it and provide it with stdin=subprocess.DEVNULL since the default stdin is in use.

Sending a PR shortly with needed edit.

Describe the bug It takes quite a while to hash a video.

To Reproduce

from videohash import VideoHash
import time

start = time.time()

url = 'https://user-images.githubusercontent.com/47534140/185008752-da1f09c7-a177-4a46-9c64-230744e998c1.mp4'
v1 = VideoHash(url=url, frame_interval=12)

print(f"Finished in {time.time() - start} secs")

Expected behavior It should realistically be doable in under a second

Please complete the following information:

• Operating system: Windows 10
• Python Version: 3.10.2
• VideoHash version: 2.1.9

Additional context Currently takes about 3/4 seconds

Describe the bug Hash collision occurs with videos of the same length and with similar colour schemes.

To Reproduce

v1 = VideoHash(url='https://user-images.githubusercontent.com/47534140/185008752-da1f09c7-a177-4a46-9c64-230744e998c1.mp4')
v2 = VideoHash(url='https://user-images.githubusercontent.com/47534140/185008748-b8922142-37cc-48a0-bad9-1385ba016587.mov')
print (v1 == v2)

Expected behavior The hashes of the videos should be different.

Screenshots NA

Please complete the following information:

• Operating system: NA
• Python Version: 3.10.5
• VideoHash version: 3.0.1

Additional context