OpenCV-Erlang/Elixir bindings

Related tags

Computer Vision opencv elixir elixir-library erlang-library
Overview

evision [WIP]

: OS : arch Build Status
Ubuntu 20.04 arm64 CI
Ubuntu 20.04 armv7 CI
Ubuntu 20.04 s390x CI
Ubuntu 20.04 ppc64le CI
Ubuntu 20.04 x86_64 CI
macOS 11 Big Sur x86_64 CI

Apple Silicon M1-series are supported, actually I coded and tested this project on my M1 Max MacBook Pro, but M1 GitHub Action runners are not yet available.

Nerves Support

Nerves

Prebuilt firmwares are available here. Select the most recent run and scroll down to the Artifacts section, download the firmware file for your board and run

fwup /path/to/the/downloaded/firmware.fw

SSH keys can be found in nerves/id_rsa[.pub]. For obvious security reason, please use these prebuilt firmwares for evaluation only.

Description

evision will pull OpenCV source code from GitHub, then parse and automatically generate corresponding OpenCV-Elixir bindings.

This project uses and modifies gen2.py and hdr_parser.py from the python module in the OpenCV repo so that they output header files that can be used in Elixir bindings.

We hope this project can largely reduce the work of manually porting OpenCV functions/modules to Elixir.

Current available modules:

  • calib3d
  • core
  • features2d
  • flann
  • highgui
  • imgcodecs
  • imgproc
  • ml
  • photo
  • stitching
  • ts
  • video
  • videoio

Note, edit config/config.exs to enable/disable OpenCV modules and image coders.

Dependencies

Required

  • Python3 (Only during the compilation, to generate binding files)
  • CMake
  • Erlang development library/headers.

Optional

  • curl/wget. To download OpenCV source zip file.

    Optional if you put the source zip file to 3rd_party/cache/opencv-${OPENCV_VER}.zip.

  • unzip. To unzip the OpenCV source zip file.

    Optional if you supply OpenCV source code at 3rd_party/opencv/opencv-${OPENCV_VER}.

Installation

In order to use evision, you will need Elixir installed. Then create an Elixir project via the mix build tool:

$ mix new my_app

Then you can add evision as dependency in your mix.exs. At the moment you will have to use a Git dependency while we work on our first release:

def deps do
  [
    {:evision, "~> 0.1.0-dev", github: "cocoa-xu/evision", branch: "main"}
  ]
end

Note

  • Use MAKE_BUILD_FLAGS="-j$(nproc)" environment variable to set number of jobs for compiling.

    Default value: "-j#{System.schedulers_online()}". In mix.exs.

  • Use TOOLCHAIN_FILE="/path/to/toolchain.cmake" to set your own toolchain.

    Default value: "nerves/toolchain.cmake". In Makefile.

  • Edit config/config.exs to enable/disable OpenCV modules and image coders.

  • Some useful commands

    MIX_ENV=dev
OPENCV_VER=4.5.4
MIX_TARGET=rpi4

# delete OpenCV related CMake build caches.
rm -rf "_build/${MIX_ENV}/lib/evision/cmake_opencv_${OPENCV_VER}"
## for nerves
rm -rf "_build/${MIX_TARGET}_${MIX_ENV}/lib/evision/cmake_opencv_${OPENCV_VER}"

# remove downloaded OpenCV source zip file.
rm -f "3rd_party/cache/opencv-${OPENCV_VER}"

# delete evision.so (so that `make` can rebuild it, useful when you manually modified C/C++ source code)
rm -f "_build/${MIX_ENV}/lib/evision/priv/evision.so"
## for nerves
rm -rf "_build/${MIX_TARGET}_${MIX_ENV}/lib/evision/priv/evision.so"

# delete evision related CMake build caches.
rm -rf "_build/${MIX_ENV}/lib/evision/cmake_evision"
## for nerves
rm -rf "_build/${MIX_TARGET}_${MIX_ENV}/lib/evision/cmake_evision"

Current Status

{:ok, gray_mat} = OpenCV.imread("/path/to/img.png", flags: OpenCV.cv_imread_grayscale)
{:ok, gray_blur_mat} = OpenCV.blur(gray_mat, [10,10], anchor: [1,1])
{:ok, colour_mat} = OpenCV.imread("/path/to/img.png")
{:ok, colour_blur_mat} = OpenCV.blur(colour_mat, [10,10], anchor: [1,1])
:ok = OpenCV.imwrite("/path/to/img-gray-and-blur.png", gray_blur_mat)
:ok = OpenCV.imwrite("/path/to/img-colour-and-blur.png", colour_blur_mat)

{:ok, cap} = OpenCV.VideoCapture.videocapture(0)
{:ok, cap_mat} = OpenCV.VideoCapture.read(cap)
:ok = OpenCV.imwrite("/path/exists/capture-mat.png", cap_mat)
:error = OpenCV.imwrite("/path/not/exists/capture-mat.png", cap_mat)

Todo

  • Update .py files in py_src so that they output header files for Erlang bindings.

  • Automatically generate erl_cv_nif.ex.

  • Automatically generate opencv_*.ex files using Python.

  • Automatically convert enum constants in C++ to "constants" in Elixir

  • When a C++ function's return value's type is bool, map true to :ok and false to :error.

    # not this
{:ok, true} = OpenCV.imwrite("/path/to/save.png", mat, [])
# but this
:ok = OpenCV.imwrite("/path/to/save.png", mat, [])

  • Make optional parameters truly optional.

    # not this
{:ok, cap} = OpenCV.VideoCapture.videocapture(0, [])
# but this
{:ok, cap} = OpenCV.VideoCapture.videocapture(0)
# this also changes the above example to
:ok = OpenCV.imwrite("/path/to/save.png", mat)

  • Nerves support (rpi4 for now).

  • Add OpenCV.Mat module.

    {:ok, type} = OpenCV.Mat.type(mat)
{:ok, {:u, 8}}

{:ok, {height, weight, channel}} = OpenCV.Mat.shape(mat)
{:ok, {1080, 1920, 3}}
{:ok, {height, weight}} = OpenCV.Mat.shape(gray_mat)
{:ok, {1080, 1920}}

{:ok, bin_data} = OpenCV.Mat.to_binary(mat)
{:ok, << ... binary data ... >>}

  • Add support for Nx.

    nx_tensor = OpenCV.Mat.to_nx(mat)
#Nx.Tensor<
   u8[1080][1920][3]
   [[ ... pixel data ... ]]
>
{:error, reason} = OpenCV.Mat.to_nx(invalid_mat)

  • Edit config/config.exs to enable/disable OpenCV modules and image coders.

  • Add tests.

How does this work?

  1. This project will first pull OpenCV source code from git (as git submodules).

  2. Inside the OpenCV project, there is an opencv-python module, 3rd_party/opencv/modules/python. If the opencv-python module is enabled,

    cmake ...
    -D PYTHON3_EXECUTABLE=$(PYTHON3_EXECUTABLE) \
    ...

    It will generate files for opencv-python bindings in cmake build dir, cmake_build_dir/modules/python_bindings_generator.

    We are interested in the headers.txt file as it tells us which headers should we parse (this header list changes based on the enabled modules).

    We also need to check another file, pyopencv_custom_headers.h. This file includes pyopencv compatible headers from modules that need special handlings to enable interactions with Python. We will talk about this later.

  3. Originally, the headers.txt file will be passed to 3rd_party/opencv/modules/python/src2/gen2.py and that Python script will then generate pyopencv_*.h in cmake_build_dir/modules/python_bindings_generator. Here we copy that Python script and modify it so that it outputs c_src/evision_*.h files which use c_src/erlcompat.hpp and c_src/nif_utils.hpp to make everything compatible with Erlang.

  4. c_src/opencv.cpp includes almost all specialised and generic evision_to and evision_from functions. They are used for making conversions between Erlang and C++. Some conversion functions are defined in module custom headers.

  5. This is where we need to make some changes to pyopencv_custom_headers.h. We first copy it to c_src/evision_custom_headers.h and copy every file it includes to c_src/evision_custom_headers/. Then we make corresponding changes to c_src/evision_custom_headers/*.hpp files so that these types can be converted from and to Erlang terms. The header include path in c_src/evision_custom_headers.h should be changed correspondingly.

  6. However, it is hard to do step 5 automatically. We can try to create a PR which puts these changed files to the original OpenCV repo's {module_name}/mics/erlang/ directory. Now we just manually save them in c_src/evision_custom_headers. Note that step 5 and 6 are done manually, calling py_src/gen2.py will not have effect on c_src/evision_custom_headers.h and *.hpp files in c_src/evision_custom_headers.

  7. Another catch is that, while function overloading is easy in C++ and optional arguments is simple in Python, they are not quite friendly to Erlang/Elixir. There is basically no function overloading in Erlang/Elixir. Although Erlang/Elixir support optional argument (default argument), it also affects the function's arity and that can be very tricky to deal with. For example,

    defmodule OpenCV.VideoCapture do
  def open(self, camera_index, opts \\ []), do: :nil
  def open(self, filename), do: :nil
  # ... other functions ...
end

    In this case, def open(self, camera_index, opts \\ []), do: :nil will define open/3 and open/2 at the same time. This will cause conflicts with def open(self, filename), do: :nil which defines open/2.

    So we cannot use default arguments. Now, say we have

    defmodule OpenCV.VideoCapture do
  def open(self, camera_index, opts), do: :nil
  def open(self, filename, opt), do: :nil
end

    Function overloading in C++ is relatively simple as compiler does that for us. In this project, we have to do this ourselves in the Erlang/Elixir way. For the example above, we can use guards.

    defmodule OpenCV.VideoCapture do
  def open(self, camera_index, opts) when is_integer(camera_index) do
    # do something
  end
  def open(self, filename, opt) when is_binary(filename) do
    # do something
  end
end

    But there are some cases we cannot distinguish the argument type in Erlang/Elixir becauase they are resources (instance of a certain C++ class).

    defmodule OpenCV.SomeModule do
  # @param mat: Mat
  def func_name(mat) do
    # do something
  end

  # @param mat: UMat
  def func_name(mat) do
    # do something
  end
end

    In such cases, we only keep one definition. The overloading will be done in c_src/opencv.cpp (by evision_to).

  8. Enum handling. Originally, PythonWrapperGenerator.add_const in py_src/gen2.py will be used to handle those enum constants. They will be saved to a map with the enum's string representation as the key, and, of course, enum's value as the value. In Python, when a user uses the enum, say cv2.COLOR_RGB2BGR, it will perform a dynamic lookup which ends up calling corresponding evision_[to|from]. evision_[to|from] will take the responsibility to convert between the enum's string representation and its value. Although in Erlang/Elixir we do have the ability to both create atoms and do the similar lookups dynamically, the problem is that, if an enum is used as one of the arguments in a C++ function, it may be written as void func(int enum) instead of void func(ENUM_TYPE_NAME enum). However, to distinguish between overloaded functions, some types (int, bool, string, char, vector) will be used for guards. For example, void func(int enum) will be translated to def func(enum) when is_integer(enum), do: :nil. Adding these guardians help us to make some differences amongst overloaded functions in step 7. However, that prevents us froming passing an atom to def func(enum) when is_integer(enum), do: :nil. Technically, we can add one more variant def func(enum) when is_atom(enum), do: :nil for this specific example, but there are tons of functions has one or more ints as their input arguments, which means the number of variants in Erlang will increase expoentially (for each int in a C++ function, it can be either a real int or an enum). Another way is just allow it to be either an integer or an atom:

    def func(enum) when is_integer(enum) or is_atom(enum) do
  :nil
end

    But in this way, atoms are created on-the-fly, users cannot get code completion feature for enums from their IDE. But, finally, we have a good news that, in Erlang/Elixir, when a function has zero arguments, you can write its name without explictly calling it, i.e.,

    defmodule M do
  def enum_name(), do: 1
end

1 = M.enum_name
1 = M.enum_name()

    So, in this project, every enum is actually transformed to a function that has zero input arguments.

How do I make it compatible with more OpenCV modules?

Because of the reason in step 6, when you enable more modules, if that module has specialised custom header for python bindings, the custom headers will be added to cmake_build_dir/modules/python_bindings_generator/pyopencv_custom_headers.h. Then you can manually copy corresponding specialised custom headers to c_src/evision_custom_headers and modify these conversion functions in them.

Acknowledgements

  • gen2.py, hdr_parser.py and c_src/erlcompat.hpp were directly copied from the python module in the OpenCV repo. Changes applied.
  • Makefile, CMakeLists.txt and c_src/nif_utils.hpp were also copied from the torchx module in the elixir-nx repo. Minor changes applied.
Comments
  • Proposal: bridging with Nx

    Proposal: bridging with Nx

    I'm developing Excv, a bridge between Nx and OpenCV: https://github.com/zeam-vm/excv

    I hope that evision will be integrated with such bridging. I'd like to contribute to evision if you wish so. Would you please inform me where the #Reference structure includingMat?

    opened by zacky1972 14
  • Share image data between Vix and eVision (and Nx)

    Share image data between Vix and eVision (and Nx)

    I am the author of the Image library that is only possible because of @akash-akya's Vix which is a binding to libvips. Given Vix and eVision have similar problem domains, and very similar matrix implementations, I would like to be able to share image data between the two libraries. This would allow Image (and of course any other library) to leverage the capabilities of both Vix and eVision (and very desirably Nx) in a very memory efficient and performant manner.

    Objectives

    1. Share image data without copying the matrix (I think a shared interpretation is possible) - but need to make sure we know who has memory ownership and who is managing the memory (and I'm not good at this part).
    2. Having a way to agree and share data formats (ie a common lexicon to refer to image data type and matrix shape - perhaps using the Nx terminology?)
    3. Act as a model of how to do the same integration with Nx. I know eVision has Nx integration but it appears to be based upon converting to and from Elixir binaries - which I'd prefer not to do since lots of NIF copying would contradict the memory efficiency and performance objectives.

    I'm looking for guidance, suggestions and ideas - and hopefully someone motivated to tackle this because its a bit beyond my competence. :-)

    Given the intent to also provide integration with Nx, any comments from @josevalim would also be very welcome.

    enhancement 
    opened by kipcole9 13
  • Import Nx tensor that is in {width, height, channels} shape with RGB data format

    Import Nx tensor that is in {width, height, channels} shape with RGB data format

    Cocoa, I have tried everything I can think of to take an Nx tensor that is of the shape {width, height, channels} that contains RGB format data, and convert it in eVision to {height, width, channels} in BGR format.

    Any chance I might ask for your advice and recommendations?

    What I've tried

    Here is what I have tried which is, I think, the required process but the saved image is definitely not what is expected!

    tensor = File.read!("path_to/color_checker.etf") |> :erlang.binary_to_term()
{:ok, mat} = Evision.Nx.to_mat(tensor)
{:ok, transposed} = Evision.Mat.transpose(mat, [1, 0, 2])
{:ok, bgr} = Evision.cvtColor(transposed, Evision.cv_COLOR_RGB2BGR())
Evision.imwrite "some_path/color_checker.jpg", bgr

    I have followed the converse process in Image and the results line up with expectations. Which is not too surprising since Image expects data in {width, height, channels} and RGB format. I mention this just to note that I have verified that the tensor does represent the underlying example image.

    Artifacts

    The image converted to a tensor and stored with :erlang.term_to_binary/1 and zipped: color_checker.etf.zip

    The original image: color_checker

    opened by kipcole9 10
  • cropping of image using ROI not working

    cropping of image using ROI not working

    Please refer to the below images to understand about the issue. This is a function used to get image: Screenshot from 2022-10-10 10-50-12 This is the error window: Screenshot from 2022-10-10 10-52-21 Its working in python for reference: Screenshot from 2022-10-10 10-52-55

    opened by navrkr 9
  • Building OpenCV is failed when describing evision in deps of mix.exs of a new mix project

    Building OpenCV is failed when describing evision in deps of mix.exs of a new mix project

    Hi,

    I did the following steps:

    mix new evision_test
cd evision_test

    Then, I added evision to mix.exs as follows:

      defp deps do
    [
      {:evision, "~> 0.1.0-dev", github: "cocoa-xu/evision", branch: "main"}
    ]
  end

    Then, I did the following steps:

    mix deps.get
mix compile

    Then, I got the following error:

    ==> evision
[/Users/zacky/evision_test/deps/evision/3rd_party/cache/opencv-4.5.5.zip]
  End-of-central-directory signature not found.  Either this file is not
  a zipfile, or it constitutes one disk of a multi-part archive.  In the
  latter case the central directory and zipfile comment will be found on
  the last disk(s) of this archive.
unzip:  cannot find zipfile directory in one of /Users/zacky/evision_test/deps/evision/3rd_party/cache/opencv-4.5.5.zip or
        /Users/zacky/evision_test/deps/evision/3rd_party/cache/opencv-4.5.5.zip.zip, and cannot find /Users/zacky/evision_test/deps/evision/3rd_party/cache/opencv-4.5.5.zip.ZIP, period.
make: *** [/Users/zacky/evision_test/deps/evision/3rd_party/opencv/opencv-4.5.5/modules/core/include/opencv2/core/utils/configuration.private.hpp] Error 9
could not compile dependency :evision, "mix compile" failed. Errors may have been logged above. You can recompile this dependency with "mix deps.compile evision", update it with "mix deps.update evision" or clean it with "mix deps.clean evision"
==> evision_test
** (Mix) Could not compile with "make" (exit status: 2).
You need to have gcc and make installed. Try running the
commands "gcc --version" and / or "make --version". If these programs
are not installed, you will be prompted to install them.

    Then, I did the following steps:

    mix deps.compile evision

    Then, I got the following fatal error:

    ==> evision
-- The CXX compiler identification is AppleClang 13.0.0.13000029
-- The C compiler identification is AppleClang 13.0.0.13000029
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
(snip)
-- General configuration for OpenCV 4.5.5 =====================================
--   Version control:               ca737c9
-- 
--   Platform:
--     Timestamp:                   2022-01-26T09:38:53Z
--     Host:                        Darwin 21.2.0 arm64
--     CMake:                       3.22.1
--     CMake generator:             Unix Makefiles
--     CMake build tool:            /usr/bin/make
--     Configuration:               RELEASE
-- 
--   CPU/HW features:
--     Baseline:                    NEON FP16
-- 
--   C/C++:
--     Built as dynamic libs?:      YES
--     C++ standard:                11
--     C++ Compiler:                /Library/Developer/CommandLineTools/usr/bin/c++  (ver 13.0.0.13000029)
--     C++ flags (Release):         -DPNG_ARM_NEON_OPT=0   -fsigned-char -W -Wall -Werror=return-type -Werror=non-virtual-dtor -Werror=address -Werror=sequence-point -Wformat -Werror=format-security -Wmissing-declarations -Wmissing-prototypes -Wstrict-prototypes -Wundef -Winit-self -Wpointer-arith -Wshadow -Wsign-promo -Wuninitialized -Wno-delete-non-virtual-dtor -Wno-unnamed-type-template-args -Wno-comment -fdiagnostics-show-option -Qunused-arguments -Wno-semicolon-before-method-body -ffunction-sections -fdata-sections    -fvisibility=hidden -fvisibility-inlines-hidden -O3 -DNDEBUG  -DNDEBUG
--     C++ flags (Debug):           -DPNG_ARM_NEON_OPT=0   -fsigned-char -W -Wall -Werror=return-type -Werror=non-virtual-dtor -Werror=address -Werror=sequence-point -Wformat -Werror=format-security -Wmissing-declarations -Wmissing-prototypes -Wstrict-prototypes -Wundef -Winit-self -Wpointer-arith -Wshadow -Wsign-promo -Wuninitialized -Wno-delete-non-virtual-dtor -Wno-unnamed-type-template-args -Wno-comment -fdiagnostics-show-option -Qunused-arguments -Wno-semicolon-before-method-body -ffunction-sections -fdata-sections    -fvisibility=hidden -fvisibility-inlines-hidden -g  -O0 -DDEBUG -D_DEBUG
--     C Compiler:                  /Library/Developer/CommandLineTools/usr/bin/cc
--     C flags (Release):           -DPNG_ARM_NEON_OPT=0   -fsigned-char -W -Wall -Werror=return-type -Werror=non-virtual-dtor -Werror=address -Werror=sequence-point -Wformat -Werror=format-security -Wmissing-declarations -Wmissing-prototypes -Wstrict-prototypes -Wundef -Winit-self -Wpointer-arith -Wshadow -Wsign-promo -Wuninitialized -Wno-delete-non-virtual-dtor -Wno-unnamed-type-template-args -Wno-comment -fdiagnostics-show-option -Qunused-arguments -Wno-semicolon-before-method-body -ffunction-sections -fdata-sections    -fvisibility=hidden -fvisibility-inlines-hidden -O3 -DNDEBUG  -DNDEBUG
--     C flags (Debug):             -DPNG_ARM_NEON_OPT=0   -fsigned-char -W -Wall -Werror=return-type -Werror=non-virtual-dtor -Werror=address -Werror=sequence-point -Wformat -Werror=format-security -Wmissing-declarations -Wmissing-prototypes -Wstrict-prototypes -Wundef -Winit-self -Wpointer-arith -Wshadow -Wsign-promo -Wuninitialized -Wno-delete-non-virtual-dtor -Wno-unnamed-type-template-args -Wno-comment -fdiagnostics-show-option -Qunused-arguments -Wno-semicolon-before-method-body -ffunction-sections -fdata-sections    -fvisibility=hidden -fvisibility-inlines-hidden -g  -O0 -DDEBUG -D_DEBUG
--     Linker flags (Release):      -Wl,-dead_strip  
--     Linker flags (Debug):        -Wl,-dead_strip  
--     ccache:                      NO
--     Precompiled headers:         NO
--     Extra dependencies:
--     3rdparty dependencies:
-- 
--   OpenCV modules:
--     To be built:                 calib3d core dnn features2d flann highgui imgcodecs imgproc ml objdetect photo python2 stitching ts video videoio
--     Disabled:                    gapi world
--     Disabled by dependency:      -
--     Unavailable:                 java python3
--     Applications:                perf_tests apps
--     Documentation:               NO
--     Non-free algorithms:         NO
-- 
--   GUI:                           COCOA
--     Cocoa:                       YES
--     VTK support:                 NO
-- 
--   Media I/O: 
--     ZLib:                        zlib (ver 1.2.11)
--     JPEG:                        build-libjpeg-turbo (ver 2.1.2-62)
--     WEBP:                        build (ver encoder: 0x020f)
--     PNG:                         libpng (ver 1.6.37)
--     TIFF:                        libtiff (ver 42 / 4.2.0)
--     JPEG 2000:                   build (ver 2.4.0)
--     OpenEXR:                     OpenEXR::OpenEXR (ver 3.1.3)
--     HDR:                         YES
--     SUNRASTER:                   YES
--     PXM:                         YES
--     PFM:                         YES
-- 
--   Video I/O:
--     DC1394:                      NO
--     FFMPEG:                      YES
--       avcodec:                   YES (58.134.100)
--       avformat:                  YES (58.76.100)
--       avutil:                    YES (56.70.100)
--       swscale:                   YES (5.9.100)
--       avresample:                YES (4.0.0)
--     GStreamer:                   NO
--     AVFoundation:                YES
-- 
--   Parallel framework:            GCD
-- 
--   Trace:                         YES (with Intel ITT)
-- 
--   Other third-party libraries:
--     Lapack:                      YES (/Library/Developer/CommandLineTools/SDKs/MacOSX12.1.sdk/System/Library/Frameworks/Accelerate.framework -lm -ldl)
--     Eigen:                       YES (ver 3.4.0)
--     Custom HAL:                  YES (carotene (ver 0.0.1))
--     Protobuf:                    build (3.19.1)
-- 
--   OpenCL:                        YES (no extra features)
--     Include path:                NO
--     Link libraries:              -framework OpenCL
-- 
--   Python 2:
--     Interpreter:                 /usr/bin/python2.7 (ver 2.7.18)
--     Libraries:                   /usr/lib/libpython2.7.dylib (ver 2.7.18)
--     numpy:                       /System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/numpy/core/include (ver 1.8.0rc1)
--     install path:                lib/python2.7/site-packages/cv2/python-2.7
-- 
--   Python (for build):            /usr/bin/python2.7
-- 
--   Java:                          
--     ant:                         NO
--     JNI:                         NO
--     Java wrappers:               NO
--     Java tests:                  NO
-- 
--   Install to:                    /Users/zacky/evision_test/_build/dev/lib/evision/priv
-- -----------------------------------------------------------------
-- 
-- Configuring done
-- Generating done
-- Build files have been written to: /Users/zacky/evision_test/_build/dev/lib/evision/cmake_opencv_4.5.5
[  0%] Building C object 3rdparty/ittnotify/CMakeFiles/ittnotify.dir/src/ittnotify/ittnotify_static.c.o
[  0%] Generate opencv4.pc
[  0%] Building C object 3rdparty/zlib/CMakeFiles/zlib.dir/adler32.c.o
[  0%] Building C object 3rdparty/openjpeg/openjp2/CMakeFiles/libopenjp2.dir/thread.c.o
(snip)
[ 72%] Building CXX object modules/dnn/CMakeFiles/opencv_dnn.dir/src/vkcom/vulkan/vk_loader.cpp.o
[ 72%] Linking CXX shared library ../../lib/libopencv_dnn.dylib
[ 72%] Built target opencv_dnn
make[1]: *** [all] Error 2
make: *** [/Users/zacky/evision_test/_build/dev/lib/evision/cmake_opencv_4.5.5/modules/python_bindings_generator/headers.txt] Error 2
could not compile dependency :evision, "mix compile" failed. Errors may have been logged above. You can recompile this dependency with "mix deps.compile evision", update it with "mix deps.update evision" or clean it with "mix deps.clean evision"
==> evision_test
** (Mix) Could not compile with "make" (exit status: 2).
You need to have gcc and make installed. Try running the
commands "gcc --version" and / or "make --version". If these programs
are not installed, you will be prompted to install them.

    My environment is as follows:

    $ uname -a 
Darwin zackym1air.local 21.2.0 Darwin Kernel Version 21.2.0: Sun Nov 28 20:29:10 PST 2021; root:xnu-8019.61.5~1/RELEASE_ARM64_T8101 arm64
$ elixir -v
Erlang/OTP 24 [erts-12.2.1] [source] [64-bit] [smp:8:8] [ds:8:8:10] [async-threads:1]

Elixir 1.13.2 (compiled with Erlang/OTP 24)

    Thank you.

    opened by zacky1972 9
  • Evision.DNN.DetectionModel can't setInputParams at YOLOv4

    Evision.DNN.DetectionModel can't setInputParams at YOLOv4

    I tried to use YOLOv4 at Evision.DNN.DetectionModel

    dog = "yolov4/dog.jpg"
weights = "yolov4/yolov4.weights"
config = "yolov4/yolov4.cfg"
classes = "yolov4/coco.names"
mat = Evision.imread(dog)

Evision.DNN.DetectionModel.detectionModel(weights, config: config)
|> Evision.DNN.DetectionModel.detect(mat)

    return error

    {:error,
 "OpenCV(4.6.0) /Users/runner/work/evision/evision/3rd_party/opencv/opencv-4.6.0/modules/dnn/src/model.cpp:98: error: (-201:Incorrect size of input array) Input size not specified in function 'processFrame'\n"}

    i guess need such as python code

    model.setInputParams(1.0, (416, 416), (0.0, 0.0, 0.0), true, false)

    but

    Evision.DNN.DetectionModel.detectionModel(weights, config: config)
|> Evision.DNN.Model.setInputParams(scale: 1.0, size: {416, 416}, mean: {0, 0, 0}, swapRB: true, crop: false)

    ** (UndefinedFunctionError) function Evision.DNN.Model.setInputParams/2 is undefined or private. Did you mean:

    Evision.DNN.DetectionModel.detectionModel(weights, config: config)
|> Evision.DNN.Model.setInputParams({416, 416})

    {:error, "cannot get `cv::dnn::Model` from `self`: mismatched type or invalid resource?"}

    How about the following solution? 1 Convert the loaded model to Evision.DNN.Model 2 Enable setInputParams in DetectionModel as well

    Evision.DNN.readFromDarknet load and detect are success but, I wan't to impl post process :-(

    Enviroments evision 0.1.16 elixir 1.14.1 elrang 25.0.1 os macOS 12.6

    opened by thehaigo 8
  • small doubt regarding contours in evision

    small doubt regarding contours in evision

    i get the values of findContours after getting is their any method to seperate contours using in built function like in python imutil.grabcontours

    the code is here

    `def grab_contours(cnts):
    # if the length the contours tuple returned by cv2.findContours
    # is 2 then we are using either OpenCV v2.4, v4-beta, or
    # v4-official
    if len(cnts) == 2:
        cnts = cnts[0]

    # if the length of the contours tuple is 3 then we are using
    # either OpenCV v3, v4-pre, or v4-alpha
    elif len(cnts) == 3:
        cnts = cnts[1]

    # otherwise OpenCV has changed their cv2.findContours return
    # signature yet again and I have no idea WTH is going on
    else:
        raise Exception(("Contours tuple must have length 2 or 3, "
            "otherwise OpenCV changed their cv2.findContours return "
            "signature yet again. Refer to OpenCV's documentation "
            "in that case"))

    # return the actual contours array
    return cnts`

    could please help us sir @cocoa-xu @josevalim

    opened by Jagan3534 8
  • An error occur when compiling on Jetson (Linux, aarch64)

    An error occur when compiling on Jetson (Linux, aarch64)

    Hi,

    I compiled evision on NVIDIA Jetson AGX Xavier (Ubuntu 18.04 Linux Kernel 4.9.201-tegra, aarch64), but the following error occurred:

    ==> evision
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100   117    0   117    0     0    393      0 --:--:-- --:--:-- --:--:--   393
100 89.8M    0 89.8M    0     0  9401k      0 --:--:--  0:00:09 --:--:-- 14.4M
CMake Error: The source directory "/mnt/nvme/home/zacky/evision/_build/dev/lib/evision/cmake_opencv_4.5.5/BUILD_OPENEXR=ON" does not exist.
Specify --help for usage, or press the help button on the CMake GUI.
Makefile:96: recipe for target '/mnt/nvme/home/zacky/evision/_build/dev/lib/evision/cmake_opencv_4.5.5/modules/python_bindings_generator/headers.txt' failed
make: *** [/mnt/nvme/home/zacky/evision/_build/dev/lib/evision/cmake_opencv_4.5.5/modules/python_bindings_generator/headers.txt] Error 1
** (Mix) Could not compile with "make" (exit status: 2).
You need to have gcc and make installed. If you are using
Ubuntu or any other Debian-based system, install the packages
"build-essential". Also install "erlang-dev" package if not
included in your Erlang/OTP version. If you're on Fedora, run
"dnf group install 'Development Tools'".

    Are Linux and aarch64 not supported?

    opened by zacky1972 8
  • imshow function is broken in pre-compiled library.

    imshow function is broken in pre-compiled library.

    I recently updated from elixir 1.0.0-dev to 1.0.9 version of Evision and everything is great till now, just tried getting the image as output and this error popped (Note: All the issue is with cvwait and imshow doesn't work without cvwait).

    IMG_20221011_070312_455

    opened by navrkr 7
  • sorry for what is did . we have a small problem in sort sir

    sorry for what is did . we have a small problem in sort sir

    I apologise if i sound rude, i did not know english that well may be this is why i sounds rude.

    here same we got the error in the elixir in python we wrote like this in elixir we tried to give but in sort 2 nd parameter is doubted sir in Evision hexdoxs we tried to see the example but in new version they dont show contours = sorted(contours, key=cv2.contourArea, reverse=True)

    could you please help us

    opened by Jagan3534 6
  • Can't round trip QRcode encode/decode

    Can't round trip QRcode encode/decode

    QRcode detection is working well (as far as I can test) but when I generate a QRcode and attempt to detect it I am not having any luck. For example:

    iex> qrcode = Evision.QRCodeEncoder.encode Evision.QRCodeEncoder.create(), "This is a string"           
%Evision.Mat{
  channels: 1,
  dims: 2,
  type: {:u, 8},
  raw_type: 0,
  shape: {25, 25},
  ref: #Reference<0.1002129465.3225026580.34308>
}
iex> Evision.QRCodeDetector.detectAndDecode Evision.QRCodeDetector.qrCodeDetector(), qrcode             
{"",
 %Evision.Mat{
   channels: 2,
   dims: 2,
   type: {:f, 32},
   raw_type: 13,
   shape: {1, 4, 2},
   ref: #Reference<0.1002129465.3225026580.34310>
 }, {:error, "empty matrix"}}
    1. The encoded image is very small, {25, 25} and I don't actually know how - even if its possible - to define parameters for the encoder to make the image larger.
    2. I can see there is a module Evision.QRCodeEncoder.Params but I'm not having any success try to define a param structure. For example:
    iex> Evision.QRCodeEncoder.Params.get_mode Evision.QRCodeEncoder.Params.qrcodeencoder_params()
zsh: segmentation fault  iex -S mix

    Thanks for your patience @cocoa-xu while I work out have to navigate the wonderful world of OpenCV/eVision - help and guidance much appreciated.

    opened by kipcole9 6
  • [Feature] opencv-contrib modules

    [Feature] opencv-contrib modules

    I plan to add support for modules in opencv_contrib gradually. Of course, this would bring an inevitable question: not everyone will use modules in opencv_contrib, so including them in the precompiled binaries will increase the footprint.

    Will it be feasible to have an :evision_contrib library in Elixir (like its python counterpart, where we have opencv-python and opencv-python-contrib)?

    Modules (part 1)

    • [x] barcode
    • [x] dnn_superres
    • [x] face
    • [x] img_hash
    • [x] plot
    • [x] quality
    • [x] rapid
    • [x] sfm
    • [x] shape
    • [x] structured_light
    • [x] text
    • [x] tracking
    • [x] xfeatures2d
    • [x] ximgproc
    • [x] xphoto

    Modules (part 2)

    • [x] aruco
    • [x] bgsegm
    • [x] bioinspired
    • [x] hfs
    • [x] line_descriptor
    • [x] mcc
    • [x] reg
    • [x] rgbd
    • [x] saliency
    • [x] stereo
    • [x] surface_matching
    • [x] wechat_qrcode

    Modules (part 3)

    • [x] cudaarithm
    • [x] cudabgsegm
    • [x] cudacodec
    • [x] cudafeatures2d
    • [x] cudafilters
    • [x] cudaimgproc
    • [x] cudalegacy
    • [x] cudaobjdetect
    • [x] cudaoptflow
    • [x] cudastereo
    • [x] cudawarping
    • [x] cudev

    todo

    • [x] Check module name and struct name
    • [ ] Check _Params classes
    • [ ] Examples
    • [ ] Tests
    enhancement 
    opened by cocoa-xu 2
  • [Feature-GUI] Implementing `Evision.Wx`

    [Feature-GUI] Implementing `Evision.Wx`

    As mentioned in #103:

    Even compiling OpenCV with highgui module, sometimes the Evision.HighGui.imshow function does not really work as expected (in evision): the window(s) it opened will not respond to any GUI events until calling Evision.HighGui.waitKey again.

    To make it as good as one would expect it to be, like how it is in C++/Python, perhaps it would require one to invest a few weeks of time for it. But I'm focusing on other aspects of this library right now, also I'm hesitating to do it because I don't know if this would worth that amount time.

    Also, as :wx has official support from Erlang, and it's well tested, and one can even build a relatively complex GUI app with :wx if they want.

    This module will try to make itself easy to use. Meanwhile, the Evision.HighGui module will stay in the source code even after Evision.Wx is completed.

    Here is a non-exhaustive list of desired/prospective functions in Evision.Wx. All of them, including the type and the number of their input arguments, are subject to changes in the future, or will not be implemented if the desired functionality is partially or wholly impossible to achieve in Erlang/Elixir with :wx.

    Functions available in OpenCV's highgui module can be found here.

    Any contributions or suggestions (either about Evision.Wx or Evision.HighGui) are more than welcomed - this also applies to the whole project. :)

    enhancement 
    opened by cocoa-xu 0
  • Evision (OpenCV's `cv::Mat`) as an Nx.Backend?

    Evision (OpenCV's `cv::Mat`) as an Nx.Backend?

    cv::Mat (and its variants) is the OpenCV's implementation of multi-dimensional array, so technically, it can be used as a backend for the Nx (numerical-elixir).

    opened by cocoa-xu 8
  • OpenCV gapi module

    OpenCV gapi module

    Based on the contents of pyopencv_gapi.hpp, it seems that the binding code will need to call an Elixir function and fetch the result.

    This perhaps could be done if we send related ERL_NIF_TERMs to a separate Erlang thread and send the result back. But it's not easy to properly design and write the whole thing.

    Maybe it's not worth the effort... I probably won't port this module anytime soon.

    This module is highly likely to be excluded from evision 0.1.0.

    opened by cocoa-xu 0
Releases(v0.1.25)
Owner
Cocoa
Ph.D student @ UofG
Cocoa
GitHub
Run tesseract with the tesserocr bindings with @OCR-D's interfaces

ocrd_tesserocr Crop, deskew, segment into regions / tables / lines / words, or recognize with tesserocr Introduction This package offers OCR-D complia

OCR-D 38 Oct 14, 2022
Python bindings for JIGSAW: a Delaunay-based unstructured mesh generator.

JIGSAW: An unstructured mesh generator JIGSAW is an unstructured mesh generator and tessellation library; designed to generate high-quality triangulat

Darren Engwirda 26 Dec 13, 2022
A simple document layout analysis using Python-OpenCV

Run the application: python main.py *Note: For first time running the application, create a folder named "output". The application is a simple documen

Roinand Aguila 109 Dec 12, 2022
Text Detection from images using OpenCV

EAST Detector for Text Detection OpenCV’s EAST(Efficient and Accurate Scene Text Detection ) text detector is a deep learning model, based on a novel

Abhishek Singh 88 Oct 20, 2022
python ocr using tesseract/ with EAST opencv detector

pytextractor python ocr using tesseract/ with EAST opencv text detector Uses the EAST opencv detector defined here with pytesseract to extract text(de

Danny Crasto 38 Dec 5, 2022
Solution for Problem 1 by team codesquad for AIDL 2020. Uses ML Kit for OCR and OpenCV for image processing

CodeSquad PS1 Solution for Problem Statement 1 for AIDL 2020 conducted by @unifynd technologies. Problem Given images of bills/invoices, the task was

Burhanuddin Udaipurwala 111 Nov 27, 2022
~1000 book pages + OpenCV + python = page regions identified as paragraphs, lines, images, captions, etc.

cosc428-structor I had an open-ended Computer Vision assignment to complete, and an out-of-copyright book that I wanted to turn into an ebook. Convent

Chad Oliver 45 Dec 6, 2022
This is a Computer vision package that makes its easy to run Image processing and AI functions. At the core it uses OpenCV and Mediapipe libraries.

CVZone This is a Computer vision package that makes its easy to run Image processing and AI functions. At the core it uses OpenCV and Mediapipe librar

CVZone 648 Dec 30, 2022
LEARN OPENCV IN 3 HOURS USING PYTHON - INCLUDING EXAMPLE PROJECTS

LEARN OPENCV IN 3 HOURS USING PYTHON - INCLUDING EXAMPLE PROJECTS

Murtaza Hassan 815 Dec 29, 2022
Bu uygulamada Python ve Opencv kullanarak bilgisayar kamerasından yüz tespiti yapıyoruz.

opencv_yuz_bulma Bu uygulamada Python ve Opencv kullanarak bilgisayar kamerasından yüz tespiti yapıyoruz. Bilgisarın kendi kamerasını kullanmak için;

Ahmet Haydar Ornek 6 Apr 16, 2022
Developed an AI-based system to control the mouse cursor using Python and OpenCV with the real-time camera.

Developed an AI-based system to control the mouse cursor using Python and OpenCV with the real-time camera. Fingertip location is mapped to RGB images to control the mouse cursor.

Ravi Sharma 71 Dec 20, 2022
Introduction to Augmented Reality (AR) with Python 3 and OpenCV 4.2.

Introduction to Augmented Reality (AR) with Python 3 and OpenCV 4.2.

fernanda rodríguez 85 Jan 2, 2023
Volume Control using OpenCV

Gesture-Volume-Control Volume Control using OpenCV Here i made volume control using Python and OpenCV in which we can control the volume of our laptop

Mudit Sinha 3 Oct 10, 2021
fishington.io bot with OpenCV and NumPy

fishington.io-bot fishington.io bot with using OpenCV and NumPy bot can continue to fishing fully automatically how to use Open cmd in fishington.io-b

Bahadır Araz 77 Jan 2, 2023
This is a project to detect gestures to zoom in or out, using the real-time distance between the index finger and the thumb. It's based on OpenCV and Mediapipe.

Pinch-zoom This is a python project based on real-time hand-gesture detection, to zoom in or out, using the distance between the index finger and the

Harshit Bhalla 6 Jul 11, 2022
Automatically resolve RidderMaster based on TensorFlow & OpenCV

AutoRiddleMaster Automatically resolve RidderMaster based on TensorFlow & OpenCV 基于 TensorFlow 和 OpenCV 实现的全自动化解御迷士小马谜题 Demo How to use Deploy the ser

神龙章轩 5 Nov 19, 2021
This repo contains a script that allows us to find range of colors in images using openCV, and then convert them into geo vectors.

Vectorizing color range This repo contains a script that allows us to find range of colors in images using openCV, and then convert them into geo vect

Development Seed 9 Jul 27, 2022
Image Detector and Convertor App created using python's Pillow, OpenCV, cvlib, numpy and streamlit packages.

Image Detector and Convertor App created using python's Pillow, OpenCV, cvlib, numpy and streamlit packages.

Siva Prakash 11 Jan 2, 2022
Virtual Zoom Gesture using OpenCV

Virtual_Zoom_Gesture I have created a virtual zoom gesture where we can Zoom in and Zoom out any image and even we can move that image anywhere on the

Mudit Sinha 2 Dec 26, 2021