Kornia is a open source differentiable computer vision library for PyTorch.

Overview


English | 简体中文

WebsiteDocsTry it NowTutorialsExamplesBlogCommunity

PyPI python PyPI version Downloads License Slack Twitter

tests-cpu-versions tests-cuda codecov Documentation Status pre-commit.ci status

Kornia - Computer vision library for deep learning | Product Hunt

Kornia is a differentiable computer vision library for PyTorch.

It consists of a set of routines and differentiable modules to solve generic computer vision problems. At its core, the package uses PyTorch as its main backend both for efficiency and to take advantage of the reverse-mode auto-differentiation to define and compute the gradient of complex functions.

Overview

Inspired by existing packages, this library is composed by a subset of packages containing operators that can be inserted within neural networks to train models to perform image transformations, epipolar geometry, depth estimation, and low-level image processing such as filtering and edge detection that operate directly on tensors.

At a granular level, Kornia is a library that consists of the following components:

Component Description
kornia a Differentiable Computer Vision library, with strong GPU support
kornia.augmentation a module to perform data augmentation in the GPU
kornia.color a set of routines to perform color space conversions
kornia.contrib a compilation of user contrib and experimental operators
kornia.enhance a module to perform normalization and intensity transformation
kornia.feature a module to perform feature detection
kornia.filters a module to perform image filtering and edge detection
kornia.geometry a geometric computer vision library to perform image transformations, 3D linear algebra and conversions using different camera models
kornia.losses a stack of loss functions to solve different vision tasks
kornia.morphology a module to perform morphological operations
kornia.utils image to tensor utilities and metrics for vision problems

Installation

From pip:

pip install kornia
Other installation options

From source:

python setup.py install

From source with symbolic links:

pip install -e .

From source using pip:

pip install git+https://github.com/kornia/kornia

Examples

Run our Jupyter notebooks tutorials to learn to use the library.

Cite

If you are using kornia in your research-related documents, it is recommended that you cite the paper. See more in CITATION.

@inproceedings{eriba2019kornia,
  author    = {E. Riba, D. Mishkin, D. Ponsa, E. Rublee and G. Bradski},
  title     = {Kornia: an Open Source Differentiable Computer Vision Library for PyTorch},
  booktitle = {Winter Conference on Applications of Computer Vision},
  year      = {2020},
  url       = {https://arxiv.org/pdf/1910.02190.pdf}
}

Contributing

We appreciate all contributions. If you are planning to contribute back bug-fixes, please do so without any further discussion. If you plan to contribute new features, utility functions or extensions, please first open an issue and discuss the feature with us. Please, consider reading the CONTRIBUTING notes. The participation in this open source project is subject to Code of Conduct.

Community

  • Forums: discuss implementations, research, etc. GitHub Forums
  • GitHub Issues: bug reports, feature requests, install issues, RFCs, thoughts, etc. OPEN
  • Slack: Join our workspace to keep in touch with our core contributors and be part of our community. JOIN HERE
  • For general information, please visit our website at www.kornia.org
Issues
  • [Feature Request] Implement morphological operations like dilation

    [Feature Request] Implement morphological operations like dilation

    https://github.com/kornia/kornia/pull/292

    enhancement :rocket: feature request Difficulty: medium 
    opened by ananyamukh6 23
  • adopt torch.testing.assert_close

    adopt torch.testing.assert_close

    Closes #1029. torch.testing.assert_close will only be shipped with torch==1.9. Thus, this PR should not be merged before.

    All files except for test/utils.py are 1-to-1 ports and probably do not need special review, but I let you be the judge of that.

    enhancement :rocket: P1 🚨 code heatlh :pill: 
    opened by pmeier 23
  • Make kornia.augmentation.random_crop_generator safer

    Make kornia.augmentation.random_crop_generator safer

    🚀 Feature

    Make kornia.augmentation.random_crop_generator safer.

    Motivation

    Within kornia.augmentation.random_crop_generator _adapted_uniform is used to generate random starting points for the crop. Since it returns a torch.float, the upper delimiter (high) is increased by one (high = x_diff + 1) and cast to torch.long, which is basically a floor division. While this generates a uniform distribution of the integers, it opens up an edge-case: what if Uniform.rsample (used from _adapted_uniform) returns exactly (or within the precision) high?

    from unittest import mock
    import torch
    from kornia.augmentation.random_generator import _adapted_uniform
    
    batch_size = 1
    input_size = (None, 2)
    size = (None, 1)
    same_on_batch = True
    
    x_diff = input_size[1] - size[1]
    
    
    with mock.patch(
        "kornia.augmentation.utils.Uniform.rsample",
        new=lambda self, shape: torch.ones(shape) * self.high,
    ):
        x_start = _adapted_uniform((batch_size,), 0, x_diff + 1, same_on_batch).long()
    
    print(f"x_start {'<=' if x_start <= x_diff else '>'} x_diff")
    

    x_start should lesser or equal to x_diff, but as you can see

    x_start > x_diff
    

    Pitch

    Although this edge case is unlikely, I think we should implement this properly. Especially since the fix is trivial:

    from typing import Tuple, Union
    
    
    def _adapted_uniform_int(
        shape: Union[Tuple, torch.Size],
        low: Union[float, torch.Tensor],
        high: [float, torch.Tensor],
        same_on_batch: bool = False,
    ) -> torch.Tensor:
        return _adapted_uniform(shape, low, high + 1 - 1e-6, same_on_batch).int()
    
    
    with mock.patch(
        "kornia.augmentation.utils.Uniform.rsample",
        new=lambda self, shape: torch.ones(shape) * self.high,
    ):
        x_start = _adapted_uniform_int((batch_size,), 0, x_diff, same_on_batch).long()
    
    print(f"x_start {'<=' if x_start <= x_diff else '>'} x_diff")
    
    x_start <= x_diff
    

    _adapted_uniform_int has the same signature as _adapted_uniform. The only difference to the above call is that we subtract a small constant from high + 1 to prevent it from being drawn exactly. As long as this constant 1e-6 is smaller than torch.finfo(torch.float).eps ~= 1.19e-7 this should be fine.

    Additional Context

    I've encountered this while working with kornia.augmentation.random_crop_generator. If this used in other places throughout the code base the fix should be applicable everywhere.

    bug :bug: wontfix P3 :palm_tree: 
    opened by pmeier 22
  • Simple draw rectangle implementation for feat #607 https://github.com…

    Simple draw rectangle implementation for feat #607 https://github.com…

    …/kornia/kornia/issues/607

    Description

    As part of feat #607, this is a basic implementation of non differentiable draw rectangle.

    Status

    WIP

    Types of changes

    • [x] New tests added to cover the changes
    • [x] New Feature

    PR Checklist

    PR Implementer

    This is a small checklist for the implementation details of this PR.

    If there are any questions regarding code style or other conventions check out our summary.

    • [x] Did you discuss the functionality or any breaking changes before ?
    • [x] Pass all tests: did you test in local ? make test
    • [x] Unittests: did you add tests for your new functionality ?
    • [x] Documentations: did you build documentation ? make build-docs
    • [x] Implementation: is your code well commented and follow conventions ? make lint
    • [x] Docstrings & Typing: has your code documentation and typing ? make mypy
    • [x] Update notebooks & documentation if necessary -not necessary yet

    KorniaTeam

    KorniaTeam workflow
    • [ ] Assign correct label
    • [ ] Assign PR to a reviewer
    • [ ] Does this PR close an Issue? (add closes #IssueNumber at the bottom if not already in description)

    Reviewer

    Reviewer workflow
    • [ ] Do all tests pass? (Unittests, Typing, Linting, Documentation, Environment)
    • [ ] Does the implementation follow kornia design conventions?
    • [ ] Is the documentation complete enough ?
    • [ ] Are the tests covering simple and corner cases ?
    PR: Good to Merge :ok_hand: P1 🚨 
    opened by mmathew23 21
  • [Bug] warp_perspective does not return the input if used with identity matrix

    [Bug] warp_perspective does not return the input if used with identity matrix

    🐛 Bug

    The output of kornia.geometry.warp_perspective does not equal the input if the identity matrix is used.

    To Reproduce

    Steps to reproduce the behavior:

    import torch
    from kornia.geometry import warp_perspective
    
    torch.manual_seed(0)
    
    dsize = (32, 16)
    src = torch.rand(1, 3, *dsize)
    M = torch.eye(3).unsqueeze(0)
    
    dst = warp_perspective(src, M, dsize)
    
    mae = torch.mean(torch.abs(dst - src))
    print(mae.item())
    
    0.14952071011066437
    

    Expected behavior

    0.0
    

    Environment

    kornia==0.4.1

    bug :bug: help wanted P1 🚨 module: geometry 
    opened by pmeier 20
  • refactor homogeneous transforms module

    refactor homogeneous transforms module

    This PR tries to fulfill the feature request in #74

    Implements:

    • relative_transformation
    • inverse_transformation
    • compose_transformations.

    Moves transform_points from module conversions to transformations.

    PR: Good to Merge :ok_hand: 
    opened by edgarriba 20
  • add option for mean and std to be tuples

    add option for mean and std to be tuples

    Description

    I propose to allow the user to instantiate Normalize object with mean and std as tuples.

    Status

    Ready

    Types of changes

    • [ ] Bug fix (non-breaking change which fixes an issue)
    • [ ] Breaking change (fix or new feature that would cause existing functionality to change)
    • [x] New tests added to cover the changes
    • [ ] Docstrings/Documentation updated
    enhancement :rocket: P1 🚨 module: enhance 
    opened by JoanFM 19
  • [Bug] convert_points_from_homogeneous - NaN gradients in backward pass

    [Bug] convert_points_from_homogeneous - NaN gradients in backward pass

    I just experienced a NaN-gradient problem while doing a backward pass here: https://github.com/kornia/kornia/blob/4b0ae70f7f806c5eff5ab87f8b1f2d9ab4ff1e45/kornia/geometry/conversions.py#L99

    torch.where works absolutely fine, but if you have zero divisions you find yourself with NaN-gradients for sure 💩

    Here is a toy example:

    eps = 1e-8
    
    z_vec: torch.Tensor = torch.tensor([4., 6., 0., -3., 1e-9], requires_grad=True)
    
    scale: torch.Tensor = torch.where(
        torch.abs(z_vec) > eps,
        torch.tensor(1.) / z_vec,
        torch.ones_like(z_vec)
    )
    scale.backward(torch.ones_like(scale))
    

    And these are z_vec gradients: tensor([-0.0625, -0.0278, nan, -0.1111, -0.0000])

    For now my little hack is:

    ...
        # we check for points at infinity
        z_vec: torch.Tensor = points[..., -1:]
        if z_vec.requires_grad:
            def z_vec_backward_hook(grad: torch.Tensor) -> torch.Tensor:
                grad[grad != grad] = 0.
                return grad
            z_vec.register_hook(z_vec_backward_hook)
    ...
    

    But not sure if it's good enough.

    bug :bug: 
    opened by poxyu 19
  • Supporting arbitrary number of leading dimensions

    Supporting arbitrary number of leading dimensions

    Description

    This is the PR for #1068. These first commits include:

    1. Added perform_keep_shape_video decorator. Utilizing _to_bchw and _to_bcdhw in the decorator.
    2. Fixed some bugs for functions supposed to support any number of leading dimensions, but not due to a hardcode bug.
    3. Added support for the kornia.enhance.adjust.

    Further, before moving on, I would like to discuss the following matters

    1. How can I write tests for arbitrary dimensions? Could you suggest some sample?
    2. JIT tests break due to *args and **kwargs in the decorators (currently I skip these tests).
    3. Some users' codes may break when inputs not have shape (b, c, h, w) because some functions use _to_bchw and _to_bcdhw.

    Please let me know your comments. Thanks!

    Status

    Work in progress

    Types of changes

    • [x] Bug fix (non-breaking change which fixes an issue)
    • [x] Breaking change (fix or new feature that would cause existing functionality to change)
    • [ ] New tests added to cover the changes
    • [ ] Docstrings/Documentation updated

    PR Checklist

    PR Implementer

    This is a small checklist for the implementation details of this PR.

    If there are any questions regarding code style or other conventions check out our summary.

    • [ ] Did you discuss the functionality or any breaking changes before ?
    • [ ] Pass all tests: did you test in local ? make test
    • [ ] Unittests: did you add tests for your new functionality ?
    • [ ] Documentations: did you build documentation ? make build-docs
    • [ ] Implementation: is your code well commented and follow conventions ? make lint
    • [ ] Docstrings & Typing: has your code documentation and typing ? make mypy
    • [ ] Update notebooks & documentation if necessary

    KorniaTeam

    KorniaTeam workflow
    • [ ] Assign correct label
    • [ ] Assign PR to a reviewer
    • [ ] Does this PR close an Issue? (add closes #IssueNumber at the bottom if not already in description)

    Reviewer

    Reviewer workflow
    • [ ] Do all tests pass? (Unittests, Typing, Linting, Documentation, Environment)
    • [ ] Does the implementation follow kornia design conventions?
    • [ ] Is the documentation complete enough ?
    • [ ] Are the tests covering simple and corner cases ?
    enhancement :rocket: P1 🚨 code heatlh :pill: 
    opened by justanhduc 19
  • [QUESTION] Image segmentation mask transform

    [QUESTION] Image segmentation mask transform

    I am doing image segmentation and using random transforms. How do I get the image and mask to have the same transform done and the mask not have stuff like brightness happen to it? The tut applies then separately: https://kornia.readthedocs.io/en/latest/tutorials/data_augmentation.html#sphx-glr-tutorials-data-augmentation-py

    question :question: wontfix feature request 
    opened by bluesky314 18
  • Fixed scale parameter generation

    Fixed scale parameter generation

    Changes

    The change was motivated by issue described in #1392 The shape of scale parameters being generated earlier was incorrectly as a result of which for batch_size > 1 the error was observed. With this change I reshape the tensor first (to shape Bx1) post which the repeat converts the shape to Bx3 which is the expected shape.

    Fixes # (issue) Fixes #1392

    Type of change

    • [ ] 📚 Documentation Update
    • [ ] 🧪 Tests Cases
    • [x] 🐞 Bug fix (non-breaking change which fixes an issue)
    • [ ] 🔬 New feature (non-breaking change which adds functionality)
    • [ ] 🚨 Breaking change (fix or feature that would cause existing functionality to not work as expected)
    • [ ] 📝 This change requires a documentation update

    Checklist

    • [x] My code follows the style guidelines of this project
    • [x] I have performed a self-review of my own code
    • [ ] I have commented my code, particularly in hard-to-understand areas
    • [ ] I have made corresponding changes to the documentation
    • [x] My changes generate no new warnings
    • [ ] Did you update CHANGELOG in case of a major change?
    bug :bug: module: augmentations 
    opened by anirudhb11 2
  • :stars: deprecate 1.6/1.7 and add 1.9.1

    :stars: deprecate 1.6/1.7 and add 1.9.1

    Changes

    • deprecates PyTorch 1.6/1.7 support and includes 1.9.1
    • deprecation is based on pytorch downloading versions from : https://pepy.tech/project/torch?versions=1.8.1&versions=1.9.0&versions=1.9.1&versions=1.6.0&versions=1.7.0&versions=1.7.1&versions=1.8.0
    • Checks also that cuda tests are :ok:

    Type of change

    • [ ] 📚 Documentation Update
    • [ ] 🧪 Tests Cases
    • [ ] 🐞 Bug fix (non-breaking change which fixes an issue)
    • [ ] 🔬 New feature (non-breaking change which adds functionality)
    • [ ] 🚨 Breaking change (fix or feature that would cause existing functionality to not work as expected)
    • [ ] 📝 This change requires a documentation update

    Checklist

    • [ ] My code follows the style guidelines of this project
    • [ ] I have performed a self-review of my own code
    • [ ] I have commented my code, particularly in hard-to-understand areas
    • [ ] I have made corresponding changes to the documentation
    • [ ] My changes generate no new warnings
    • [ ] Did you update CHANGELOG in case of a major change?
    enhancement :rocket: code heatlh :pill: 
    opened by edgarriba 0
  • Clarify keypoints behaviour and their limitations

    Clarify keypoints behaviour and their limitations

    Describe the bug

    TL; TR: document kornia convention for keypoints and their current limitations or fix their calculation. This affects to bbox_v2 PR (#1304).

    Although kornia uses floats for keypoints, it assumes that they come from integers pointing out the pixel coordinate (index based convention). So, they are really meaning pixel center -> i+0.5 ( point (3,4) refers to the center of the pixel (3,4) that it's (3.5, 4.5)). As they are floats, users expect that keypoints represent exact coordinates, for example, (0.04, 6.4). Below, you can see an example.

    Why this happens?

    1. Pixels in images are index based. So their matrix transforms assumes it.
    2. Augmentation containers shares the same matrix transform between images, keypoints and bboxes.
    3. Keypoints under exact coordinates uses an slightly different formulas. See here and here
    4. Bboxes using "width = xmax - xmin" convention assumes that underlying vertices (boxes keypoints) are expressed as exact coordinates while "width = xmax - xmin +1" convention as index based.

    This leads to:

    1. Kornia can only handle keypoints coming from integers.
    2. Keypoints using exact coordinates values after transforms are inaccurate. The real value may be 1 pixel off. I don't think that zoom affects (not 100% sure here).
      • This affects model accuracy.
      • It could lead to subtle bugs when manually inverting a transform made by kornia. When detectron fixed a similar error, boxes AP increased between 0.4 - 0.7, see here.
    3. Boxes/Bouding boxes computation must be made with "width = xmax - xmin +1" convention as the offset to exact coordinates is compensated.
      • This affects to Bbox_v2 PR. Internal boxes representation must be changed from "width = xmax - xmin" to "width = xmax - xmin +1" convention.

    So, a decision should be made:

    1. Refactor kornia to be accurate with keypoints by using exact coordinates.
    2. Leave as it + document in augmentation containers + update bbox_v2 internal format.
    3. Leave as it + update bbox_v2 internal format. Keypoints in exact coordinates (float) will be slightly inaccurate.

    Reproduction steps

    Note: keypoints valid range depends on their convention.

    • Pixel indexes (integers): [0, W-1 or H-1]. For example (3, 5).
    • Exact coordinates (must be floats): [0, W or H].

    Here, you can see a graphical example

    from kornia import augmentation as K
    import torch
    
    def to_xyxy(bbox):
        return torch.stack([bbox.amin(dim=-2), bbox.amax(dim=-2)], dim=-2).view(
                bbox.shape[0], bbox.shape[1], 4
            )
    
    img = torch.eye(4).expand(1,3, -1, -1) # torch.rand(1, 3, 4, 4)
    h, w = img.shape[-2:]
    pts = torch.tensor([[0, 0],
                        [w-1, h-1], # Max value for pixel indexes convention
                        [w, h]])  #  Max value for exact coordinate convention
    bbox = torch.tensor([[
        [[0,h//4],[w-1,0],[w-1,h//2],[0, h//2]],  # convention width = xmax - xmin + 1
        [[0,h//4],[w,0],[w,h//2],[0, h//2]], # convention width = xmax - xmin
    ]])
    
    print('Original values')
    print(img[0,0,...], pts, bbox, to_xyxy(bbox))
    
    (tensor([[1., 0., 0., 0.],
             [0., 1., 0., 0.],
             [0., 0., 1., 0.],
             [0., 0., 0., 1.]]),
     tensor([[0, 0],
             [3, 3],
             [4, 4]]),
     tensor([[[[0, 1],
               [3, 0],
               [3, 2],
               [0, 2]],
     
              [[0, 1],
               [4, 0],
               [4, 2],
               [0, 2]]]]),
     tensor([[[0, 0, 3, 2],
              [0, 0, 4, 2]]]))
    

    Here, what happens for horizontal flips:

    tfm= K.AugmentationSequential(
        K.RandomHorizontalFlip(p=1.),
        data_keys=["input", "bbox", "keypoints"],
        return_transform=True,
        same_on_batch=False
    )
    
    (img_out, M), pts_out, bbox_out = tfm(img, pts.float(), bbox.float())
    print(img_out[0, 0, ...], pts_out, bbox_out, to_xyxy(bbox_out))
    
    (tensor([[0., 0., 0., 1.],
             [0., 0., 1., 0.],
             [0., 1., 0., 0.],
             [1., 0., 0., 0.]]),
     tensor([[ 3.,  0.],
             [ 0.,  3.],
             [-1.,  4.]]),
     tensor([[[[ 3.,  1.],
               [ 0.,  0.],
               [ 0.,  2.],
               [ 3.,  2.]],
     
              [[ 3.,  1.],
               [-1.,  0.],
               [-1.,  2.],
               [ 3.,  2.]]]]),
     tensor([[[ 0.,  0.,  3.,  2.],
              [-1.,  0.,  3.,  2.]]]))
    

    As you can see, keypoints and bboxes are only accurate is under pixel indexes convention (integers). Points at top-left (origin) are flipped correctly assuming pixel index convention. Points at top-right under pixel index convention are flipped correctly to the origin (0,0) but not those under exact coordinates as one becomes negative (-1, 4). Finally, you can see that only boxes under width=... +1 convention are right.

    The same happens with random rotations:

    tfm1= K.AugmentationSequential(
        K.RandomAffine([180,180], p=1.),
        data_keys=["input", "bbox", "keypoints"],
        return_transform=True,
        same_on_batch=False
    )
    
    (img_out, M1), pts_out, bbox_out = tfm1(img, pts.float(), bbox.float())
    print(img_out[0, 0, ...].round(), pts_out.round(), bbox_out.round(), to_xyxy(bbox_out))
    
    (tensor([[1., 0., 0., 0.],
             [0., 1., 0., 0.],
             [0., 0., 1., 0.],
             [0., 0., 0., 1.]]),
     tensor([[ 3.,  3.],
             [ 0., -0.],
             [-1., -1.]]),
     tensor([[[[ 3.,  2.],
               [ 0.,  3.],
               [ 0.,  1.],
               [ 3.,  1.]],
     
              [[ 3.,  2.],
               [-1.,  3.],
               [-1.,  1.],
               [ 3.,  1.]]]]),
     tensor([[[ 0.0000,  1.0000,  3.0000,  3.0000],
              [-1.0000,  1.0000,  3.0000,  3.0000]]]))
    
    help wanted 
    opened by hal-314 1
  • Gradients from Boxes3D.to_tensor aren't accurate

    Gradients from Boxes3D.to_tensor aren't accurate

    Describe the bug

    Landing issue for those wanting to compute gradients in for Boxes3D as discussed in #1304.

    Gradients from Boxes3D.to_tensor aren't accurate as analytical gradient doesn't match with numerical gradient. At the time when Boxes and Boxes3D were implemented (PR #1304), we didn't know how to fix them. Feel free to fix them :) .

    Exactly, the error happens in this line of kornia.geometry.bbox_v2.

        boxes = torch.stack([batched_boxes.amin(dim=-2), batched_boxes.amax(dim=-2)], dim=-2).view(
                batched_boxes.shape[0], batched_boxes.shape[1], 6
            )
    

    Here, computed and analytical gradients diverge. This only happens for Boxes3D and not Boxes. See discussion in PR #1304 when Boxes and Boxes3D were implemented for more information.

    Reproduction steps

    Below there is a code to reproduce the error:

    Working code for 2D boxes

    import torch
    from torch.autograd import gradcheck
    
    def tensor_to_gradcheck_var(tensor, dtype=torch.float64, requires_grad=True):
        """Convert the input tensor to a valid variable to check the gradient.
    
        `gradcheck` needs 64-bit floating point and requires gradient.
        """
        if not torch.is_tensor(tensor):
            raise AssertionError(type(tensor))
        return tensor.requires_grad_(requires_grad).type(dtype)
    
    t_boxes2D = torch.tensor([[[1.0, 1.0], [3.0, 1.0], [3.0, 2.0], [1.0, 2.0]]])
    t_boxes2D = tensor_to_gradcheck_var(t_boxes2D)
    
    def to_tensor(batched_boxes):
        batched_boxes = batched_boxes if batched_boxes.ndim == 4 else batched_boxes.unsqueeze(0)
        
        boxes = torch.stack([batched_boxes.amin(dim=-2), batched_boxes.amax(dim=-2)], dim=-2).view(
                batched_boxes.shape[0], batched_boxes.shape[1], 4
            )
        return boxes
    
    gradcheck(to_tensor, t_boxes2D)  # Ok when boxes are 2D
    

    The same code for 3D boxes fails :/

    t_boxes3d = torch.tensor([[[0, 1, 2], [10, 1, 2], [10, 21, 2], [0, 21, 2], [0, 1, 32], [10, 1, 32], [10, 21, 32], [0, 21, 32]]], dtype=float)
    t_boxes3d = tensor_to_gradcheck_var(t_boxes3d)
    
    def to_tensor3d(batched_boxes):
        batched_boxes = batched_boxes if batched_boxes.ndim == 4 else batched_boxes.unsqueeze(0)
        boxes = torch.stack([batched_boxes.amin(dim=-2), batched_boxes.amax(dim=-2)], dim=-2).view(
                batched_boxes.shape[0], batched_boxes.shape[1], 6
            )
        return boxes
    
    gradcheck(to_tensor3d, t_boxes3d)  # Fails :/ 
    

    Expected behavior

    To work and enable gradient checks for Boxes3D.to_tensor ;)

    Environment

    Tested in Pytorch 1.7.1, 1.8.1, 1.9.0 and 1.9.1, both in linux (ubuntu 20.04) and windows
    
    help wanted 
    opened by hal-314 0
  • [feat] Added histogram matching

    [feat] Added histogram matching

    null

    feature request module: enhance 
    opened by shijianjian 1
  • RandomAffine3D throws RuntimeError when batch_size > 1

    RandomAffine3D throws RuntimeError when batch_size > 1

    Describe the bug

    When the batch_size of input > 1, kornia.augmentation.RandomAffine3D throws RuntimeError: "The size of tensor a (3) must match the size of tensor b (6) at non-singleton dimension 2".

    Reproduction steps

    import torch
    import kornia as K
    
    rng = torch.manual_seed(0)
    input = torch.rand(2, 1, 3, 3, 3)
    aug = K.augmentation.RandomAffine3D((15., 20., 20.), p=1., return_transform=True)
    aug(input)
    

    Expected behavior

    We should expect the valid results.

    Environment

    wget https://raw.githubusercontent.com/pytorch/pytorch/master/torch/utils/collect_env.py
    # For security purposes, please check the contents of collect_env.py before running it.
    python collect_env.py
    
    • PyTorch Version (e.g., 1.0): 1.7.0
    • OS (e.g., Linux): Linux
    • How you installed PyTorch (conda, pip, source): pip
    • Build command you used (if compiling from source):
    • Python version: 3.7.3
    • CUDA/cuDNN version: 10.1
    • GPU models and configuration:
    • Any other relevant information:
    
    
    ### Additional context
    
    _No response_
    help wanted 
    opened by hnyGuo 1
  • :dancer: Homography-Tracker-rebase

    :dancer: Homography-Tracker-rebase

    Changes

    Clean version of https://github.com/kornia/kornia/pull/1378 Adds 3 things:

    • DescriptorMatcher - simple class, which is module version of our match_*nn functions
    • LocalFeatureMatcher - wrapper around local feature detector and descriptor to extract features and match them in the same API, as LoFTR does
    • HomographyTracker - module to track a planar object in the sequence of the frames in form of homography. Now is based on the local feature matching and RANSAC, but in principle can accept anything, which takes 2 images and outputs you sparse or dense matches.

    The docs and tests are missing now, let's discuss and maybe change API?

    You can check the example of usage for AR application here https://colab.research.google.com/drive/1wO0TX0iJSpQsqfYT8g_d7J0pYyMwhuNe?usp=sharing

    image

    image

    Type of change

    • [ ] 📚 Documentation Update
    • [ ] 🧪 Tests Cases
    • [ ] 🐞 Bug fix (non-breaking change which fixes an issue)
    • [x] 🔬 New feature (non-breaking change which adds functionality)
    • [ ] 🚨 Breaking change (fix or feature that would cause existing functionality to not work as expected)
    • [ ] 📝 This change requires a documentation update

    Checklist

    • [ ] My code follows the style guidelines of this project
    • [ ] I have performed a self-review of my own code
    • [ ] I have commented my code, particularly in hard-to-understand areas
    • [ ] I have made corresponding changes to the documentation
    • [ ] My changes generate no new warnings
    • [ ] Did you update CHANGELOG in case of a major change?
    feature request P1 🚨 module: tracking 
    opened by ducha-aiki 0
  • A new algorithm for view synthesis based on Neural Radiance Field (NeRF)

    A new algorithm for view synthesis based on Neural Radiance Field (NeRF)

    🚀 Feature

    An implementation of the algorithm in Wang et al. (2021) - https://arxiv.org/abs/2102.07064 - for Neural Radiance Field view synthesis without known camera parameters

    Motivation

    We would like the have a simple API where you pass a sequence of images (batch, list of frames, etc.) and you get back the optimized camera intrinsic so that can be used jointly with undistort_image (by @jhacsonmeza). For the nature of Nerf-- seems that the relative poses (or respect to world) could be also retrieved.

    Pitch

    Create the class CameraCalibrator to implement Algorithm 1 in Wang et al. (2021)

    Alternatives

    Additional context

    Preliminary usage design

    from kornia.contrib import CameraCalibrator
    
    calib = CameraCalibrator(...)
    calib.add_image(load_image(...))
    ....  # BxCxHxW
    
    res = calib.run()  # perform nerf training
    res['K']  # intrinsics Bx3x3
    res['P']  # projection Bx3x4
    

    Consider also to contribute to Kornia universe projects :)

    • Tutorials: our repository containing the tutorials.
    help wanted feature request module: geometry 
    opened by YanivHollander 3
  • Implement classical object tracking algorithms like Lucas-Kande and KCF

    Implement classical object tracking algorithms like Lucas-Kande and KCF

    🚀 Feature

    Since we have started our tracking module, it would be nice to populate it with classical and CNN-based things

    References:

    Lukas-Kanade Tracker KCF

    help wanted feature request 
    opened by ducha-aiki 0
  • Find homography Jacobian return sometimes nan values

    Find homography Jacobian return sometimes nan values

    Describe the bug

    While using the kornia.geometry.find_homography_dlt function, in some scenarios the Jacobian computed by the autograd return nan values.

    Reproduction steps

    import torch
    import kornia as K
    
    """Code example to verify the DLT Jacobian for the homographies.
    
    |x2|    |h1 h2 h3|   |x1|
    |y2| =  |h4 h5 h6| @ |y1|
    |1 |    |h7 h8 h9|   |1 |
    
    DLT => x' = H @ x = 0
    
    """
    
    
    def f(x):
        x1, x2 = x.unbind(-1)
        # TODO: this returns jacobian with nans
        return K.geometry.find_homography_dlt(x1, x2)
        # NOTE: this works as expected
        # return K.geometry.get_perspective_transform(x1, x2)
    
    
    # define some points in the corners with a scale factor of 100
    x1 = torch.tensor([[[0., 0.], [0., 1.], [1., 0.], [1., 1.]]])
    x2 = torch.tensor([[[0., 0.], [0., 100.], [100., 0.], [100., 100.]]])
    
    x = torch.stack((x1, x2), dim=-1)
    # NOTE: this guy does not produce nan values
    # x = torch.rand_like(x)
    
    H = f(x)  # compute the homography between the set of points
    print(H)
    print(H.shape)
    
    J = torch.autograd.functional.jacobian(f, x)
    print(J)
    print(J.shape)
    

    Expected behavior

    We should expect the jacobian J to always contain valid values.

    Environment

    PyTorch version: 1.6.0
    Is debug build: False
    CUDA used to build PyTorch: 10.2
    ROCM used to build PyTorch: N/A
    
    OS: Ubuntu 20.04.2 LTS (x86_64)
    GCC version: (Ubuntu 9.3.0-17ubuntu1~20.04) 9.3.0
    Clang version: Could not collect
    CMake version: version 3.20.5
    Libc version: glibc-2.17
    
    Python version: 3.7.11 (default, Jul 27 2021, 14:32:16)  [GCC 7.5.0] (64-bit runtime)
    Python platform: Linux-5.11.0-37-generic-x86_64-with-debian-bullseye-sid
    Is CUDA available: True
    CUDA runtime version: Could not collect
    GPU models and configuration: GPU 0: NVIDIA GeForce GTX 1650 with Max-Q Design
    Nvidia driver version: 470.63.01
    cuDNN version: Could not collect
    HIP runtime version: N/A
    MIOpen runtime version: N/A
    
    Versions of relevant libraries:
    [pip3] mypy==0.910
    [pip3] mypy-extensions==0.4.3
    [pip3] numpy==1.21.2
    [pip3] pytest-mypy==0.8.1
    [pip3] torch==1.9.0
    [pip3] torchmetrics==0.5.1
    [pip3] torchvision==0.7.0
    [conda] blas                      1.0                         mkl  
    [conda] cudatoolkit               10.2.89              hfd86e86_1  
    [conda] ffmpeg                    4.3                  hf484d3e_0    pytorch
    [conda] mkl                       2021.3.0           h06a4308_520  
    [conda] mkl-service               2.4.0            py37h7f8727e_0  
    [conda] mkl_fft                   1.3.0            py37h42c9631_2  
    [conda] mkl_random                1.2.2            py37h51133e4_0  
    [conda] mypy                      0.910                    pypi_0    pypi
    [conda] mypy-extensions           0.4.3                    pypi_0    pypi
    [conda] numpy                     1.21.2                   pypi_0    pypi
    [conda] numpy-base                1.20.3           py37h74d4b33_0  
    [conda] pytest-mypy               0.8.1                    pypi_0    pypi
    [conda] pytorch                   1.6.0           py3.7_cuda10.2.89_cudnn7.6.5_0    pytorch
    [conda] torch                     1.9.0                    pypi_0    pypi
    [conda] torchmetrics              0.5.1                    pypi_0    pypi
    [conda] torchvision               0.7.0                py37_cu102    pytorch
    

    Additional context

    Seems only to happen when zeros come to the game for some points.

    bug :bug: help wanted module: geometry 
    opened by edgarriba 0
Releases(0.5.11)
  • 0.5.11(Sep 19, 2021)

    :rocket: Release Note (0.5.11)

    Release time: 2021-09-19

    :new: New Features

    • Add Vision Transformer (ViT) (#1296)
    • Add ImageRegistrator API (#1253)
    • Add LoFTR inference (#1218)
    • Added differentiable Hausdorff Distance (HD) loss (#1254)
    • Add PadTo to kornia.augmentation (#1286)

    :zap: Code refactor

    • Return all learned modules by default in eval() mode (#1266)
    • Enable ImageSequential and VideoSequential to AugmentationSequential (#1231)
    • Specify that angles are in radians (#1287)
    • Removed deprecated codes for v6.0 (#1281)

    :lady_beetle: Bug fixes

    • Fix save_pointcloud_ply fn counting point with inf coordinates (#1263)
    • Fixes torch version parse and add temporal packaging dependency (#1284)
    • Fix issue of image_histogram2d (#1295)

    :woman_technologist: :man_technologist: We would like to thank all contributors for this new release ! @Abdelrhman-Hosny @ducha-aiki @edgarriba @EStorm21 @lyhyl @shijianjian @thatbrguy

    If we forgot someone let us know :sunglasses:

    Source code(tar.gz)
    Source code(zip)
    kornia-0.5.11-py2.py3-none-any.whl(328.17 KB)
    kornia-0.5.11.tar.gz(254.24 KB)
  • 0.5.10(Aug 30, 2021)

    [0.5.10] - 2021-08-30

    Added

    • Added Basic pool request for DeFMO: Deblurring and Shape Recovery of Fast Moving Objects. (#1135)
    • Added homography error metrics, and improved find_homography_iter (#1222)

    Fixed

    • Fixed wrong param name (#1197)
    • Fixed NotImplementedError for the rtvec (#1215))
    • Fixes warnings and add compatibility stub in torch solve (#1235)

    Changed

    • Ensure CenterCrop indices are integers (#1208)
    • Added tests, fixed docstrings and made some other changes (#1211)
    • Upgrade to modern Python syntax (#1213)
    • Code health improvements [#1199, #1200, #1198, #1202, #1203, #1205, #1208, #1210, #1214, #1220]
    • Enable pyupgrade as pre-commit (#1221)
    • Add bandit tool in the pre-commit (#1228)

    Contributors

    @cclauss @ducha-aiki @edgarriba @MichaelMonashev @rozumden @shijianjian @thatbrguy

    If we forgot someone let us know :sunglasses:

    Source code(tar.gz)
    Source code(zip)
    kornia-0.5.10-py2.py3-none-any.whl(302.51 KB)
    kornia-0.5.10.tar.gz(237.74 KB)
  • 0.5.8(Aug 6, 2021)

    Kornia release

    [0.5.8] - 2021-08-06

    Added

    • Add the connected components labeling algorithm (#1184)

    Fixed

    • Partial fix for horizontal and vertical flips (#1166)
    • Fix even kernel and add test (#1183)
    • Fix wrong source points for RandomThinPlateSpline (#1187)
    • Fix RandomElasticTransform ignores same_on_batch (#1189)
    • Fixed bugs in patchsequential. Remove fill_diagonal operation for better ONNX support (#1178)

    Changed

    • Differentiable image histogram using kernel density estimation (#1172)

    Contributors

    @bkntr @bsuleymanov @ducha-aiki @edgarriba @hal-314 @kingsj0405 @shijianjian

    If we forgot someone let us know :sunglasses:

    Source code(tar.gz)
    Source code(zip)
    kornia-0.5.8-py2.py3-none-any.whl(296.83 KB)
    kornia-0.5.8.tar.gz(233.13 KB)
  • 0.5.7(Jul 26, 2021)

  • 0.5.6(Jul 12, 2021)

    Kornia 0.5.6 release

    [0.5.6] - 2021-07-12

    Added

    • Added mix augmentations in containers (#1139)

    Fixed

    • Fixed non-4-dim input error for sequential (#1146)

    Changed

    • Moving bbox-related functionality to bbox module (#1103)
    • Optimized version of hls_to_rgb and rgb_to_hls (#1154)

    Removed

    • Remove numpy dependency (#1136)

    Contributors

    @dkoguciuk @edgarriba @lferraz @shijianjian

    If we forgot someone let us know :sunglasses:

    Source code(tar.gz)
    Source code(zip)
    kornia-0.5.6-py2.py3-none-any.whl(294.10 KB)
    kornia-0.5.6.tar.gz(230.52 KB)
  • 0.5.5(Jun 27, 2021)

    Kornia 0.5.5 release

    [0.5.5] - 2021-06-27

    Added

    Changed

    • Change GaussianBlur to RandomGaussianBlur (#1118)
    • Update ci with pytorch 1.9.0 (#1120)
    • Changed option for mean and std to be tuples in normalization (#987)
    • Adopt torch.testing.assert_close (#1031)

    Removed

    • Remove numpy import (#1116)

    Contributors

    @copaah @ducha-aiki @edgarriba @eugene87222 @JoanFM @justanhduc @pmeier @shijianjian

    If we forgot someone let us know :sunglasses:

    Source code(tar.gz)
    Source code(zip)
    kornia-0.5.5-py2.py3-none-any.whl(285.53 KB)
    kornia-0.5.5.tar.gz(223.74 KB)
  • 0.5.4(Jun 11, 2021)

    Kornia 0.5.4 release

    [0.5.4] - 2021-06-11

    Added

    • Add Canny edge detection (#1020)
    • Added Batched forward function (#1058)
    • Added denormalize homography function (#1061)
    • Added more augmentations containers (#1014)
    • Added calibration module and Undistort 2D points function (#1026)
    • Added patch augmentation container (#1095)

    Fixed

    Changed

    • Resize regardless of number of dims, considering the last two dims as image (#1047)
    • Raise error if converting to unit8 image to gray with float weights (#1057)
    • Filter 2D->2d, 3D->3d (#1069)
    • Removed augmentation functional module. (#1067)
    • Make Morphology compatible with both OpenCV and Scipy (#1084)

    Contributors

    @asottile @Borda @ducha-aiki @edgarriba @jhacsonmeza @justanhduc @Manza12 @priba @shijianjian

    Special thanks to @Borda @carmocca @asottile for the help to improve the code health of the package.

    If we forgot someone let us know :sunglasses:

    Source code(tar.gz)
    Source code(zip)
    kornia-0.5.4-py2.py3-none-any.whl(278.97 KB)
    kornia-0.5.4.tar.gz(218.61 KB)
  • 0.5.3(May 30, 2021)

    Kornia 0.5.3 release

    [0.5.3] - 2021-05-29

    Added

    • Added inverse for augmentations (#1013)
    • Add advanced augmentations: RandomFisheye, RandomElasticTransform, RandomThinPlateSpline, RandomBloxBlur (#1015)

    Fixed

    • Correct Sobel test_noncontiguous. Nothing was tested before. (#1018)
    • Fixing #795: find_homography_dlt_iterated sometimes fails (#1022)

    Changed

    • Refactorization of the morphology package (#1034)
    • Optimised clipping in clahe and some other minor optimisation (#1035)

    Contributors

    @Borda @dkoguciuk @edgarriba @Manza12 @lferraz @priba @shijianjian

    If we forgot someone let us know :sunglasses:

    Source code(tar.gz)
    Source code(zip)
    kornia-0.5.3-py2.py3-none-any.whl(274.51 KB)
    kornia-0.5.3.tar.gz(215.01 KB)
  • 0.5.2(May 14, 2021)

    Kornia 0.5.2 release

    [0.5.2] - 2021-05-14

    Added

    • Added unsharp mask filtering (#1004)

    Fixed

    • Fixed angle axis to quaternion order bug (#926)
    • Fixed type error for lab_to_rgb conversion when using coremltools. (#1002)

    Changed

    • Mask with unbatched motion from essential choose solution (#998)

    thanks to all your contributions @amonszpart @AnimeshMaheshwari22 @askaradeniz @edgarriba @jatentaki

    The Kornia Team :nerd_face:

    Source code(tar.gz)
    Source code(zip)
  • 0.5.1(Apr 30, 2021)

    Kornia 0.5.1 release

    Highlights

    In this patch release we include the following features

    • Fast version of RandomCropResize, RandomCrop
    • Add antialias support in kornia.geometry.resize
    • Added HardNet8 deep features
    • Experimental support for torch.float16
    • Added the following modules for augmentations:
      • ImageToTensor
      • RandomInvert
      • RandomChannelShuffle
      • RandomGaussianNoise

    [0.5.1] - 2021-04-30

    Added

    • Added dtype for create_mesh (#919)
    • Added Hardnet8 (#955)
    • Added normalize boolean for remap (#921)
    • Added custom weights option for rgb2gray (#944)
    • Added fp16 support (#963)
    • Added ImageToTensor module and resize for non-batched images (#978)
    • Add more augmentations (#960)
    • Anti alias resize (#989)

    Changed

    • Improve kornia porphology (#965)
    • Improve cuda ci workflow speed (#975)
    • Refactor augmentation module (#948)
    • Implement fast version of crop function in augmentations (#967)
    • Implement missing jit ops in kornia.geometry.transform (#981)

    Fixed

    • Fixed RandomAffine translation range check (#917
    • Fixed the issue of NaN gradients by adding epsilon in focal loss (#924)
    • Allow crop size greater than input size. (#957)
    • Fixed RandomCrop bug (#951)

    Removed

    • Deprecate some augmentation functionals (#943)
    Source code(tar.gz)
    Source code(zip)
  • v0.5.0(Mar 17, 2021)

    Kornia 0.5.0 release

    In this release we have focus in bringing more classic Computer Vision functionalities to the PyTorch ecosystem, like morphological operators and more diversity with Deep Local Descriptors, color conversions and drawing functions. In addition, we have worked towards improving the integration with TPU and better support with Torchscript.

    Highlights

    Morphological Operators

    As a highlight we include a kornia.morphology that implements several functionalities to work with morphological operators on high-dimensional tensors and differentiability. Contributed by @Juclique

    Morphology implements the following methods: dilation, erosion, open, close, close, gradient, top_hat and black_hat.

    from kornia import morphology as morph
    
    dilated_image = morph.dilation(tensor, kernel) # Dilation
    plot_morph_image(dilated_image) # Plot
    

    image

    See a full tutorial here: https://github.com/kornia/tutorials/blob/master/source/morphology_101.ipynb

    Deep Descriptors

    We have added a set of local feature-related models: MKDDescriptor #841 by implemented and ported to kornia by @manyids2; also we ported TFeat, AffNet, OriNet from authors repos #846.

    Here is notebook, showing the usage and benefits of new features. We also show how to seamlessly integrate kornia and opencv code via new conversion library kornia_moons.

    Also: exposed set_laf_orientation function #869

    image

    Video Augmentations

    We include a new operator to perform augmentations with videos VideoSequential. The module is based in nn.Sequential and has the ability to concatenate our existing kornia.augmentations for multi-dimensional video tensors. Contributed by @shijianjian

    import kornia
    import torchvision
    
    clip, _, _ = torchvision.io.read_video("drop.avi")
    clip = clip.permute(3, 0, 1, 2)[None] / 255.  # To BCTHW
    input = torch.randn(2, 3, 1, 5, 6).repeat(1, 1, 4, 1, 1)
    
    aug_list = VideoSequential(
        kornia.augmentation.ColorJitter(0.1, 0.1, 0.1, 0.1, p=1.0),
        kornia.augmentation.RandomAffine(360, p=1.0),
        data_format="BTCHW",
        same_on_frame=False)
    )
    
    out = aug(input)
    

    image

    See a full example in the following Colab: https://colab.research.google.com/drive/12dmHNkvEQrG-PHElbCXT9FgCr_aAGQSI?usp=sharing

    Draw functions

    We include an experimental functionality draw rectangle implemented in pure torch.tensor. Contributed by @mmathew23

    rects = torch.tensor([[[110., 50., 310., 275.], [325., 100., 435., 275.]]])
    color = torch.tensor([255., 0., 0.])
    
    x_out = K.utils.draw_rectangle(x_rgb, rects, color)
    

    image

    See full example here: https://colab.research.google.com/drive/1me_DxgMvsHIheLh-Pao7rmrsafKO5Lg3?usp=sharing

    More user contrib

    Infrastructure

    • Update CI to pytorch 1.7.x and 1.8.0 @edgarriba
    • Improve testing matrix with different versions
    • TPU support @edgarriba @shijianjian
    • Better JIT support @edgarriba @shijianjian @ducha-aiki
    • Improved and test docs @shijianjian @edgarriba

    Deprecations

    • Deprecated kornia.geometry.warp module.
      • DepthWarper is now in kornia.geometry.depth
      • HomographyWarper and related functions are now inside kornia.geometry.transform.
    • Deprecated kornia.contrib module.
      • max_pool_blurd2d is now in kornia.filters
    • Dropped support of Pytorch 1.5.1 #854

    Warp and Crop

    We refactored the interface of the functions warp_perspective, warp_affine, center_crop, crop_and_resize and crop_by_boxes in order to expose to the user the needed parameters by grid_sample [mode, padding_mode, align_corners]. #896

    The param align_corners has been set by default to None that maps to True in case the user does not specify. This comes from the motivation to match the behavior of the warping functions with OpenCV.

    Example of warp_perspective:

    def warp_perspective(src: torch.Tensor, M: torch.Tensor, dsize: Tuple[int, int],
                         mode: str = 'bilinear', padding_mode: str = 'zeros',
                         align_corners: Optional[bool] = None) -> torch.Tensor:
    

    Please review the full release notes here: https://github.com/kornia/kornia/blob/master/CHANGELOG.md

    Thanks to all our contributors !!! :tada: :sunglasses:

    Source code(tar.gz)
    Source code(zip)
  • v0.4.1(Oct 20, 2020)

    Kornia 0.4.1 release

    Highlights

    We include new features for 3D augmentations:

    • RandomCrop3D
    • CenterCrop3D
    • RandomMotionBlur3D
    • RandomEqualize3D

    Few more core functionalities to work on 3D volumetric tensors:

    • warp_affine3d
    • warp_perspective3d
    • get_perspective_transform3d
    • crop_by_boxes3d
    • motion_blur3d
    • equalize3d
    • warp_grid3d

    Details changes

    Added

    • Update docs for get_affine_matrix2d and get_affine_matrix3d (#618)
    • Added docs for solarize, posterize, sharpness, equalize (#623)
    • Added tensor device conversion for solarize params (#624)
    • Added rescale functional and transformation (#631)
    • Added Mixup data augmentation (#609)
    • Added equalize3d (#639)
    • Added decompose 3x4projection matrix (#650)
    • Added normalize_min_max functionality (#684)
    • Added random equalize3d (#653)
    • Added 3D motion blur (#713)
    • Added 3D volumetric crop implementation (#689)
      • warp_affine3d
      • warp_perspective3d
      • get_perspective_transform3d
      • crop_by_boxes3d
      • warp_grid3d

    Changed

    • Replace convolution with unfold in contrib.extract_tensor_patches (#626)
    • Updates Affine scale with non-isotropic values (#646)
    • Enabled param p for each augmentation (#664)
    • Enabled RandomResizedCrop batch mode when same_on_batch=False (#683)
    • Increase speed of transform_points (#687)
    • Improves find_homography_dlt performance improvement and weights params made optional (#690)
    • Enable variable side resizing in kornia.resize (#628)
    • Added Affine transformation as nn.Module (#630)
    • Accelerate augmentations (#708)

    Fixed

    • Fixed error in normal_transform_pixel3d (#621)
    • Fixed pipelining multiple augmentations return wrong transformation matrix (#645)(645)
    • Fixed flipping returns wrong transformation matrices (#648)
    • Fixed 3d augmentations return wrong transformation matrix (#665)
    • Fix the SOSNet loading bug (#668)
    • Fix/random perspective returns wrong transformation matrix (#667)
    • Fixes Zca inverse transform (#695)
    • Fixes Affine scale bug (#714)

    Removed

    • Removed warp_projective (#689)

    Contributors

    @gaurav104 @shijianjian @mshalvagal @pmeier @ducha-aiki @qxcv @FGeri @vribeiro1 @ChetanPatil28 @alopezgit @jatentaki @dkoguciuk @ceroytres @ag14774

    Source code(tar.gz)
    Source code(zip)
  • v0.4.0(Aug 6, 2020)

    Kornia 0.4.0 release

    In this release we are including the following main features:

    • Support to PyTorch v1.6.0.
    • Local descriptors matching, homography and epipolar geometry API.
    • 3D augmentations and low level API to work with volumetric data.

    kornia_medical

    Highlights

    Local features matching

    We include an kornia.feature.matching API to perform local descriptors matching such classical and derived version of the nearest neighbour (NN).

    import torch
    import kornia as K
    
    desc1 = torch.rand(2500, 128)
    desc2 = torch.rand(2500, 128)
    
    dists, idxs = K.feature.matching.match_nn(desc1, desc2)  # 2500 / 2500x2
    

    Homography and epipolar geometry

    We also introduce kornia.geometry.homography including different functionalities to work with homographies and differentiable estimators based on the DLT formulation and the iteratively-reweighted least squares (IRWLS).

    
    import torch
    import kornia as K
    
    pts1 = torch.rand(1, 8, 2)
    pts2 = torch.rand(1, 8, 2)
    H = K.find_homography_dlt(pts1, pts2, weights=torch.rand(1, 8))  # 1x3x3
    

    In addition, we have ported some of the existing algorithms from opencv.sfm to PyTorch under kornia.geometry.epipolar that includes different functionalities to work with Fundamental, Essential or Projection matrices, and Triangulation methods useful for Structure from Motion problems.

    3D augmentations and volumetric

    We expand the kornia.augmentaion with a series of operators to perform 3D augmentations for volumetric data BxCxDxHxW. In this release, we include the following first set of geometric 3D augmentations methods:

    • RandomDepthicalFlip3D (along depth axis)
    • RandomVerticalFlip3D (along height axis)
    • RandomHorizontalFlip3D (along width axis)
    • RandomRotation3D
    • RandomAffine3D

    The API for 3D augmentation work same as with 2D image augmentations:

    import torch
    import kornia as K
    
    x = torch.eye(3).repeat(3, 1, 1)
    aug = K.augmentation.RandomVerticalFlip3D(p=1.0)
    
    print(aug(x))
    
    tensor([[[[[0., 0., 1.],
                [0., 1., 0.],
                [1., 0., 0.]],
    <BLANKLINE>
                [[0., 0., 1.],
                [0., 1., 0.],
                [1., 0., 0.]],
    <BLANKLINE>
                [[0., 0., 1.],
                [0., 1., 0.],
                [1., 0., 0.]]]]])
    

    Finally, we introduce also a low level API to perform 4D features transformations kornia.warp_projective and extending the filtering operators to support 3D kernels kornia.filter3D.

    More 2d operators

    We expand as well the list of the 2D image augmentations based on the paper AutoAugment: Learning Augmentation Policies from Data.

    • Solarize
    • Posterize
    • Sharpness
    • Equalize
    • RandomSolarize
    • RandomPosterize
    • RandomShaprness
    • RandomEqualize

    Improvements

    • add zca whitening (#458)
    • add epipolar geometry package (#569)
    • Jit warp perspective (#574)
    • Autoaugment functions. (#571)
    • Dog and fix features (#591)
    • implement filter3D (#575)
    • Implement warp_projective (#587)
    • Feature matching and H/F/E estimation for SFM (#552)
    • 3D augmentations (#592)

    Breaking changes

    • Create kornia.enhance submodule (#614) -> see details in here

    Bugs/Fixes

    • fixed affine 2d shearing matrix translations (#612)
    • Now SIFTdesc throws and exception when the input parameters are incompatible (#598)
    • back to group conv backend for filter2d (#600)
    • updates sosnet git paths (#606)

    Docs

    • Updated doc & example for augmentation (#583)
    • fix Tversky equation (#579)
    • clean docs warnings (#604)
    • add kornia.geometry.homography docs (#608)
    • create kornia.geometry.subpix (#610)

    Dev

    • improve conftest fixtures and remove device, dtype imports (#568)
    • pin versions for pytest plugins and fix flake8 issues (#580)
    • made kornia versions explicit to pytorch version (#597)
    Source code(tar.gz)
    Source code(zip)
  • v0.3.2(Aug 6, 2020)

  • v0.3.1(May 10, 2020)

    Kornia 0.3.1 release

    This release mainly introduces the following items:

    • Add support to Python 3.8

    • Exposes and fixes issues around align_corners.

    • Improve testing infrastructure adding parametrize for different devices and dtype and flake8/mypy support throw pytest by caching intermediate results. Test usage example:

      pytest -v --device cpu,cuda --dtype float16,float32,float64 --flake8 --mypy

    Improvements

    • Update to python 3.8 (#550)
    • Improve testing framework (#560)
    • Local feature fixes and nms improvements (#545)
    • Random motion blur improvments (#562)

    Fixes

    • Expose align_corners everywhere, where interpolation occurs (#546)
    • Soft-argmax test fixes, renaming and enables jit (#553)

    Bugs

    • Fix tests in TestSpatialSoftArgmax2d (#544)

    Docs

    • Updated docstring for augmentation module (#554)
    Source code(tar.gz)
    Source code(zip)
  • v0.3.0(Apr 27, 2020)

    Kornia 0.3.0 release

    Today we released 0.3.0 which aligns with PyTorch releases cycle and includes:

    • Full support to PyTorch v1.5.
    • Semi-automated GPU tests coverage.
    • Documentation has been reorganized [docs]
    • Data augmentation API compatible with torchvision v0.6.0.
    • Well integration with ecosystem e.g. Pytorch-Lightning.

    For more detailed changes check out v0.2.1 and v0.2.2.

    Highlights

    Data Augmentation

    We provide kornia.augmentation a high-level framework that implements kornia-core functionalities and is fully compatible with torchvision supporting batched mode, multi device cpu, gpu, and xla/tpu (comming), auto differentiable and able to retrieve (and chain) applied geometric transforms. To check how to reproduce torchvision in kornia refer to this Colab: Kornia vs. Torchvision @shijianjian

    import kornia as K
    import torchvision as T
    
    # kornia
    
    transform_fcn = torch.nn.Sequential(
      K.augmentation.RandomAffine(
        [-45., 45.], [0., 0.5], [0.5, 1.5], [0., 0.5], return_transform=True),
      K.color.Normalize(0.1307, 0.3081),
    )
    
    # torchvision
    
    transform_fcn = T.transforms.Compose([
      T.transforms.RandomAffine(
        [-45., 45.], [0., 0.5], [0.5, 1.5], [0., 0.5]),
      T.transforms.ToTensor(),
      T.transforms.Normalize((0.1307,), (0.3081,)),
    ])
    

    Ecosystem compatibility

    Kornia has been designed to be very flexible in order to be integrated in other existing frameworks. See the example below about how easy you can define a custom data augmentation pipeline to later be integrated into any training framework such as Pytorch-Lighting. We provide examples in [here] and [here].

    class DataAugmentatonPipeline(nn.Module):
        """Module to perform data augmentation using Kornia on torch tensors."""
        def __init__(self, apply_color_jitter: bool = False) -> None:
            super().__init__()
            self._apply_color_jitter = apply_color_jitter
    
            self._max_val: float = 1024.
    
            self.transforms = nn.Sequential(
                K.augmentation.Normalize(0., self._max_val),
                K.augmentation.RandomHorizontalFlip(p=0.5)
            )
    
            self.jitter = K.augmentation.ColorJitter(0.5, 0.5, 0.5, 0.5)
    
        @torch.no_grad()  # disable gradients for effiency
        def forward(self, x: torch.Tensor) -> torch.Tensor:
            x_out = self.transforms(x)
            if self._apply_color_jitter:
                x_out = self.jitter(x_out)
            return x_out
    

    GPU tests

    Now easy to run GPU tests with pytest --typetest cuda

    Source code(tar.gz)
    Source code(zip)
  • v0.2.2(Apr 26, 2020)

    Kornia 0.2.2 Release Notes

    This release is a checkpoint with minimum data augmentation API stability plus fixing some GPU tests before kornia upgrades to PyTorch v.1.5.0.

    • API changes
    • Improvements
    • Bug Fixes
    • Documentation

    API changes

    • Decoupled return_transform from apply_* function (#534)

    Improvements

    • improve setup packaging and build manywheel script (#543)

    Bug Fixes

    • fix broken gpu tests (#538)
    • update sosnet urls (#541)

    Documentation

    • reorganises color docs and adds ycbcr (#540)
    • reorganise documenation in subsections (#542)
    Source code(tar.gz)
    Source code(zip)
  • v0.2.1(Apr 21, 2020)

    Kornia 0.2.1 Release Notes

    • Highlights
    • API changes
    • New Features
    • Improvements
    • Bug Fixes
    • Performance

    Highlights

    In this release we support compatibility between kornia.augmentation and torchvision.transforms.

    We now support all the same existing operations with torch.Tensor in the GPU with extra features such as returning for each operator the transformation matrix generated to produce such transformation.

    import kornia as K
    import torchvision as T
    
    # kornia
    
    transform_fcn = torch.nn.Sequential(
      K.augmentation.RandomAffine(
        [-45., 45.], [0., 0.5], [0.5, 1.5], [0., 0.5], return_transform=True),
      K.color.Normalize(0.1307, 0.3081),
    )
    
    # torchvision
    
    transform_fcn = T.transforms.Compose([
      T.transforms.RandomAffine(
        [-45., 45.], [0., 0.5], [0.5, 1.5], [0., 0.5]),
      T.transforms.ToTensor(),
      T.transforms.Normalize((0.1307,), (0.3081,)),
    ])
    

    Check the online documentations with the updated API [DOCS]

    Check this Google Colab to see how to reproduce same results [Colab]

    kornia.augmentation as a framework

    In addition, we have re-designed kornia.augmentation such in a way that users can easily contribute with more operators, or just use it as a framework to create their custom operators.

    Each of the kornia.augmentation modules inherit from AugmentationBase and one can easily define a new operator by creating a subclass and overriding a couple of methods.

    Let's take a look at a custom MyRandomRotation . The class inherits from AugmentationBase making it a nn.Module so that can be stacked in a nn.Sequential to compute chained transformations.

    To implement a new functionality two things needed: override get_params and apply

    The get_params receives the shape of the input tensor and returns a dictionary with the parameters to use in the apply function.

    The applyfunction receives as input a tensor and the dictionary defined in get_params; and returns a tuple with the transformed input and the transformation applied to it.

    class MyRandomRotation(AugmentationBase):
        def __init__(self, angle: float, return_transform: bool = True) -> None:
          super(MyRandomRotation, self).__init__(self.apply, return_transform)
          self.angle = angle
    
        def get_params(self, batch_shape: torch.Size) -> Dict[str, torch.Tensor]:
          angles_rad torch.Tensor = torch.rand(batch_shape) * K.pi
          angles_deg = kornia.rad2deg(angles_rad) * self.angle
          return dict(angles=angles_deg)
    
        def apply(self, input: torch.Tensor, params: Dict[str, torch.Tensor]):
          # compute transformation
          angles: torch.Tensor = params['angles'].type_as(input)
          center = torch.tensor([[W / 2, H / 2]]).type_as(input)
          transform = K.get_rotation_matrix2d(
            center, angles, torch.ones_like(angles))
    
          # apply transformation
          output = K.warp_affine(input, transform, (H, W))
    
          return (output, transform)
    
    # how to use it
    
    # load an image and cast to tensor
    img1: torch.Tensor = imread(...)  # BxDxHxW
    
    # instantiate and apply the transform
    aug = MyRandomRotation(45., return_transformation=True)
    
    img2, transform = aug(img1)  # BxDxHxW - Bx3x3
    

    New Features

    kornia.color

    • Implement RGB to XYZ (#436)
    • Implement RGB to LUV (#442)
    • Implement histogramd2 (#530)

    kornia.feature

    • Implement hardnet descriptor (#498)
    • Implement deep descriptor sosnet (#521)

    kornia.jit

    • Create kornia.jit module and exposes rgb_to_grayscale (#261)

    API Changes

    • Remove PIL dependency (#512)
    • Remove float casting in image_to_tensor (#497)

    Improvements

    • Adds gradcheck for RandomCrop and RandomResizedCrop (#439)
    • Update spatial_soft_argmax.py (#496)
    • Add epsilon value to make hessian matrix robust (#504)
    • Add normalize_points flag in depth to 3d (#511)
    • Functional augmentation performance test against Torchvision (#482)
    • AffineTransformation alignment and other fixes (#514)

    Performance

    • Filter speed up conditional (#433)
      • Improves by far the time performance for filtering.
    • Speed-up warp_affine and fix bugs in RandomAffine (#474)
    • Improve homography warper (#528)

    Docs

    • Make link work for PSNRLoss (#449)
    • Change psnr to psnr_loss in docs (#450)
    • Fix import problem and fix docs for LuvToRgb and PSNR (#447)
    • Fix outdated example (#465)
    • Update color_adjust.py (#479)
    • Missing commas in bibtex (#500)

    Bug fixes

    • Fix device problem in test (#456)
    • Bug fixed in device tests (#475)
    • Add epsilon value to sobel to improve backprop stability (#513)
    Source code(tar.gz)
    Source code(zip)
  • v0.2.0(Jan 27, 2020)

    Kornia 0.2.0 Release Notes

    • Highlights
    • New Features
      • kornia.color
      • kornia.feature
      • kornia.geometry
      • kornia.losses
    • Improvements
    • Bug Fixes

    Kornia v0.2.0 release is now available.

    The release contains over 50 commits and updates support to PyTorch 1.4. This is the result of a huge effort in the desing of the new data augmentation module, improvements in the set of the color space conversion algorithms and a refactor of the testing framework that allows to test the library using the cuda backend.

    Highlights

    Data Augmentation API

    From this point forward, we will give support to the new data augmentation API. The kornia.augmentation module mimics the best of the existing data augmentation frameworks such torchvision or albumentations all re-implemented assuming as input torch.Tensor data structures that will allowing to run the standard transformations (geometric and color) in batch mode in the GPU and backprop through it.

    In addition, a very interesting feature we are very proud to include, is the ability to return the transformation matrix for each of the transform which will make easier to concatenate and optimize the transforms process.

    A quick overview of its usage:

    
    import torch
    import kornia
    
    input: torch.Tensor = load_tensor_data(....)  # BxCxHxW
    
    transforms = torch.nn.Sequential(
        kornia.augmentation.RandomGrayscale(),
        kornia.augmentation.RandomAffine(degrees=(-15, 15)),
    )
    
    out: torch.Tensor = transforms(input)         # CPU
    out: torch.Tensor = transforms(input.cuda())  # GPU
    
    # same returning the transformation matrix
    
    transforms = torch.nn.Sequential(
        kornia.augmentation.RandomGrayscale(return_transformation=True),
        kornia.augmentation.RandomAffine(degrees=(-15, 15), return_transformation=True),
    )
    
    out, transform = transforms(input) # BxCxHxW , Bx3x3
    

    This are the following features found we introduce in the module:

    • BaseAugmentation (#407)
    • ColorJitter (#329)
    • RandomHorizontalFlip (#309)
    • MotionBlur (#328)
    • RandomVerticalFlip (#375)
    • RandomErasing (#344)
    • RandomGrayscale (#384)
    • Resize (#394)
    • CenterCrop (#409)
    • RandomAffine (#403)
    • RandomPerspective (#403)
    • RandomRotation (#397, #418)
    • RandomCrop (#408)
    • RandomResizedCrop (#408)
    • Grayscale

    GPU Test

    We have refactored our testing framework and we can now easily integrate GPU tests within our library. At this moment, this features is only available to run locally but very soon we will integrate with CircleCI and AWS infrastructure so that we can automate the process.

    From root one just have to run: make test-gpu

    Tests look like this:

    import torch
    from test.common import device
    
    def test_rgb_to_grayscale(self, device):
            channels, height, width = 3, 4, 5
            img = torch.ones(channels, height, width).to(device)
            assert kornia.rgb_to_grayscale(img).shape == (1, height, width)
    

    Ref PR:

    • parametrize test functions to accept torch.device cpu/cuda @edgarriba @ducha-aiki da793cd 0fcb85e

    New Features

    kornia.color

    We have added few more algorithms for color space conversion:

    • rgb_to_hsv (#299)
    • rgb_to_hls (#342)
    • rgb_to_ycbcr (#345)
    • ycbcr_to_rgb (#345)
    • rgb_to_yuv (#337)
    • yuv_to_rgb (#337)
    • rgb_to_rgba (#401)
    • rgba_to_rgb (#401)
    • bgr_to_bgra (#401)
    • bgra_to_bgr (#401)
    • bgr_to_gray (#266)
    • add_weighted (#295)

    kornia.geometry

    • Implement kornia.hflip, kornia.vflip and kornia.rot180 (#268)
    • Implement kornia.transform_boxes (#368)

    kornia.losses

    • Implements to total_variation loss (#250)
    • Implement PSNR loss (#272)

    kornia.feature

    • Added convenience functions for work with LAF: get keypoint, orientation (#340)

    Improvements

    • Fixed conv_argmax2d/3d behaviour for even-size kernel and added test (#227)
    • Normalize accepts floats and allows broadcast over channel dimension (#236)
    • Single value support for normalize function (#301)
    • Added boundary check function to local features detector (#254)
    • Correct crop_and_resize on aspect ratio changes. (#305)
    • Correct adjust brightness and contrast (#304)
    • Add tensor support to Hue, Saturation and Gamma (#324)
    • Double image option for scale pyramid (#351)
    • Filter2d speedup for older GPUs (#356)
    • Fix meshgrid3d function (#357)
    • Added support for even-sized filters in filter2d (#374)
    • Use latest version of CircleCI (#373)
    • Infer border and padding mode to homography warper (#379)
    • Apply normalization trick to conv_softmax (#383)
    • Better nms (#371)
      • added spatial gradient 3d
      • added hardnms3d and tests for hardnms 2d
      • quadratic nms interp
      • update the tests because of changed gaussian blur kernel size in scale pyramid calculation
      • no grad for spatial grad
    • Focal loss flat (#393)
    • Add optional mask parameter in scale space (#389)
    • Update to PyTorch 1.4 (#402)

    Bug fixes

    • Add from homogeneous zero grad test and fix it (#369)
    • Filter2d failed with noncontiguous input (view --> reshape) (#377)
    • Add ceil_mode to maxblur pool to be able to be used in resnets (#395)

    Breaking Changes

    • crop_and_resize before: "The tensor must have the shape of Bx4x2, where each box is defined in the following order: top-left, top-right, bottom-left and bottom-right. The coordinates order must be in y, x respectively" after: "The tensor must have the shape of Bx4x2, where each box is defined in the following (clockwise) order: top-left, top-right, bottom-right and bottom-left. The coordinates must be in the x, y order."

    As usual, thanks to the community to keep this project growing. Happy coding ! :sunrise_over_mountains:

    Source code(tar.gz)
    Source code(zip)
  • v0.1.4(Oct 5, 2019)

    Table of Contents

    We have just released Kornia: a differentiable computer vision library for PyTorch.

    It consists of a set of routines and differentiable modules to solve generic computer vision problems. At its core, the package uses PyTorch as its main backend both for efficiency and to take advantage of the reverse-mode auto-differentiation to define and compute the gradient of complex functions.

    Inspired by OpenCV, this library is composed by a subset of packages containing operators that can be inserted within neural networks to train models to perform image transformations, epipolar geometry, depth estimation, and low level image processing such as filtering and edge detection that operate directly on tensors.

    It has over 300 commits and majorly refactors the whole library including over than 100 functions to solve generic Computer Vision problems.

    Highlights

    Version 0.1.4 includes a reorganization of the internal API grouping functionalities that consists of the following components:

    • kornia | a Differentiable Computer Vision library like OpenCV, with strong GPU support.
    • kornia.color | a set of routines to perform color space conversions.
    • kornia.contrib | a compilation of user contrib and experimental operators.
    • kornia.feature | a module to perform local feature detection.
    • kornia.filters | a module to perform image filtering and edge detection.
    • kornia.geometry | a geometric computer vision library to perform image transformations, 3D linear algebra and conversions using differen camera models.
    • kornia.losses | a stack of loss functions to solve different vision tasks.
    • kornia.utils | image to tensor utilities and metrics for vision problems.

    Big contribution in kornia.features:

    • Implemented anti-aliased local patch extraction.
    • Implemented classical local features cornerness functions: Harris, Hessian, Good Features To Track.
    • Implemented basic functions for work with local affine features and their patches.
    • Implemented convolutional soft argmax 2d and 3d operators for differentable non-maxima suppression.
    • implemented second moment matrix affine shape estimation, dominant gradient orientation and SIFT patch descriptor.

    Infrastructure

    • Migration to CircleCI towards GPU testing 30099b4b0481f4151b893f77d7bf297ee47d268b
    • Added Code of Conduct file 5e0848a2d41780d632632afb81e0371e9dca6a33
    • Redefined the CONTRIBUTING notes
    • Enforce python minimal Python 3.6 usage 34e21e5f81a0376fc8bb45da52003f20a101d591
    • Creation of an external repo to host the website www.kornia.org.

    Breaking Changes

    • Removed nms and normalization from Harris response function 2209807ce8db8fe82eabbe6ca51e6370beea2934
    • Renames GaussianBlur -> GaussianBlur2d and added an input to specify pad b0c522e60ef4c82a3d1881dd5901a25d7a4a02c5
    • Chaneged batch support for tensor2img and img2tensor 705a82f1ca308087b7d62d8ce452cde1aeeaabae
    • Fixed torch.clamp for homogeneous division 506b0c98ed245373544732dd49fc4612d7075501

    New Features

    • Several functionalities for Local Affine Frame (LAF): extract_patches_from_pyramid, extract_patches_simple, normalize_laf, ellipse_to_laf, make_upright, scale_laf, get_laf_scale 0a3cbb02850ac78059e0615da93144b5a64d3330
    • Differentiable SIFT descriptor 7f0eb809f1509c452d85000fd002b12c22e358ca
    • Added implementation of the differentiable spatial to numerical (DSNT) layer and related operations. abb4afabe1a37082e8938cfe7f227e57042d9803
    • Spatial gradient 1d 362adfc1af06e0abd945e84bf00c5b8a437f3aa3
    • Scale pyramid for local features detection 413051eb4c1b36fe3548a65cee9ab2d8ba45086f
    • Added geometry.depth submodule including: depth_to_3d, depth_to_normals, warp_frame_depth d1dedb8d37f99b752467ed4acaf4f767afbbad49
    • Implement Gaussian Pyramid bc586cb4bf8454d33fed721bc6f045767191374e
    • Implement Filter2D to apply arbitrary depthwise 2d kernels 94b56f2d43ed87a259aca3e6313d0f7a1222baf5
    • Implement to save/load pointclouds 4f32351c0dfd2d0d1779e9eb1d0028e3d3b904ab
    • implement project_points 636f4f5338e4fc1b6d32140c6f1febae3b64eb96
    • Implement unproject_points b02f403feaf1fdeb574fb87e1d70157ec0b4dbff
    • Implement denormalize_coordinates b07ec45410f45469bad2067ce03b83dddcabb7c0
    • Implement harris_corner detector 977a1f6a8c7beef9c339fdd695032dec2705c7d3
    • Implement non_maxima_suppression_2d 84cc1287fcd9df2a437a2d25a61f171097047a76
    • Implement median_filter 6b6cf0543028dcf3bfb25a0ae9104e6ade26037e
    • Implement blur_filter d4c8df933570fa95546e84517a6d676e302e6e7d
    • Implement sobel_filter operator 9abe4c5afdbe486baadf07b427ad5468d57da603
    • Implement max_blur_pool_2d 621be3b59055f000896c45fe33a28fa3ca680841
    • Implement pyrup and pyrdown a4e110cd47dd6c7792751fb7294d068b7655486a
    • Implemen crop_and_resize 41b4fed573c37c7310e4d7e03b73a54bce1eb2ab
    • Implement center_crop b1188d50f7ecae001832e05e606ca55d0d630ae6
    • Implement inverse_affine_matrix 6e10fb9a0859ef35f82b6e2dfd58af828bda7a8c
    • Implement opencv like remap function b0401deac4b54e201095705ec8c18eabe943cd2b
    • Implement affine ceb3faf3b89596ba23bdc7e0f616b218edf997df
    • Implement shear 81c5a2798f00663ee64ff74db87340daa6edb08d
    • Implement scale 75a84a373e9ce142fb4a1ac0d7fde8f3790b861c
    • Implement translate 11af4dde591258e057d1973bb00529f49fa6d63f
    • Implement rotate 89c6d964c5a18254adf73a5f8da00d8a5068e7bc
    • Implement Laplacian filter 5e3a89a2e630ae9d0199ff388217e2d5a11c4f86
    • Implement rgb_to_gray 9a2bea6057f4cf99eb6c16c96d5a1c952d95b4b2
    • Implement vectorised confusion_matrix f30606209f20f9f2d879b2eaec80215cc274a80a
    • Implement normalization on tensors 4c3f8fa52d3b9d86843716a99d5c833e80929212
    • Implement rgb_to_bgr e25f6a4900ede8786a0eee38f58f4ffd0908535a
    • Implement hsv_to_rgb 9726872019d71c3b9a3e7cabaf51e77a96220a45
    • Implement adjust_brightness b8fd8b6bce1707ea8a0b2fd5ba9498fe10d586b8

    Bug Fixes

    • Normalize filtering functions kernels a301e3cf6192aff4cbbadda979cc48b17504684f
    • Fix the bug in spatial gradient with padding 5635e45830461d9f40da68a3318755d50a425b17
    • Disable JIT tests 464931720d7b2609ca25f95f29b7a47ba5af2e2f
    • Fix pyrdown with avg_pool2d b83514302232cf8bc31c30f3981a168dd7b55e39
    • Fix formulation issue in rotation_matrix_to_quaternion 58c6e8e7038ad1ca4d9051e04b54b6a42fd72a74
    • Switch torch.gesv -> torch.solve in get_perspective_transform c347a41e85eae78d73ea821b06623383d7a142a4
    • Fix and refactor test_warp_perspective d19121effb69d4c17d53f6bea010941cb7730f32
    • Fixed and updated the quaternion related docs to reflect quaternions 0161f65831ab9f975575586c4c1b1aec6e8a6b11
    • Remove some unused test functions a64a8fb80e5e666caefc806325a2c338ee50f81f

    Contributors

    • @anibali
    • @carlosb1
    • @ducha-aiki
    • @dvd42
    • @edgarriba
    • @jiangwei221
    • @priba
    • @varunagrawal
    Source code(tar.gz)
    Source code(zip)
  • v0.1.2(Mar 14, 2019)

    Package

    • Migrated the project to Arraiy Open Source Organization: https://github.com/arraiyopensource/torchgeometry. 48ad11f39f69be95fe35c164414ad58e0034f5d4
    • Update with support of PyTorch v1.0.1. In fact, we test each time against nightly builds. 5c9d9ae1ccf13fc2381d62be6f5b4c81c265608b
    • Fix issue with pip package PyTorch minimal version. Now we require at least v1.0.0. 6e16734d68074f22fb67a8b1c4418e4917e5b1f1
    • Package version file is auto-generate and too keep tracked sha. f337b3c131b2482a961f5aa895a9b344814bff5a
    • Added codecov support to keep tracked tested code. e609b2112f25806d02364681d57a29b977950f59

    Breaking Changes

    • Refactor DepthWarper API - now accepts PinholeCamera objects as parameters:
    >>> # pinholes camera models
    >>> pinhole_dst = tgm.PinholeCamera(...)
    >>> pinhole_src = tgm.PinholeCamera(...)
    >>> # create the depth warper, compute the projection matrix
    >>> warper = tgm.DepthWarper(pinhole_dst, height, width)
    >>> warper.compute_projection_matrix(pinhole_src)
    >>> # warp the destionation frame to reference by depth
    >>> depth_src = torch.ones(1, 1, 32, 32)  # Nx1xHxW
    >>> image_dst = torch.rand(1, 3, 32, 32)  # NxCxHxW
    >>> image_src = warper(depth_src, image_dst)  # NxCxHxW
    

    New Features

    • Added new PinholeCamera API to represent pinhole camera models. b6ec592bc4d00ba942d0f3d2085534acdefd783f pinhole_model
    • Refactor and moved code from conversions.py and created a dedicated module for linear transforms transformations.py. a1c25b1c5e3a5ac4e5109ea0e1b7256ba8e4ee56
      • boxplus_transformation, boxminus_transformation, inverse_transformation, transform_points.
    • Added a collection of losses:
      • Image: SSIM f08812168984174d6054c5b21298963cdf421cd8
      • Depth: InverseDepthSmoothnessLoss 42a1d22df0691444664c182eae7fc10acaa428cc
      • Semantic segmentation:
        • Diceloss 9b0fddf9055cb9a948856d087b52073551c44129
        • TerskyLoss 89246d269739f89ed0731f52ff543863882efa48
        • FocalLoss ffe4cb1b74ecb81baef05f97fe6c62f176336fd7
    • Added SpatialSoftArgmax2d operator to extract 2D coordinates from probability maps. cf7bb292dbe19242e0b207a8747da601a27e4cf3
    • Added extract_tensor_patches routine similar to tf.extract_image_patches but for multidimensional tensors instead of images. f60fa57b4dcf9462e443ee71bf571cc6e31a7939
    • Added boxplus_transform and boxminus_transform to compose or compute relative pose functions. e0882ea32bb13e62275b678ddb60915058397d35

    Bug Fixes

    • Fixed DepthWarper in order to accept mini-batch computation. 7175b4f93f5cb855eb8ab4011c33e79cb32bf3fa
    • Added missing tests for warp_affine. 57cbd29aa291dc0cf60e6cff6b0665c91db39330
    • Fixed and refactored quaternion_to_axis_angle and axis_angle_to_quaternion to avoid nans. 4aa0bca9cd2ab95b3edb7d043ff16d473b2e04b7

    Test

    • Updated code with python typing: https://docs.python.org/3/library/typing.html to perform static analysis tests using MyPy: http://mypy-lang.org 3c02b58e1d4de3e7583020c332a7b982c9e97d74
    • Added pytest fixtures to split between CPU/CUDA tests. Additionally, we added Makefile commands to launch the tests. 1aff7f65d6535e88abc0d5383846be75d37e4af9
    • Improved InversePose tests. d6a508c7600e5e464d6910028f7771f8d18fe722
    • Improved HomographyWarper tests. 43bd8c2ea669d89d57a063998b491be18d2ab39a

    contributors:

    • @edgarriba
    • @carlosb1
    • @Wizaron
    • @prlz77
    • @kajal-puri
    Source code(tar.gz)
    Source code(zip)
  • v0.1.1(Jan 8, 2019)

    Table of Contents

    • Breaking Changes
    • New Features
    • Bug Fixes
    • Documentation improvements

    Breaking Changes

    • tgm.inverse has been removed since now Pytorch supports batched version for torch.inverse f6c210d87ef63e6a07917cdf88b44091c52ede69

    New Features

    • Added tgm.warp_perspective matching OpenCV interface d53cbce5d779fbc92b3449ed3c3c88f55f8b88ec
    • Added tgm.get_perspective_transform matching OpenCV interface a7db348c5978efc7870c649757947123bfbf78fb
    • Added tgm.get_rotation_matrix2d matching OpenCV interface 876b2c601d4d5028ab84cc2735fa3fbb018b4c1b

    Bug Fixes

    • Fixed bug for inplace operation in tgm.inverse_pose 0aba15d0b3ea79cfc3ed6a31ba088f86db643658

    Documentation improvements

    • Added notebook tutorial for tgm.warp_affine and tgm.warp_pesrpective 894bf52b6c1be9780e190051e8d7b9b679733e2e

    Other improvements

    • Update to Pytorch v1.0.0 3ee14c83285f05ce8d29725a0489d0c9f8b058ed
    • Refactor in testing framework. Removed unittest and now using pytest since it's easy to parametrize unit tests.
      • parametrized tests for different batch sizes
      • parametrized tests for device types: cpu and cuda. Note: need to test on cuda yet.
    • Now we have official pip package to install the library: pip install torchgeometry
      • URL: https://pypi.org/project/torchgeometry/0.1.1
    Source code(tar.gz)
    Source code(zip)
  • v0.1.0(Oct 3, 2018)

    This is the initial release of the torchvision package.

    It contains a set of routines and modules for geometric computer vision implementing multi-view reprojection primitives that work on images and feature maps warping. In addition, we provide routines for conversions and utilities for the pinhole model cameras.

    Table of Contents

    API

    • Pinhole
      • inverse_pose 001097c
      • pinhole_matrix 063b0c6
      • inverse_pinhole_matrix 063b0c6
      • scale_pinhole 063b0c6
      • homography_i_H_ref 063b0c6
    • Conversions
      • pi fb56c9c
      • rad2deg fb56c9c
      • deg2rad fb56c9c
      • convert_points_from_homogeneous f13c462
      • convert_points_to_homogeneous f13c462
      • transform_points 1a1511c
      • angle_axis_to_rotation_matrix 4c929d5
      • rtvec_to_pose 4c929d5
      • rotation_matrix_to_angle_axis f999165
      • rotation_matrix_to_quaternion f999165
      • quaternion_to_angle_axis f999165
    • Warping
      • HomographyWarper 2e54229
      • DepthWarper cd03569
    • Utils
      • tensor_to_image 3ad7d05
      • image_to_tensor 3ad7d05
      • inverse 1a1511c

    Test

    • docker kit ae8d9e1
    • automated test with TravisCI c213e3e
    • lint tests c213e3e

    Documentation

    • sphinx documentation b9251ab
    • development e800c36

    Examples:

    • homography regression ce1524d
    • image warped by depth f999165
    • jupiter notebooks 3c96835
    Source code(tar.gz)
    Source code(zip)
Owner
kornia
Computer Vision for PyTorch
kornia
Aloception is a set of package for computer vision: aloscene, alodataset, alonet.

Aloception is a set of package for computer vision: aloscene, alodataset, alonet.

Visual Behavior 65 Oct 21, 2021
Fast image augmentation library and easy to use wrapper around other libraries. Documentation: https://albumentations.ai/docs/ Paper about library: https://www.mdpi.com/2078-2489/11/2/125

Albumentations Albumentations is a Python library for image augmentation. Image augmentation is used in deep learning and computer vision tasks to inc

null 9k Oct 22, 2021
A general list of resources to image text localization and recognition 场景文本位置感知与识别的论文资源与实现合集 シーンテキストの位置認識と識別のための論文リソースの要約

Scene Text Localization & Recognition Resources Read this institute-wise: English, 简体中文. Read this year-wise: English, 简体中文. Tags: [STL] (Scene Text L

Karl Lok (Zhaokai Luo) 835 Oct 20, 2021
CVPR 2021 Oral paper "LED2-Net: Monocular 360˚ Layout Estimation via Differentiable Depth Rendering" official PyTorch implementation.

LED2-Net This is PyTorch implementation of our CVPR 2021 Oral paper "LED2-Net: Monocular 360˚ Layout Estimation via Differentiable Depth Rendering". Y

Fu-En Wang 48 Oct 20, 2021
📷 This repository is focused on having various feature implementation of OpenCV in Python.

?? This repository is focused on having various feature implementation of OpenCV in Python. The aim is to have a minimal implementation of all OpenCV features together, under one roof.

Aditya Kumar Gupta 115 Oct 19, 2021
The Open Source Framework for Machine Vision

SimpleCV Quick Links: About Installation [Docker] (#docker) Ubuntu Virtual Environment Arch Linux Fedora MacOS Windows Raspberry Pi SimpleCV Shell Vid

Sight Machine 2.5k Oct 13, 2021
Scene text detection and recognition based on Extremal Region(ER)

Scene text recognition A real-time scene text recognition algorithm. Our system is able to recognize text in unconstrain background. This algorithm is

HSIEH, YI CHIA 150 Aug 31, 2021
👄 The most accurate natural language detection library for Java and the JVM, suitable for long and short text alike

Quick Info this library tries to solve language detection of very short words and phrases, even shorter than tweets makes use of both statistical and

Peter M. Stahl 422 Oct 13, 2021
A curated list of resources for text detection/recognition (optical character recognition ) with deep learning methods.

awesome-deep-text-detection-recognition A curated list of awesome deep learning based papers on text detection and recognition. Text Detection Papers

null 2.3k Oct 26, 2021
pulse2percept: A Python-based simulation framework for bionic vision

pulse2percept: A Python-based simulation framework for bionic vision Retinal degenerative diseases such as retinitis pigmentosa and macular degenerati

null 55 Oct 21, 2021
OpenMMLab Text Detection, Recognition and Understanding Toolbox

Introduction English | 简体中文 MMOCR is an open-source toolbox based on PyTorch and mmdetection for text detection, text recognition, and the correspondi

OpenMMLab 1.7k Oct 23, 2021
Open Source Computer Vision Library

OpenCV: Open Source Computer Vision Library Resources Homepage: https://opencv.org Courses: https://opencv.org/courses Docs: https://docs.opencv.org/m

OpenCV 57.4k Oct 23, 2021
A collection of resources (including the papers and datasets) of OCR (Optical Character Recognition).

OCR Resources This repository contains a collection of resources (including the papers and datasets) of OCR (Optical Character Recognition). Contents

Zuming Huang 343 Sep 25, 2021
Roboflow makes managing, preprocessing, augmenting, and versioning datasets for computer vision seamless.

Roboflow makes managing, preprocessing, augmenting, and versioning datasets for computer vision seamless. This is the official Roboflow python package that interfaces with the Roboflow API.

Roboflow 7 Oct 13, 2021
Introduction to Augmented Reality (AR) with Python 3 and OpenCV 4.2.

Introduction to Augmented Reality (AR) with Python 3 and OpenCV 4.2.

fernanda rodríguez 44 Oct 4, 2021
An organized collection of tutorials and projects created for aspriring computer vision students.

A repository created with the purpose of teaching students in BME lab 308A- Hanoi University of Science and Technology

Givralnguyen 5 Sep 17, 2021
Links to awesome OCR projects

Awesome OCR This list contains links to great software tools and libraries and literature related to Optical Character Recognition (OCR). Contribution

Konstantin Baierer 1.9k Oct 20, 2021
Text recognition (optical character recognition) with deep learning methods.

What Is Wrong With Scene Text Recognition Model Comparisons? Dataset and Model Analysis | paper | training and evaluation data | failure cases and cle

Clova AI Research 2.6k Oct 20, 2021
In this project we will be using the live feed coming from the webcam to create a virtual mouse with complete functionalities.

Virtual Mouse Using OpenCV In this project we will be using the live feed coming from the webcam to create a virtual mouse using hand tracking. Projec

Hassan Shahzad 4 Oct 1, 2021