Banpei is a Python package of the anomaly detection.

Related tags

Machine Learning python time-series-analysis anomaly-detection
Overview

Banpei

Build Status

Banpei is a Python package of the anomaly detection.
Anomaly detection is a technique used to identify unusual patterns that do not conform to expected behavior.

System

Python ^3.6 (2.x is not supported)

Installation

$ pip install banpei

After installation, you can import banpei in Python.

$ python
>>> import banpei

Usage

Example

Singular spectrum transformation(sst)

import banpei 
model   = banpei.SST(w=50)
results = model.detect(data)

The input 'data' must be one-dimensional array-like object containing a sequence of values.
The output 'results' is Numpy array with the same size as input data.
The graph below shows the change-point scoring calculated by sst for the periodic data.

sst_example

The data used is placed as '/tests/test_data/periodic_wave.csv'. You can read a CSV file using the following code.

import pandas as pd
raw_data = pd.read_csv('./tests/test_data/periodic_wave.csv')
data = raw_data['y']

SST processing can be accelerated using the Lanczos method which is one of Krylov subspace methods by specifying True for the is_lanczos argument like below.

results = model.detect(data, is_lanczos=True)

Real-time monitoring with Bokeh

Banpei is developed with the goal of constructing the environment of real-time abnormality monitoring. In order to achieve the goal, Banpei has the function corresponded to the streaming data. With the help of Bokeh, which is great visualization library, we can construct the simple monitoring tool.
Here's a simple demonstration movie of change-point detection of the data trends.

sst detection
https://youtu.be/7_woubLAhXk
The sample code how to construct real-time monitoring environment is placed in '/demo' folder.

The implemented algorithm

Outlier detection

  • Hotelling's theory

Change point detection

  • Singular spectrum transformation(sst)

License

This project is licensed under the terms of the MIT license, see LICENSE.

Comments
  • Citation for banpei

    Citation for banpei

    Hello, I am using banpei for change point detection in my thesis. I would like to cite this work but could not find any citations. It would be great to have Bibtex entry for citation. Thank You

    opened by djoshi5 4
  • test_sst.py

    test_sst.py

    Hello,

    I have tried to run the tests/test_sst.py. I got following output:

    ``test_sst.py -v test_detect_by_lanczos (main.TestSST) ... ok test_detect_by_svd (main.TestSST) ... ok test_stream_detect (main.TestSST) ... ok

    Ran 3 tests in 0.019s

    OK `` How do I get the change point score as an output?

    Thanks

    opened by codeXR1 2
  • Get the number positivity

    Get the number positivity

    I have a rule that throws numbers between -50 and 50 randomly, is there any way to predict the sign (positive or negative) of the next release with at least 90% accuracy based on a historical record?

    opened by lorenzolopez928 2
  • detect anomalies on sequential data

    detect anomalies on sequential data

    Hi i'm working on anomaly detection on energy consumption so i have a semi-sequential data. i tested the sst method and i don't think it works form me because it just detect a changing point, i tried to test the Hotelling method but i didn't understand how it works, can someone help me please.

    opened by FeizNouri 2
  • ERROR: Could not find a version that satisfies the requirement banpei

    ERROR: Could not find a version that satisfies the requirement banpei

    I am trying to do pipenv install banpei in my virtual env, it is throwing the following error:

    Installing banpei…
Error:  An error occurred while installing banpei!
Error text: 
ERROR: Could not find a version that satisfies the requirement banpei (from -r /tmp/pipenv-00uw2ca1-requirements/pipenv-x3l4m4in-requirement.txt (line 1)) (from versions: none)
ERROR: No matching distribution found for banpei (from -r /tmp/pipenv-00uw2ca1-requirements/pipenv-x3l4m4in-requirement.txt (line 1))

    banpei

    opened by kashyapm94 1
  • ERROR: No matching distribution found for banpei

    ERROR: No matching distribution found for banpei

    pip install banpei is throwing error:

    ERROR: Could not find a version that satisfies the requirement banpei (from versions: none) ERROR: No matching distribution found for banpei

    I have been using Python 3.7.4 2020-07-16

    opened by mnvsekhar 1
  • module 'banpei' has no attribute 'SST'

    module 'banpei' has no attribute 'SST'

    Following the Usage tutorial got the following error:

    AttributeError                            Traceback (most recent call last)
<ipython-input-6-99f3ee034564> in <module>
----> 1 model   = banpei.SST(w=20)
      2 results = model.detect(data)

AttributeError: module 'banpei' has no attribute 'SST'
    opened by eechava6 0
Owner
Hirofumi Tsuruta
Researcher
Hirofumi Tsuruta
GitHub
High performance, easy-to-use, and scalable machine learning (ML) package, including linear model (LR), factorization machines (FM), and field-aware factorization machines (FFM) for Python and CLI interface.

What is xLearn? xLearn is a high performance, easy-to-use, and scalable machine learning package that contains linear model (LR), factorization machin

Chao Ma 3k Jan 8, 2023
Python package for stacking (machine learning technique)

vecstack Python package for stacking (stacked generalization) featuring lightweight functional API and fully compatible scikit-learn API Convenient wa

Igor Ivanov 671 Dec 25, 2022
A Python Package to Tackle the Curse of Imbalanced Datasets in Machine Learning

imbalanced-learn imbalanced-learn is a python package offering a number of re-sampling techniques commonly used in datasets showing strong between-cla

null 6.2k Jan 1, 2023
A Python package for time series classification

pyts: a Python package for time series classification pyts is a Python package for time series classification. It aims to make time series classificat

Johann Faouzi 1.4k Jan 1, 2023
ELI5 is a Python package which helps to debug machine learning classifiers and explain their predictions

A library for debugging/inspecting machine learning classifiers and explaining their predictions

null 154 Dec 17, 2022
ArviZ is a Python package for exploratory analysis of Bayesian models

ArviZ (pronounced "AR-vees") is a Python package for exploratory analysis of Bayesian models. Includes functions for posterior analysis, data storage, model checking, comparison and diagnostics

ArviZ 1.3k Jan 5, 2023
MBTR is a python package for multivariate boosted tree regressors trained in parameter space.

MBTR is a python package for multivariate boosted tree regressors trained in parameter space.

SUPSI-DACD-ISAAC 61 Dec 19, 2022
Python package for machine learning for healthcare using a OMOP common data model

This library was developed in order to facilitate rapid prototyping in Python of predictive machine-learning models using longitudinal medical data from an OMOP CDM-standard database.

Sontag Lab 75 Jan 3, 2023
UpliftML: A Python Package for Scalable Uplift Modeling

UpliftML is a Python package for scalable unconstrained and constrained uplift modeling from experimental data. To accommodate working with big data, the package uses PySpark and H2O models as base learners for the uplift models. Evaluation functions expect a PySpark dataframe as input.

Booking.com 254 Dec 31, 2022
scikit-multimodallearn is a Python package implementing algorithms multimodal data.

scikit-multimodallearn is a Python package implementing algorithms multimodal data. It is compatible with scikit-learn, a popul

null 12 Jun 29, 2022
MICOM is a Python package for metabolic modeling of microbial communities

Welcome MICOM is a Python package for metabolic modeling of microbial communities currently developed in the Gibbons Lab at the Institute for Systems

null 57 Dec 21, 2022
A data preprocessing package for time series data. Design for machine learning and deep learning.

A data preprocessing package for time series data. Design for machine learning and deep learning.

Allen Chiang 152 Jan 7, 2023
A simple machine learning package to cluster keywords in higher-level groups.

Simple Keyword Clusterer A simple machine learning package to cluster keywords in higher-level groups. Example: "Senior Frontend Engineer" --> "Fronte

Andrea D'Agostino 10 Dec 18, 2022
Data science, Data manipulation and Machine learning package.

duality Data science, Data manipulation and Machine learning package. Use permitted according to the terms of use and conditions set by the attached l

David Kundih 3 Oct 19, 2022
YouTube Spam Detection with python

YouTube Spam Detection This code deletes spam comment on youtube videos based on two characteristics (currently) If the author of the comment has a se

MohamadReza Taalebi 5 Sep 27, 2022
This repo implements a Topological SLAM: Deep Visual Odometry with Long Term Place Recognition (Loop Closure Detection)

This repo implements a topological SLAM system. Deep Visual Odometry (DF-VO) and Visual Place Recognition are combined to form the topological SLAM system.

Best of Australian Centre for Robotic Vision (ACRV) 32 Jun 23, 2022
Credit Card Fraud Detection, used the credit card fraud dataset from Kaggle

Credit Card Fraud Detection, used the credit card fraud dataset from Kaggle

Sean Zahller 1 Feb 4, 2022
Python library which makes it possible to dynamically mask/anonymize data using JSON string or python dict rules in a PySpark environment.

pyspark-anonymizer Python library which makes it possible to dynamically mask/anonymize data using JSON string or python dict rules in a PySpark envir

null 6 Jun 30, 2022
Educational python for Neural Networks, written in pure Python/NumPy.

Educational python for Neural Networks, written in pure Python/NumPy.

null 127 Oct 27, 2022