a dnn ai project to classify which food people are eating on audio recordings

Overview

Deep Learning - EAT Challenge

About

This project is part of an AI challenge of the DeepLearning course 2021 at the University of Augsburg. The objective to be learned is a classification task telling which food people are eating on audio recordings.

Students

This project was created by:

  • Benjamin Möckl
  • Julian Göser
  • Marco Tröster

EAT Dataset Setup

For your convenience, the download of all external project assets (dataset and evaluation metrics) has been automated by a shell script. After executing the script you should be ready to run / develop the project code.

# download and unpack the dataset and metric files
./init_dataset_and_metrics.sh <dataset zip password>

How to Run

First, cache the input dataset as TFRecord files for a training session (e.g. naive training). This should massively improve your training performance (especially with low CPU / GPU resources).

# cache the preprocessed audio dataset as TFRecord file
python src/main.py preprocess_dataset naive

Now, you can launch a training session (e.g. naive training).

# process a training session
python src/main.py run_training naive

After that you can sample all inputs of the unknown test dataset using a trained model and export the prediction results for EAT challenge submission.

# evaluate the results for submission
python src/main.py eval_results naive

Valid training configurations are:

  • naive
  • noisy
  • autoenc
  • amplitude

Remark: Use a GPU empowered machine for amplitude training (although it won't be too rewarding anyways). Tested on Ubuntu 20.04. For running on Windows, the keras ModelCheckpoint Callback has to be switched to our SaveBestAccuracyCallback.

Training Results

Training Approach Description Test Acc. Real Acc.
Naive Train on audio melspectrograms using Conv2D 0.41 0.36
Noisy Train on audio melspectrograms using custom noisy Conv2D 0.44 0.39
Amplitude Train on audio amplitude using Conv1D 0.23 ?.??
AutoEnc Train on audio melspectrograms using an Auto Encoder 0.25 ?.??
You might also like...
An energy estimator for eyeriss-like DNN hardware accelerator
An energy estimator for eyeriss-like DNN hardware accelerator

Energy-Estimator-for-Eyeriss-like-Architecture- An energy estimator for eyeriss-like DNN hardware accelerator This is an energy estimator for eyeriss-

BYOL for Audio: Self-Supervised Learning for General-Purpose Audio Representation
BYOL for Audio: Self-Supervised Learning for General-Purpose Audio Representation

BYOL for Audio: Self-Supervised Learning for General-Purpose Audio Representation This is a demo implementation of BYOL for Audio (BYOL-A), a self-sup

Classify bird species based on their songs using SIamese Networks and 1D dilated convolutions.
Classify bird species based on their songs using SIamese Networks and 1D dilated convolutions.

The goal is to classify different birds species based on their songs/calls. Spectrograms have been extracted from the audio samples and used as features for classification.

ML model to classify between cats and dogs

Cats-and-dogs-classifier This is my first ML model which can classify between cats and dogs. Here the accuracy is around 75%, however , the accuracy c

Learning Open-World Object Proposals without Learning to Classify
Learning Open-World Object Proposals without Learning to Classify

Learning Open-World Object Proposals without Learning to Classify Pytorch implementation for "Learning Open-World Object Proposals without Learning to

Classify the disease status of a plant given an image of a passion fruit
Classify the disease status of a plant given an image of a passion fruit

Passion Fruit Disease Detection I tried to create an accurate machine learning models capable of localizing and identifying multiple Passion Fruits in

An implementation of a discriminant function over a normal distribution to help classify datasets.
An implementation of a discriminant function over a normal distribution to help classify datasets.

CS4044D Machine Learning Assignment 1 By Dev Sony, B180297CS The question, report and source code can be found here. Github Repo Solution 1 Based on t

A Home Assistant custom component for Lobe. Lobe is an AI tool that can classify images.
A Home Assistant custom component for Lobe. Lobe is an AI tool that can classify images.

Lobe This is a Home Assistant custom component for Lobe. Lobe is an AI tool that can classify images. This component lets you easily use an exported m

A deep learning network built with TensorFlow and Keras to classify gender and estimate age.
A deep learning network built with TensorFlow and Keras to classify gender and estimate age.

Convolutional Neural Network (CNN). This repository contains a source code of a deep learning network built with TensorFlow and Keras to classify gend

Owner
Marco Tröster
IT Student (M.Sc.) at University of Augsburg (GER) ### since 2017: IT Freelancer for C# / .NET, Cloud Services and Artificial Intelligence
Marco Tröster
Facestar dataset. High quality audio-visual recordings of human conversational speech.

Facestar Dataset Description Existing audio-visual datasets for human speech are either captured in a clean, controlled environment but contain only a

Meta Research 87 Dec 21, 2022
A two-stage U-Net for high-fidelity denoising of historical recordings

A two-stage U-Net for high-fidelity denoising of historical recordings Official repository of the paper (not submitted yet): E. Moliner and V. Välimäk

Eloi Moliner Juanpere 57 Jan 5, 2023
Real-time analysis of intracranial neurophysiology recordings.

py_neuromodulation Click this button to run the "Tutorial ML with py_neuro" notebooks: The py_neuromodulation toolbox allows for real time capable pro

Interventional Cognitive Neuromodulation - Neumann Lab Berlin 15 Nov 3, 2022
Experimenting with computer vision techniques to generate annotated image datasets from gameplay recordings automatically.

Experimenting with computer vision techniques to generate annotated image datasets from gameplay recordings automatically. The collected data will then be used to train a deep neural network that can detect enemy player models in real time, during gameplay. Finally, a virtual input device will adjust the player's crosshair based on live detections for greater accuracy.

Martin Valchev 3 Apr 24, 2022
MicRank is a Learning to Rank neural channel selection framework where a DNN is trained to rank microphone channels.

MicRank: Learning to Rank Microphones for Distant Speech Recognition Application Scenario Many applications nowadays envision the presence of multiple

Samuele Cornell 20 Nov 10, 2022
A DNN inference latency prediction toolkit for accurately modeling and predicting the latency on diverse edge devices.

Note: This is an alpha (preview) version which is still under refining. nn-Meter is a novel and efficient system to accurately predict the inference l

Microsoft 244 Jan 6, 2023
CUP-DNN is a deep neural network model used to predict tissues of origin for cancers of unknown of primary.

CUP-DNN CUP-DNN is a deep neural network model used to predict tissues of origin for cancers of unknown of primary. The model was trained on the expre

null 1 Oct 27, 2021
ResNEsts and DenseNEsts: Block-based DNN Models with Improved Representation Guarantees

ResNEsts and DenseNEsts: Block-based DNN Models with Improved Representation Guarantees This repository is the official implementation of the empirica

Kuan-Lin (Jason) Chen 2 Oct 2, 2022
PyTorchMemTracer - Depict GPU memory footprint during DNN training of PyTorch

A Memory Tracer For PyTorch OOM is a nightmare for PyTorch users. However, most

Jiarui Fang 9 Nov 14, 2022