[Machine Learning Engineer Basic Guide] 부스트캠프 AI Tech - Product Serving 자료

Related tags

Deep Learning serving mlops
Overview

Boostcamp-AI-Tech-Product-Serving

All Contributors


Repository 구조

  • part1(MLOps 개론, Model Serving, 머신러닝 프로젝트 라이프 사이클은 별도의 코드가 없으며, part2 ~ part4는 강의 자료에서 사용한 코드가 저장되어 있습니다
  • assets엔 예제로 사용할 Mask Classification Model이 저장되어 있습니다
    • 만약 실무였다면 part 아래에 있는 폴더들에 같이 저장되는 것이 조금 더 좋지만, 교육용 Github Repo라서 중복 저장을 피하기 위해 이렇게 저장했습니다
├── README.md
├── .github/workflows : Github Action Workflow
├── assets : Mask Classification Model
│   └── mask_task
├── part2
│   ├── 01-voila
│   ├── 02-streamlit
│   └── 04-cicd
├── part3
│   ├── 01-fastapi
│   ├── 02-docker
│   ├── 03-logging
│   └── 04-mlflow
└── part4
    ├── 01-bentoml
    └── 02-airflow

추천 학습 방식

  • 강의를 수강한 후, 강의에서 나온 공식 문서를 보며 코드를 작성합니다
  • 강의에서 활용한 코드를 그대로 복사 붙여넣기하지 말고, 직접 타이핑해주세요
    • 오류를 많이 경험하고, 오류가 발생했다면 그 이유와 해결 방법을 별도로 기록해주세요
  • 강의에서 진행된 코드를 더 좋게 개선해도 좋아요 :)
  • 강의에서 다룬 내용보다 더 넓은 내용을 찾아보고 정리해서 나만의 Serving 노트를 만들어보아요

만약 이슈가 생겼다면

  • 실습 코드 관련 문의는 슬랙 채널에 문의해주세요
    • 어떤 상황에서(OS, Python 버전 등) 어떤 오류가 발생했는지 자세히 알려주시면 좋아요. 같이 해결해보아요!
  • 강의 영상 및 강의 자료 관련 문의는 부스트코스를 통해서 해주세요!
  • 이해하기 어렵거나 토론하고 싶은 소재가 있다면 슬랙 채널에 공유해주세요!
    • 단순히 GET이 뭐에요? 라는 질문보다는 간단히 검색한 후, 검색한 자료를 공유하면서 어떤 부분이 어려운지 질문주시는 것을 추천드려요 :)

협업 규칙

  • 커밋 메시지 컨벤션은 conventional commit을 따릅니다
    • commitizen을 사용하면 더욱 쉽게 커밋할 수 있습니다
  • 작업은 기본적으로 별도의 브랜치를 생성하여 작업합니다. 작업이 완료되면 PR로 리뷰 받습니다
  • PR 리뷰 후 머지 방식은 Squash & Merge를 따릅니다
    • Merge 전에 PR 제목을 되도록이면 convetional commit 형태로 만들어주세요

Contributors ✨

Thanks goes to these wonderful people (emoji key):


Sung Yun Byeon
📆 🚧 💻 📖
Jungwon Seo
💻 🖋 📖 💡
humphrey
💻 🖋 📖 💡
heumsi
💻 🖋 📖 💡
YkMoon
💻 🖋 📖 💡

This project follows the all-contributors specification. Contributions of any kind welcome!

You might also like...
Boostcamp CV Serving For Python

Boostcamp-CV-Serving Prerequisites MySQL GCP Cloud Storage GCP key file Sentry Streamlit Cloud Secrets: .streamlit/secrets.toml #DO NOT SHARE THIS I

Jungwon Seo 19 Feb 22, 2022
A library for building and serving multi-node distributed faiss indices.
A library for building and serving multi-node distributed faiss indices.

About Distributed faiss index service. A lightweight library that lets you work with FAISS indexes which don't fit into a single server memory. It fol

Meta Research 170 Dec 30, 2022
a basic code repository for basic task in CV(classification,detection,segmentation)

basic_cv a basic code repository for basic task in CV(classification,detection,segmentation,tracking) classification generate dataset train predict de

null 1 Oct 15, 2021
Ratatoskr: Worcester Tech's conference scheduling system

Ratatoskr: Worcester Tech's conference scheduling system In Norse mythology, Ratatoskr is a squirrel who runs up and down the world tree Yggdrasil to

null 4 Dec 22, 2022
Automatic number plate recognition using tech: Yolo, OCR, Scene text detection, scene text recognation, flask, torch
Automatic number plate recognition using tech: Yolo, OCR, Scene text detection, scene text recognation, flask, torch

Automatic Number Plate Recognition Automatic Number Plate Recognition (ANPR) is the process of reading the characters on the plate with various optica

Meftun AKARSU 52 Dec 22, 2022
Official repository of my book:
Official repository of my book: "Deep Learning with PyTorch Step-by-Step: A Beginner's Guide"

This is the official repository of my book "Deep Learning with PyTorch Step-by-Step". Here you will find one Jupyter notebook for every chapter in the book.

Daniel Voigt Godoy 340 Jan 1, 2023
Supervised Contrastive Learning for Product Matching

Contrastive Product Matching This repository contains the code and data download links to reproduce the experiments of the paper "Supervised Contrasti

Web-based Systems Group @ University of Mannheim 18 Dec 10, 2022
In this work, we will implement some basic but important algorithm of machine learning step by step.
In this work, we will implement some basic but important algorithm of machine learning step by step.

WoRkS continued English 中文 Français Probability Density Estimation-Non-Parametric Methods(概率密度估计-非参数方法) 1. Kernel / k-Nearest Neighborhood Density Est

liziyu0104 1 Dec 30, 2021
BasicNeuralNetwork - This project looks over the basic structure of a neural network and how machine learning training algorithms work
BasicNeuralNetwork - This project looks over the basic structure of a neural network and how machine learning training algorithms work

BasicNeuralNetwork - This project looks over the basic structure of a neural network and how machine learning training algorithms work. For this project, I used the sigmoid function as an activation function along with stochastic gradient descent to adjust the weights and biases.

Manas Bommakanti 1 Jan 22, 2022
Comments
  • feat(part-2/01-voila): 기존에 사전학습된 마스크 모델을 적용한다

    feat(part-2/01-voila): 기존에 사전학습된 마스크 모델을 적용한다

    일단 1기때 모델 작성하신 캠퍼분께는 다시 한번 사용해도 된다는 허락을 받은 상태입니다!

    몇개 정도 손으로 테스트 해봤을 때는 적당히 잘 나오는 것 같습니다!

    yaml 파일을 활용해서 모델에 대한 경로나 클래스 정보들을 넣어 놨는데, 경로를 상대경로로 따라가다 보니 뭔가 코드가 살짝 지저분해 보이네요 ㅜ

    #TODO뭔가 지저분한..
asset_dir = "../../assets/mask_task/"

with open(os.path.join(asset_dir,"config.yaml")) as f:
    config = yaml.load(f, Loader=yaml.FullLoader)
model_path = os.path.join(asset_dir,config['model_name'])

    한번 확인해보시고, 특별히 크게 문제될게 없다고 생각하시면, streamit 버전도 비슷한 방식으로 진행하도록 하겠습니다!

    opened by thejungwon 3
  • <fix> input tensor device error in predict.py get_prediction

    input tensor device error in predict.py get_prediction

    OS: Linux 1632a42ce5ba 4.4.0-59-generic #80-Ubuntu SMP Fri Jan 6 17:47:47 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux

    GPU: Sun Dec 12 12:41:29 2021
    +-----------------------------------------------------------------------------+ | NVIDIA-SMI 450.80.02 Driver Version: 450.80.02 CUDA Version: 11.0 | |-------------------------------+----------------------+----------------------+ | GPU Name Persistence-M| Bus-Id Disp.A | Volatile Uncorr. ECC | | Fan Temp Perf Pwr:Usage/Cap| Memory-Usage | GPU-Util Compute M. | | | | MIG M. | |===============================+======================+======================| | 0 Tesla V100-SXM2... Off | 00000000:00:05.0 Off | Off | | N/A 32C P0 49W / 300W | 0MiB / 32510MiB | 0% Default | | | | N/A | +-------------------------------+----------------------+----------------------+

    +-----------------------------------------------------------------------------+ | Processes: | | GPU GI CI PID Type Process name GPU Memory | | ID ID Usage | |=============================================================================| | No running processes found | +-----------------------------------------------------------------------------+

    현재 코드는 cpu 환경에서의 serving만을 염두해두었다고 가정을 하였습니다.
    하지만, predict.py 에서 load_model() 모듈에서 model.to(device)로 현재 서버가 gpu인지 cpu인지 고려해주고 있습니다.

    마찬가지로 prediction에서도 input을 model에 argument로 넘겨줄 때 , 현재 서버가 gpu인지 cpu인지 고려해주는 code가 필요한거 같습니다.

    image

    opened by woongjoonchoi 1
Owner
Sung Yun Byeon
Machine Learning Engineer / Data Engineer / Engineering Manager
Sung Yun Byeon
GitHub
Streamlit App For Product Analysis - Streamlit App For Product Analysis

Streamlit_App_For_Product_Analysis Здравствуйте! Перед вами дашборд, позволяющий

Grigory Sirotkin 1 Jan 10, 2022
Roadmap to becoming a machine learning engineer in 2020

Roadmap to becoming a machine learning engineer in 2020, inspired by web-developer-roadmap.

Chris Hoyean Song 1.7k Dec 29, 2022
A simple rest api serving a deep learning model that classifies human gender based on their faces. (vgg16 transfare learning)

this is a simple rest api serving a deep learning model that classifies human gender based on their faces. (vgg16 transfare learning)

crispengari 5 Dec 9, 2021
Scripts of Machine Learning Algorithms from Scratch. Implementations of machine learning models and algorithms using nothing but NumPy with a focus on accessibility. Aims to cover everything from basic to advance.

Algo-ScriptML Python implementations of some of the fundamental Machine Learning models and algorithms from scratch. The goal of this project is not t

Algo Phantoms 81 Nov 26, 2022
Reverse engineer your pytorch vision models, in style

?? Rover Reverse engineer your CNNs, in style Rover will help you break down your CNN and visualize the features from within the model. No need to wri

Mayukh Deb 32 Sep 24, 2022
H2O is an Open Source, Distributed, Fast & Scalable Machine Learning Platform: Deep Learning, Gradient Boosting (GBM) & XGBoost, Random Forest, Generalized Linear Modeling (GLM with Elastic Net), K-Means, PCA, Generalized Additive Models (GAM), RuleFit, Support Vector Machine (SVM), Stacked Ensembles, Automatic Machine Learning (AutoML), etc.

H2O H2O is an in-memory platform for distributed, scalable machine learning. H2O uses familiar interfaces like R, Python, Scala, Java, JSON and the Fl

H2O.ai 6.1k Jan 5, 2023
Model serving at scale

Run inference at scale Cortex is an open source platform for large-scale machine learning inference workloads. Workloads Realtime APIs - respond to pr

Cortex Labs 7.9k Jan 6, 2023
Model Serving Made Easy

The easiest way to build Machine Learning APIs BentoML makes moving trained ML models to production easy: Package models trained with any ML framework

BentoML 4.4k Jan 8, 2023
Serving PyTorch 1.0 Models as a Web Server in C++

Serving PyTorch Models in C++ This repository contains various examples to perform inference using PyTorch C++ API. Run git clone https://github.com/W

Onur Kaplan 223 Jan 4, 2023
Pytorch library for end-to-end transformer models training and serving

Pytorch library for end-to-end transformer models training and serving

Mikhail Grankin 768 Jan 1, 2023