Rhyme with AI

Overview

Local development

Create a conda virtual environment and activate it:

conda env create --file environment.yml
conda activate rhyme-with-ai

Install the rhyme_with_ai package and all its dependencies:

pip install --editable .

Download the weights of the models (if you get any errors, make sure these align with those specified in app/app.py)):

make download-data

Run the app:

make streamlit

Test the Docker containers by running:

make docker-build
make docker-serve

This project uses black for code formatting. To incorporate it in your version control follow the instructions below (copied from black's own readme):


Use pre-commit. Once you have it installed, add this to the .pre-commit-config.yaml in your repository:

repos:
-   repo: https://github.com/ambv/black
    rev: stable
    hooks:
    - id: black
      language_version: python3.7

Then run pre-commit install and you're ready to go.

Deploy to App Engine

Follow Google's documentation to set up Custom Runtimes in the App Engine Flexible Environment. Deploy the app:

gcloud app deploy

And you're done!

Todo

  • Integrate TokenWeighter in the RhymeGenerator.
  • Don't block on model loading or rhyme mutations (use API?).
You might also like...
Comments
  • add dutch support

    add dutch support

    TODO:

    • [x] Experiment with other models (https://huggingface.co/models?search=dutch), we'd probably prefer a whole-word-masking model like the english one. (only dutch that's automatic in hf, assume it's the best one)
    • [x] Add caching for the Dutch model weights (and turn it back on for th english weights) (done)
    opened by RensDimmendaal 1
  • App.py needs to be updated

    App.py needs to be updated

    https://github.com/huggingface/transformers/issues/2392

    This is the new location for the huggingface library: https://huggingface.co/dbmdz/bert-large-cased-finetuned-conll03-english

    After making this alteration I got it working:

    LANGUAGE = st.sidebar.radio("Language",["english","dutch"],0) if LANGUAGE == "english": MODEL_PATH = "./dbmdz/bert-large-cased-finetuned-conll03-english" elif LANGUAGE == "dutch": MODEL_PATH = "./data/wietsedv/bert-base-dutch-cased" else: raise NotImplementedError(f"Unsupported language ({LANGUAGE}) expected 'english' or 'dutch'.")

    opened by freke70 1
Owner
GoDataDriven
GoDataDriven