What's banditml?

banditml is a lightweight contextual bandit & reinforcement learning library designed to be used in production Python services. This library is developed by Bandit ML and ex-authors of Facebook's applied reinforcement learning platform, Reagent.

Specifically, this repo contains:

• Feature engineering & preprocessing
• Model implementations
• Model training workflows
• Model serving code for Python services

Supported models

Models supported:

• Contextual Bandits (small datasets)
  • Linear bandit w/ ε-greedy exploration
  • Random forest bandit w/ ε-greedy exploration
  • Gradient boosted decision tree bandit w/ ε-greedy exploration
• Contextual Bandits (medium datasets)
  • Neural bandit with ε-greedy exploration
  • Neural bandit with UCB-based exploration (via. dropout exploration)
  • Neural bandit with UCB-based exploration (via. mixture density networks)
• Reinforcement Learning (large datasets)
  • Deep Q-learning with ε-greedy exploration
  • Quantile regression DQN with UCB-based exploration
  • Soft Actor-Critic

4 feature types supported:

• Numeric: standard floating point features
  • e.g. {totalCartValue: 39.99}
• Categorical: low-cardinality discrete features
  • e.g. {currentlyViewingCategory: "men's jeans"}
• ID list: high-cardinality discrete features
  • e.g. {productsInCart: ["productId022", "productId109"...]}
  • Handled via. learned embedding tables
• "Dense" ID list: high-cardinality discrete features, manually mapped to dense feature vectors
  • e.g {productId022: [0.5, 1.3, ...], productId109: [1.9, 0.1, ...], ...}

Docs

pip install banditml

Get started

License

GNU General Public License v3.0 or later

See COPYING to see the full text.