Scooch Configures Object Oriented Class Hierarchies for python

Related tags

Configuration scooch
Overview

Scooch

Scooch Configures Object Oriented Class Hierarchies for python.

A good place to start with Scooch is at the documentation found here.

Scooch is distributed via pypi, so it may be pip installed:

pip install scooch

Overview

What is Scooch?

Scooch is a recursive acronym for Scooch Configures Object Oriented Class Hierarchies, and that's what this package does. It is a configuration package for python codebases that simplifies the problem of configuring parameters in python code by translating YAML configuration files into object oriented class hierarchies.

Who needs Scooch?

Scooch is useful for people who need an accessible interface to enable tweakability in their code. ML practitioners are a good example. They typically write code that is intended to be continuously experimented with and adjusted in response to observations from running the code. As such, it is useful to abstract these tweakable parameters from the code into a config file, providing three major benefits:

  • The config file provides a centralized location for adjustable parameters of interest in the code, improving iteration and workflow.
  • Loading, saving and adjusting the configuration of your code is separated from the many other working variables and data structures that may exist in code.
  • The configuration of any part of the code can be hashed, logged, and indexed, to provide a record of the code configuration at any one time.

Why use Scooch?

There are many other projects out there that endeavor to translate config files into parameters in python code, for example:

However, what makes Scooch different is that it not only translates config parameters into variables in your code, but into object oriented class hierarchies. This means configurations can benefit from object oriented concepts such as Inheretance, Encapsulation, Abstraction and Polymorphism.

The benefits of Scooch are outlined in more detail in the Why Object Oriented Configs? section of the docs.

What does a Scooch config look like?

Scooch config files map parameter configurations directly to python class hierarchies, and so your config file becomes a description of all configurable class's inside your class hierarchy. For example, a class designed to batch samples for training an ML model that uses gradient descent might have a config.yaml file that looks something like:

Batcher:
    batch_size: 128
    feature:
        SpectrogramFeature:
            hop_size: 128
            n_bins: 256
    augmenters:
        - NoiseAugmenter:
            augmentations_per_sample: 3
            min_noise: -20 
            max_noise: 20 
        - TranslationAugmenter:
            augmentations_per_sample: 5
            displacement_variance: 100

Here each class is defined in camel case above, while configruable parameters of each class are written in lower-case with underscores.

How is a Scooch configuration translated into code?

Each class in this configuration corresponds directly to a scooch Configurable class in python code. For example the source code for the configuration above might have the following Configurable class definitions in batcher.py:

from scooch import Configurable
from scooch import Param
from scooch import ConfigurableParam
from scooch import ConfigList

class SpectrogramFeature(Configurable):

    _hop_size = Param(int, default=128, doc="Number of samples between successive spectrogram frames")
    _n_bins = Param(int, default=1024, doc="Number of frequency bins in the spectrogram")

    ...

class Augmenter(Configurable):

    _augmentations_per_sample = Param(int, default=3, doc="Number of augmented samples produced per input sample")

    ...

class NoiseAugmenter(Augmenter):

    _min_noise = Param(float, default=-10.0, doc="Minimum amount of noise added per sample, in dB")
    _max_noise = Param(float, default=10.0, doc="Maximum amount of noise added per sample, in dB")

    ...

class TranslationAugmenter(Augmenter):

    displacement_variance = Param(int, default=50, doc="Number of elemets to rotationally translate the sample by")

    ...

class Batcher(Configurable):
    
    _batch_size = Param(int, default=256, doc="Number of samples per batch")
    _feature = ConfigurableParam(SpectrogramFeature, doc="The feature to produce samples of")
    _augmenters = ConfigurableParam(ConfigList(Augmenter), doc="A list of data augmenters to sample from")
    
    ...

In the above snippet, we can see abstraction, polymorphism, inheritance, and encapsulation employed within the classes, and their scooch parameters. Once configured, within each of the classes above the Params and ConfigurableParams will become accessible as attributes of the encapsulating Configurable class instance. Furthermore the scooch Param / ConfigurableParam documentation will be added to the Configurable class's doc string for accessibilty in any auto-generated documentation.

With the class definitions and the config.yaml file provided above, configuring the Batcher class and running the code in a script could be as simple as:

from scooch import Config
from batcher import Batcher

# Construct the object - this is all that is required to configure your class hierarchy.
a_batcher = Batcher(Config('./config.yaml'))

# Configurable values are assigned to class attributes
print(a_batcher._batch_size) # <= prints "128"

# Configurable attributes are constructed and assigned to class attributes
print(a_batcher._feature._n_bins) # <= prints "256"

# Use the class to produce a batch with the configured parameters
samples = a_batcher.get_batch()
You might also like...
A Python library to parse PARI/GP configuration and header files

pari-utils A Python library to parse PARI/GP configuration and header files. This is mainly used in the code generation of https://github.com/sagemath

A compact library for Python 3.10x that allows users to configure their SimPads real-time

SimpadLib v1.0.6 What is this? This is a python library programmed by Ashe Muller that allows users to interface directly with their SimPad devices, a

Python YAML Environment (ymlenv) by Problem Fighter Library

In the name of God, the Most Gracious, the Most Merciful. PF-PY-YMLEnv Documentation Install and update using pip: pip install -U PF-PY-YMLEnv Please

A small example project for efficiently configuring a Python application with YAMLs and the CLI

Hydra Example Project for Python A small example project for efficiently configuring a Python application with YAMLs and the CLI. Why should I care? A

A reusable Django app that configures your project for deployment

django-simple-deploy This app gives you a management command that configures your project for an initial deployment. It targets Heroku at the moment,

An implementation of Geoffrey Hinton's paper
An implementation of Geoffrey Hinton's paper "How to represent part-whole hierarchies in a neural network" in Pytorch.

GLOM An implementation of Geoffrey Hinton's paper "How to represent part-whole hierarchies in a neural network" for MNIST Dataset. To understand this

An attempt at the implementation of GLOM, Geoffrey Hinton's paper for emergent part-whole hierarchies from data
An attempt at the implementation of GLOM, Geoffrey Hinton's paper for emergent part-whole hierarchies from data

GLOM TensorFlow This Python package attempts to implement GLOM in TensorFlow, which allows advances made by several different groups transformers, neu

Pandas Network Analysis: fast accessibility metrics and shortest paths, using contraction hierarchies :world_map:

Pandana Pandana is a Python library for network analysis that uses contraction hierarchies to calculate super-fast travel accessibility metrics and sh

Official implementation of the paper Visual Parser: Representing Part-whole Hierarchies with Transformers
Official implementation of the paper Visual Parser: Representing Part-whole Hierarchies with Transformers

Visual Parser (ViP) This is the official implementation of the paper Visual Parser: Representing Part-whole Hierarchies with Transformers. Key Feature

Python library for serializing any arbitrary object graph into JSON. It can take almost any Python object and turn the object into JSON. Additionally, it can reconstitute the object back into Python.

jsonpickle jsonpickle is a library for the two-way conversion of complex Python objects and JSON. jsonpickle builds upon the existing JSON encoders, s

Python scripts performing class agnostic object localization using the Object Localization Network model in ONNX.
Python scripts performing class agnostic object localization using the Object Localization Network model in ONNX.

ONNX Object Localization Network Python scripts performing class agnostic object localization using the Object Localization Network model in ONNX. Ori

PyTorch wrapper for Taichi data-oriented class

Stannum PyTorch wrapper for Taichi data-oriented class PRs are welcomed, please see TODOs. Usage from stannum import Tin import torch data_oriented =

convert a dict-list object from / to a typed object(class instance with type annotation)

objtyping 带类型定义的对象转换器 由来 Python不是强类型语言,开发人员没有给数据定义类型的习惯。这样虽然灵活,但处理复杂业务逻辑的时候却不够方便——缺乏类型检查可能导致很难发现错误,在IDE里编码时也没

Tracking development of the Class Schedule Siri Shortcut, an iOS program that checks the type of school day and tells you class scheduling.

Class Schedule Shortcut Tracking development of the Class Schedule Siri Shortcut, an iOS program that checks the type of school day and tells you clas

Ffxiv-blended-job-icons - All action icons for each class/job are blended together to create new backgrounds for each job/class icon!
Ffxiv-blended-job-icons - All action icons for each class/job are blended together to create new backgrounds for each job/class icon!

ffxiv-blended-job-icons All action icons for each class/job are blended together to create new backgrounds for each job/class icon! I used python to c

A lightweight, object-oriented finite state machine implementation in Python with many extensions

transitions A lightweight, object-oriented state machine implementation in Python with many extensions. Compatible with Python 2.7+ and 3.0+. Installa

Python object-oriented database

ZODB, a Python object-oriented database ZODB provides an object-oriented database for Python that provides a high-degree of transparency. ZODB runs on

An object-oriented approach to Python file/directory operations.

Unipath An object-oriented approach to file/directory operations Version: 1.1 Home page: https://github.com/mikeorr/Unipath Docs: https://github.com/m

A Python package for Bayesian forecasting with object-oriented design and probabilistic models under the hood.
A Python package for Bayesian forecasting with object-oriented design and probabilistic models under the hood.

Disclaimer This project is stable and being incubated for long-term support. It may contain new experimental code, for which APIs are subject to chang

Releases(v1.0.1)
  • v1.0.1(Dec 3, 2022)

    First complete prod stable release.

    • Bug fixes with many components, e.g.,

      • configurize helper
      • Config macros
      • Recursive population of Configurable defaults
      • etc.
    • New features:

      • Config classes can now accept overrides to programmatically adjust config files at run time
      • Exportable CLI options to add SCOOCH CLI functionality to other SCOOCh supported codebases
      • SCOOCH namespace's have been removed to avoid config clutter. Can be added as necessary by user codebases
      • Improved error handling / notifications of SCOOCH Config to Configurable mismatches
      • Removed ConfigurableParam objects - all SCOOCH Param's now use type inference to determine behavior
      • Programmatically retrieving SCOOCH classes has been moved to the ConfigurableFactory
      • All Configurable's now inherit from common metaclasses to avoid metaclass type conflicts
    Source code(tar.gz)
    Source code(zip)
  • v0.1.0(Nov 9, 2021)

Owner
Pandora Media, Inc.
Pandora Media, Inc.
Configuration Management for Python ⚙

dynaconf - Configuration Management for Python. Features Inspired by the 12-factor application guide Settings management (default values, validation,

Bruno Rocha 2.8k Jan 6, 2023
Flexible Python configuration system. The last one you will ever need.

OmegaConf Description Project Code quality Docs and support OmegaConf is a hierarchical configuration system, with support for merging configurations

Omry Yadan 1.4k Jan 2, 2023
🤫 Easily manage configs and secrets in your Python projects (with CLI support)

Installation pip install confidential How does it work? Confidential manages secrets for your project, using AWS Secrets Manager. First, store a secr

Candid™️ 63 Oct 30, 2022
A modern simfile parsing & editing library for Python 3

A modern simfile parsing & editing library for Python 3

ash garcia 38 Nov 1, 2022
Python Marlin Configurator to make valid configuration files to be used to compile Marlin with.

marlin-configurator Concept originally imagined by The-EG using PowerShell Build Script for Marlin Configurations The purpose of this project is to pa

DevPeeps 2 Oct 9, 2021
Read configuration settings from python configuration files.

Maison Read configuration settings from python configuration files. Motivation When developing a python application, e.g a command-line tool, it can b

null 9 Jan 4, 2023
Simple dataclasses configuration management for Python with hocon/json/yaml/properties/env-vars/dict support.

Simple dataclasses configuration management for Python with hocon/json/yaml/properties/env-vars/dict support, based on awesome and lightweight pyhocon parsing library.

Teo Stocco 62 Dec 23, 2022
A set of Python scripts and notebooks to help administer and configure Workforce projects.

Workforce Scripts A set of Python scripts and notebooks to help administer and configure Workforce projects. Notebooks Several example Jupyter noteboo

Esri 75 Sep 9, 2022
Apt2sbom python package generates SPDX or YAML files

Welcome to apt2sbom This package contains a library and a CLI tool to convert a Ubuntu software package inventory to a software bill of materials. You

Eliot Lear 15 Nov 13, 2022
environs is a Python library for parsing environment variables.

environs: simplified environment variable parsing environs is a Python library for parsing environment variables. It allows you to store configuration

Steven Loria 920 Jan 4, 2023