Distributed behavioral experiments

Overview

PyPI PyPI pyversions Documentation Status PyPI - Status

License: MPL 2.0 Contributor Covenant

Autopilot

Autopilot Banner Logo

Docs Paper Forum Hardware
Read the Docs Paper Forum Hardware (Coming Soon!)

Autopilot is a Python framework for performing complex, hardware-intensive behavioral experiments with swarms of networked Raspberry Pis. As a tool, it provides researchers with a toolkit of flexible modules to design experiments without rigid programming & API limitations. As a vision, it dreams of bridging the chaotic hacky creativity of scientific programmers with a standardized, communally developed library of reproducible experiment prototypes.

Autopilot was developed with three primary design principles:

  • Flexibility - Autopilot was designed for any hardware and any experiment -- its hardware API is designed to give a structured wrapper around the code you already use, and its task design is entirely non-prescriptive. It attempts to eliminate the need for researchers to use a patchwork of mutually incompatible tools to perform complex experiments. Autopilot is a hacker's plaything -- rather than a uniform, simplified experience, its modular design and complete API-level documentation is meant to encourage users to make and break core Autopilot modules.
  • Efficiency - Autopilot uses Python as a glue around high-performance, low-level libraries, and is fully concurrent across multiple threads, processes, and computers. Its distributed design eliminates the hard limits faced by by single-computer systems, letting researchers use arbitrary numbers and combinations of hardware components to perform complex, hardware-intensive experiments at scale.
  • Reproducibility - Autopilot obsessively documents data provenance, logging the entire history of an Animal's training, including any version and local code changes. Any part of an experiment that isn't documented is considered a bug. By integrating experiments and producing data that is clean at the time of acquisition, Autopilot makes it easy to do good science -- its goal is to allow exact experimental replication from a single file.

Distributed Behavior

Autopilot's premise is simple: to scale experiments, just use more computers.

Autopilot systems consist of multiple "Agents" -- computers with specialized roles in the swarm. One user-facing "Terminal" agent allows a researcher to control many "Pilots," or computers that perform experiments (typically the beloved Raspberry Pi). Each Pilot can coordinate one or many "Children" to offload subsets of an experiment's computational or hardware requirements. Users can use and misuse Autopilot's flexible modules to make whatever agent topology they need <3.

Autopilot System Diagram

Module Overview

Autopilot divides the logical structure of experiments into independent1 modules:

Module
Hardware Agents - Pilot & Terminal Runtime classes that encapsulate a computer/Pi's role in the swarm. Terminals provide the user interface and coordinate subjects and tasks, Pilots do the experiments. Formalizing the Agent API to allow additional agents like Compute or Surveillance agents is a major short-term development goal!
Hardware Hardware - Control your tools! Extensible classes to control whatever hardware you've got.
Hardware Stimuli - Stimulus management and presentation. Parametric sound generation with a realtime audio server built on Jackd. Stubs are present for future development of visual stimuli using Psychopy.
Hardware Tasks - Build experiments! Write some basic metadata to describe data, plots, and hardware and the rest is up to you :)
Hardware Subject - Data management with hdf5 and pyTables. Abstraction layer for keeping obsessive records of subject history and system configuration
Hardware Transforms - Composable data transformations. Need to control the pitch of a sound with a video? build a transformation pipeline to connect your objects
Hardware UI - UI for controlling swarms of Pilots using Qt5/PySide2
Hardware Visualization - (Mostly Prototypes) to do common visualizations

1 a continual work in progress!

Getting Started

All documentation is hosted at https://docs.auto-pi-lot.com

Installation is simple, just install with pip and use Autopilot's guided setup to configure your environment and preferences. The initial setup routine uses a CLI interface that is SSH friendly :)

pip3 install auto-pi-lot
python3 -m autopilot.setup.setup_autopilot

Autopilot Setup Console

All of Autopilot is quite new, so bugs, incomplete documentation, missing features are very much expected! Don't be shy about raising issues or asking questions in the forum.

Development Status

Branch Map

We're working on a formal contribution system, pardon the mess! Until we get that and our CI coverage up, main will lag a bit behind the development branches:

  • dev - main development branch that collects hotfixes, PRs, etc. Unstable but usually has lots of extra goodies
  • hotfix - branches from dev for building and testing hotfixes, PRs back to dev.
  • lab - branches from dev but doesn't necessarily PR back, the local branch used in the maintaining (Wehr) lab
  • parallax - experimental departure from dev to implement a particular experiment and rebuild a lot of components along the way, will eventually return to dev <3

Short-Term

See the short-term development goals in our version milestones:

  • v0.4.0 - Implement registries to separate user code extensions like tasks and local hardware devices in a user directory, preserve source code in produced data so local development isn't lost.
  • v0.5.0 - Make a unitary inheritance structure from a root Autopilot object such that a) common operations like logging and networking are implemented only once, b) the plugin system for v0.4.0 can not only add new objects, but replace core objects while maintaining provenance (ie. no monkey patching needed), c) object behavior that requires coordination across multiple instances gets much easier, making some magical things like self-healing self-discovering networking possible. This will also include a major refactoring of the code structure, finally breaking up some of the truly monstrous thousand-line modules in core into an actually modular system we can build from <3

Long-Term

Autopilot's extended development goals, in their full extravagance, can be found at the Autopilot Development Todo

What's new?

v0.3.0

After much ado, we're releasing Autopilot's first major upgrade. Cameras, Continuous data, DeepLabCut, and a lot more!

  • Autopilot has moved to Python 3!! (Tested on 3.6-3.8)
  • Capturing video with OpenCV and the Spinnaker SDK is now supported (See autopilot.hardware.cameras)
  • An I2C_9DOF motion sensor and the MLX90640 temperature sensor are now supported.
  • Timestamps from GPIO events are now microsecond-precise thanks to some modifications to the pigpio library
  • GPIO output timing is also microsecond-precise thanks to the use of pigpio scripts, so you can deliver exactly the reward volumes you intend <3
  • Hardware modules have been refactored into their own module, and have been almost wholly rebuilt to have sensible inheritance structure.
  • Networking modules are more efficient and automatically compress arrays (like video frames!) on transmission. Streaming is also easier now, check out Net_Node.get_stream() !
  • We now have a detailed development roadmap , so you can see the magnificent future we have planned.
  • We have created the autopilot-users discussion board for troubleshooting & coordinating community development :)

Supported Systems

OS

  • Ubuntu >=16.04
  • raspiOS >=Buster

Python Versions

  • Python==3.7.*

Raspberry Pi Versions

  • Raspi>=3b (Raspi 4 strongly recommended!)
Comments
  • Implement Registries - New protocols not visible

    Implement Registries - New protocols not visible

    I've made a dummy protocol, but was unable to see it from the Terminal's 'New Protocol' window. After some rummaging around I managed to achieve it by manually adding my new protocol to __init__.py in the Tasks directory. It would of course be great if it just appears on its own - and perhaps I've missed something in doing this. Otherwise, hereby a request to make that possible. :-D

    enhancement 
    opened by roaldarbol 19
  • Net_Node.send crashes when its argument

    Net_Node.send crashes when its argument "to" is a list

    Background: I am trying to control 8 speakers. Current approach is to use one Pilot and three Child, each controlling two speakers.

    Right now, I'm just trying to connect one Child to the Pilot. I'm using tasks.gonogo as an example. However, I think it may (?) have a bug in it. https://github.com/wehr-lab/autopilot/blob/dc9833ae827301219d50620fa8e1b22266bb222d/autopilot/tasks/gonogo.py#L170

    Note that the to argument is a list. The documentation of Net_Node.send says this is acceptable. But it crashes here: https://github.com/wehr-lab/autopilot/blob/dc9833ae827301219d50620fa8e1b22266bb222d/autopilot/core/networking.py#L1582 when it tries to convert the list using bytes.

    File "/home/pi/dev/autopilot/autopilot/core/networking.py", line 1582, in send
        self.sock.send_multipart([self.upstream.encode('utf-8'), bytes(msg.to, encoding="utf-8"), msg_enc])
    TypeError: encoding without a string argument
    

    What is the correct syntax for sending messages to a Child? I tried using the name of the child (e.g., "rpi02") as well as "T_rpi02" and the name of the Child task's Net_Node. Then the message sends okay but it doesn't get received. Although this might be a networking problem, I can debug further if I understand better what the to field should be. Thanks!

    bug embarassing 
    opened by cxrodgers 16
  • ZMQ Error

    ZMQ Error

    When I go to start an autopilot pilot, it starts and says:

    fatal: not a git repository (or any of the parent directories): .git fatal: not a git repository (or any of the parent directories): .git

    Followed by the splash screen. Followed by:

    Process Pilot_Station-2: Traceback (most recent call last): File "/usr/lib/python3.7/multiprocessing/process.py", line 297, in _bootstrap self.run() File "/home/pi/git/autopilot/autopilot/networking/station.py", line 173, in run self.listener.setsockopt_string(zmq.IDENTITY, self.id) File "/home/pi/.venv/autopilot/lib/python3.7/site-packages/zmq/sugar/socket.py", line 340, in set_string return self.set(option, optval.encode(encoding)) File "zmq/backend/cython/socket.pyx", line 420, in zmq.backend.cython.socket.Socket.set File "zmq/backend/cython/socket.pyx", line 277, in zmq.backend.cython.socket._setsockopt File "zmq/backend/cython/checkrc.pxd", line 28, in zmq.backend.cython.checkrc._check_rc zmq.error.ZMQError: Invalid argument

    The process doesn't terminate, it stays there, but it doesn't connect to the terminal, which I'm assuming is due to the last error.

    It looks like self.id is set to an empty string in station.py. Did I need to set the local Pilot IP somewhere?

    opened by skeltoh 13
  • Document pilot connection & setup

    Document pilot connection & setup

    Received in discussion board: https://groups.google.com/g/autopilot-users/c/JvWIPpYY0TI/m/MtKAj0h-BwAJ

    Hi Jonny,

    First up, this seems like an amazing framework and I can't wait to get started! And alert: I'm an experimentalist, so a lot of things are new to me figuring out this - which might just be a good thing for providing n00b feedback on the docs. ;-)

    I now have Autopilot installed on both a Terminal (Mac) and a Pilot. However, my knowledge is limited when it comes to connecting the two, and right now there's not much help to get from the docs. I have a vague idea that there's some ssh involved (about which I know pretty much 0), or is it simpler than that? Do they need to be connected to the same WiFi network (mine cannot currently where I am stationed)?

    An aside: A later request for updating the docs could also be a brief mention of how to install and use virtualenv, in case people are not familiar with this way of working. This was the case for me so I had to figure that out elsewhere and hope for the best (works now, but might be very helpful for other first-timers. Also, it sort of works outside a virtual environment, but the Pilot doesn't as the welcome_msg.txt apparently is missing).

    Thanks again, and hope you can help me out here! Cheers, [name removed in case they don't want it here!!!]

    docs 
    opened by sneakers-the-rat 8
  • Change setuptools to poetry

    Change setuptools to poetry

    i am tired of working around the pretty bad dependency specification that relies on keeping a bunch of plaintext requirements file in sync with one another instead of using optional dependencies. Also a bunch of functionality that should be handled by a package manager is done in the distressingly long setup_autopilot.py script. I've been using poetry for all my other projects and love it. Lockfiles are great for reliability and very much in-line with having a particular environment specified for a particular git hash that we're trying to do here. This is also the direction that python packaging is going (away from setup.py and setuptools) and i see why.

    One downside is that poetry packages don't allow editable -e installations right now. using a poetry shell is essentially equivalent, but it's still a foreign thing (and is still a little magical to me).

    enhancement packaging 
    opened by sneakers-the-rat 7
  • LEDS RGB can't be set or pull

    LEDS RGB can't be set or pull

    Hello

    I am trying to implement a simplified version of Freewater task using LEDS RGB and pushbuttons only by developping the Blink task example, I want to trigger the push button (Digital_IN) object to change the color of the led, but the pilot terminal throws always KEY ERROR related to the names of the leds rgb pins; please find attached screen shots for both json file and the script of the blink task with the errors followed in a separate comment

    Regards

    opened by mnmneE 6
  • Support for stereo (two-channel) audio output

    Support for stereo (two-channel) audio output

    My understanding from reading the code is that stereo (two-channel) audio output is not really supported in autopilot right now. I have been working on implementing this in my own fork. It isn't quite ready for a PR yet, but I wanted to share my approach and collect feedback from anyone who is interested in this functionality.

    Here is how (I think) things currently stand in autopilot: There are prefs OUTCHANNELS and NCHANNELS that relate to multi-channel output. I believe that they ultimately serve just to hook up the same "mono" audio stream to one or both physical outputs on the hifiberry. The sound generating classes seem to generate only mono sounds.

    Here are the scenarios and use-cases that I would like to support:

    • One speaker hooked up to one channel
    • Two speakers hooked up to two channels, but always playing the same thing ("mono")
    • Two speakers hooked up to two channels, playing different things ("stereo")
    • Two speakers hooked up to two channels, playing different things, but reversed ("reversed")

    The last scenario would be used in the case where you've hooked everything up but realized you swapped the speakers by accident, and just want to flip a bit in software instead of physically rewiring things.

    I can't really imagine a scenario where we have more than two channels of audio output. I am actually implementing now 8-channel output, and the way I'm doing it is with 4 Pis and 4 Hifiberrys. I would suggest that we hold off on supporting >2 channel output in software until someone comes up with a proposed hardware implementation of this, since right now we don't even know what that would look like.

    My proposal The general idea is that OUTCHANNELS should encapsulate information about how the speakers are physically hooked up, and the arguments to the sound generating classes should encapsulate information about what kind of sound is generated (mono or stereo).

    First, I'm proposing to add a channel argument to the sound-generating classes. It would have the following behavior:

    • channel=None: mono (1-dimensional) waveform, the same as it is now
    • channel=0: stereo output, with channel 0 containing a waveform and channel 1 containing zeros
    • channel=1: stereo output, with channel 1 containing a waveform and channel 0 containing zeros

    Second, I think we could simplify the logic by implementing each use-case with OUTCHANNELS (and dropping NCHANNELS). I propose the following meaning for OUTCHANNELS:

    • [0] or [1]: single output to the corresponding speaker
    • [0, 1]: stereo output. Whether they play mono or stereo sounds would be determined by the sound generating classes (the channel argument above).
    • [1, 0]: reversed stereo output. No change to the sound generating classes would be needed here, just flip this bit in prefs to swap the speakers.

    The default behavior would be:

    • channel=None in the sound generating classes. This means they would generate 1d, mono output, just as they currently do.
    • [0, 1] for OUTCHANNELS. That would send output to both speakers. If the sound generating classes make mono output (default), the same output goes to all speakers. If the user specifies a channel in the sound generating classes, stereo output works right away without changing a pref. Although this is technically a change in the default value of the pref, I don't think the actual default behavior would change. (could be wrong about this)

    Status Currently implemented in https://github.com/cxrodgers/autopilot/tree/stereo_sound

    A comparison vs the "dev" branch: https://github.com/wehr-lab/autopilot/compare/dev...cxrodgers:stereo_sound

    I believe that it's working on my end. I'm a little unsure about the logic in jackclient.py because there are a lot of conditionals relating to continuous sound (or not), currently playing sound (or not), and the queues. I'll check more carefully.

    A few stray observations:

    • I think there is some Matlab-style indexing lurking in the default pref for OUTCHANNELS of [1, 2], this should almost surely be [0, 1].
    • I'm having trouble testing the sound-generating classes because I don't know how to instantiate a jackclient outside of a task, and that jackclient is needed to get the sampling rate and so on. I have noticed a few weirdnesses with sounds, where if I generate the same sound twice it is two different lengths, one the requested length and one a multiple of the chunk size (1024). Still tracking this down

    Please let me know if you have any suggestions! I know I'm proposing a big change to an important component of the project (sound playing) so it should be something everyone is happy with.

    opened by cxrodgers 6
  • Let Terminal GUI remember its previous size

    Let Terminal GUI remember its previous size

    Current behavior is that the Terminal GUI covers the entire desktop, which probably makes sense for certain use-cases but can be annoying in others. I see there are some comments about infinitely expanding buttons so I imagine that was the rationale for this choice.

    Perhaps a more user-friendly behavior would be to remember the previous size and restore it? Can autopilot/prefs.json be a good place to store information like last window size or is there a better place? I can work on this if people think it makes sense.

    On a related note I fixed some Qt DeprecationWarnings on my end with this commit: https://github.com/cxrodgers/autopilot/commit/f22002ec929644930b49114e8392a2449d4367d7 I can include that in a PR too.

    gui 
    opened by cxrodgers 6
  • Setup Error

    Setup Error

    I'm using a RasPi 4B with Raspbian Buster 10 and Python 3.7. I've managed to successfully install autopilot and all dependencies, but when running python3 -m autopilot.setup.setup_autopilot I keep getting the following error:

    /usr/lib/python3.7/runpy.py:125: RuntimeWarning: 'autopilot.setup.setup_autopilot' found in sys.modules after import of package 'autopilot.setup', but prior to execution of 'autopilot.setup.setup_autopilot'; this may result in unpredictable behaviour warn(RuntimeWarning(msg)) Traceback (most recent call last): File "/usr/lib/python3.7/runpy.py", line 193, in _run_module_as_main "__main__", mod_spec) File "/usr/lib/python3.7/runpy.py", line 85, in _run_code exec(code, run_globals) File "/home/pi/.local/lib/python3.7/site-packages/autopilot/setup/setup_autopilot.py", line 691, in <module> setup.run() File "/home/pi/.local/lib/python3.7/site-packages/npyscreen/apNPSApplication.py", line 30, in run return npyssafewrapper.wrapper(self.__remove_argument_call_main) File "/home/pi/.local/lib/python3.7/site-packages/npyscreen/npyssafewrapper.py", line 41, in wrapper wrapper_no_fork(call_function) File "/home/pi/.local/lib/python3.7/site-packages/npyscreen/npyssafewrapper.py", line 97, in wrapper_no_fork return_code = call_function(_SCREEN) File "/home/pi/.local/lib/python3.7/site-packages/npyscreen/apNPSApplication.py", line 25, in __remove_argument_call_main return self.main() File "/home/pi/.local/lib/python3.7/site-packages/npyscreen/apNPSApplicationManaged.py", line 148, in main self.onStart() File "/home/pi/.local/lib/python3.7/site-packages/autopilot/setup/setup_autopilot.py", line 535, in onStart self.pilot_1 = self.addForm('CONFIG_PILOT_1', Pilot_Config_Form_1, name="Setup Pilot Agent - 1/2") File "/home/pi/.local/lib/python3.7/site-packages/npyscreen/apNPSApplicationManaged.py", line 55, in addForm fm = FormClass( parentApp=self, *args, **keywords) File "/home/pi/.local/lib/python3.7/site-packages/autopilot/setup/setup_autopilot.py", line 228, in __init__ super(Autopilot_Form, self).__init__(*args, **kwargs) File "/home/pi/.local/lib/python3.7/site-packages/npyscreen/fmForm.py", line 70, in __init__ self.create() File "/home/pi/.local/lib/python3.7/site-packages/autopilot/setup/setup_autopilot.py", line 498, in create self.populate_form(PILOT_PREFS) File "/home/pi/.local/lib/python3.7/site-packages/autopilot/setup/setup_autopilot.py", line 270, in populate_form widget = self.add(nps.TitleText, name=param['text'], value = str(default)) File "/home/pi/.local/lib/python3.7/site-packages/npyscreen/fmForm.py", line 384, in add_widget *args, **keywords) File "/home/pi/.local/lib/python3.7/site-packages/npyscreen/wgtitlefield.py", line 69, in __init__ self.make_contained_widgets() File "/home/pi/.local/lib/python3.7/site-packages/npyscreen/wgtitlefield.py", line 86, in make_contained_widgets self.label_widget = textbox.Textfield(self.parent, relx=self.relx, rely=self.rely, width=len(self.name)+1, value=self.name, color=self.labelColor) File "/home/pi/.local/lib/python3.7/site-packages/npyscreen/wgtextbox.py", line 44, in __init__ self.update() File "/home/pi/.local/lib/python3.7/site-packages/npyscreen/wgtextbox.py", line 132, in update self._print() File "/home/pi/.local/lib/python3.7/site-packages/npyscreen/wgtextbox.py", line 299, in _print color _curses.error: addwstr() returned ERR

    Any ideas why this appears?

    bug docs 
    opened by roaldarbol 6
  • Stereo sound

    Stereo sound

    This is a PR for issue https://github.com/wehr-lab/autopilot/issues/94

    This implements multi-channel (e.g., stereo) sound.

    Some major changes:

    • The sound classes in autopilot.stim.sound accept a channel argument. (Right now, only Noise does this.) If this is None (the default), a mono (1-dimensional) sound is produced, exactly as before. If this is 0, a Noise burst only in the first channel is produced. If 1, a Noise burst only in the second channel is produced.
    • Multi-channel sounds are properly padded to the correct chunk length in Jack_Sound.chunk (implemented here: https://github.com/cxrodgers/autopilot/blob/4dfe9e77687a8a9083ee7903136f99cc7e63671f/autopilot/stim/sound/sounds.py#L223)
    • The preference OUTCHANNELS is now interpreted differently. If empty string, then plays in mono mode (same sound to all speakers). If a list, connect channels to physical speakers in that order. If an int, treat as a list of length one. (implemented here: https://github.com/cxrodgers/autopilot/blob/4dfe9e77687a8a9083ee7903136f99cc7e63671f/autopilot/stim/sound/jackclient.py#L160)
    • I removed padding code in the process() function because 1) this should already be done by Jack_Sound.chunk; 2) process probably doesn't have time to do this kind of thing; 3) the padding code was causing errors on multi-channel sounds, would need to be reimplemented as in Jack_Sound.chunk, but see reason number 1.
    • OUTCHANNELS default in prefs.py changes to empty string (mono mode, no changes to Task code needed). The old default of [1] suggested 1-based indexing and in any case produced essentially mono results.
    • NCHANNELS deprecated in prefs.py

    Error are raised if:

    • The length of OUTCHANNELS is longer than the number of available ports
    • OUTCHANNELS indicates mono mode, but multi-channel audio is provided
    • OUTCHANNELS is length N, but M-channel audio is provided, where N != M. However, 1-dimensional (mono) audio can be provided, and this will play to all N speakers.

    jonny had suggested using None as the OUTCHANNELS flag for "connect no speakers", but [] works for this more naturally. Also, None cannot easily be put into the json.

    Known issues:

    • Presently only Noise can produce multi-channel output. Others can only produce mono output, as before. Should be an easy improvement.
    • Only works for Jack, not pyo!

    Not tested!

    • pyo is not tested!
    • AUDIOSERVER = 'docs' is not tested!
    • continuous mode is not tested!

    Tested on:

    • jackd from APT and (less thoroughly) jackd built from source

    I know this is a big change, please lmk if changes are requested/needed!

    opened by cxrodgers 5
  • no sound on box 3

    no sound on box 3

    When I put 37 into box 3 there was sound. When I took him out, the sound had stopped playing. This was on the laser step of Mike's gpr26 laser task.

    Logs: pilot_logs.tar.gz no_sound_box3.txt

    bug 
    opened by lucasogit 5
  • Bump setuptools from 62.3.2 to 65.5.1

    Bump setuptools from 62.3.2 to 65.5.1

    Bumps setuptools from 62.3.2 to 65.5.1.

    Release notes

    Sourced from setuptools's releases.

    v65.5.1

    No release notes provided.

    v65.5.0

    No release notes provided.

    v65.4.1

    No release notes provided.

    v65.4.0

    No release notes provided.

    v65.3.0

    No release notes provided.

    v65.2.0

    No release notes provided.

    v65.1.1

    No release notes provided.

    v65.1.0

    No release notes provided.

    v65.0.2

    No release notes provided.

    v65.0.1

    No release notes provided.

    v65.0.0

    No release notes provided.

    v64.0.3

    No release notes provided.

    v64.0.2

    No release notes provided.

    v64.0.1

    No release notes provided.

    v64.0.0

    No release notes provided.

    v63.4.3

    No release notes provided.

    v63.4.2

    No release notes provided.

    ... (truncated)

    Changelog

    Sourced from setuptools's changelog.

    v65.5.1

    Misc ^^^^

    • #3638: Drop a test dependency on the mock package, always use :external+python:py:mod:unittest.mock -- by :user:hroncok
    • #3659: Fixed REDoS vector in package_index.

    v65.5.0

    Changes ^^^^^^^

    • #3624: Fixed editable install for multi-module/no-package src-layout projects.
    • #3626: Minor refactorings to support distutils using stdlib logging module.

    Documentation changes ^^^^^^^^^^^^^^^^^^^^^

    • #3419: Updated the example version numbers to be compliant with PEP-440 on the "Specifying Your Project’s Version" page of the user guide.

    Misc ^^^^

    • #3569: Improved information about conflicting entries in the current working directory and editable install (in documentation and as an informational warning).
    • #3576: Updated version of validate_pyproject.

    v65.4.1

    Misc ^^^^

    • #3613: Fixed encoding errors in expand.StaticModule when system default encoding doesn't match expectations for source files.
    • #3617: Merge with pypa/distutils@6852b20 including fix for pypa/distutils#181.

    v65.4.0

    Changes ^^^^^^^

    v65.3.0

    ... (truncated)

    Commits

    Dependabot compatibility score

    Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting @dependabot rebase.


    Dependabot commands and options

    You can trigger Dependabot actions by commenting on this PR:

    • @dependabot rebase will rebase this PR
    • @dependabot recreate will recreate this PR, overwriting any edits that have been made to it
    • @dependabot merge will merge this PR after your CI passes on it
    • @dependabot squash and merge will squash and merge this PR after your CI passes on it
    • @dependabot cancel merge will cancel a previously requested merge and block automerging
    • @dependabot reopen will reopen this PR if it is closed
    • @dependabot close will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually
    • @dependabot ignore this major version will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)
    • @dependabot ignore this minor version will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)
    • @dependabot ignore this dependency will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)
    • @dependabot use these labels will set the current labels as the default for future PRs for this repo and language
    • @dependabot use these reviewers will set the current reviewers as the default for future PRs for this repo and language
    • @dependabot use these assignees will set the current assignees as the default for future PRs for this repo and language
    • @dependabot use this milestone will set the current milestone as the default for future PRs for this repo and language

    You can disable automated security fix PRs for this repo from the Security Alerts page.

    dependencies 
    opened by dependabot[bot] 0
  • Sounds sometimes play at wrong sample rate

    Sounds sometimes play at wrong sample rate

    Every once in a while, one of my Pis will just start playing sound at the wrong sample rate. I would guess it is playing at 48Khz instead of 192Khz, because the stimuli are several times slower and several times lower-pitched than they should be. Restarting autopilot doesn't fix it, but restarting the pi usually does.

    I'm confused why this is happening, I have checked that jackd is started using the normal parameters, and that the autopilot software believes the sample rate is 192KHz everywhere this variable is used. It's as if the downstream software (alsamixer?) is stuck on the wrong sample rate. Is there any way to check what the hardware thinks its current sample rate is?

    opened by cxrodgers 2
  • Bump certifi from 2022.5.18.1 to 2022.12.7

    Bump certifi from 2022.5.18.1 to 2022.12.7

    Bumps certifi from 2022.5.18.1 to 2022.12.7.

    Commits

    Dependabot compatibility score

    Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting @dependabot rebase.


    Dependabot commands and options

    You can trigger Dependabot actions by commenting on this PR:

    • @dependabot rebase will rebase this PR
    • @dependabot recreate will recreate this PR, overwriting any edits that have been made to it
    • @dependabot merge will merge this PR after your CI passes on it
    • @dependabot squash and merge will squash and merge this PR after your CI passes on it
    • @dependabot cancel merge will cancel a previously requested merge and block automerging
    • @dependabot reopen will reopen this PR if it is closed
    • @dependabot close will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually
    • @dependabot ignore this major version will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)
    • @dependabot ignore this minor version will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)
    • @dependabot ignore this dependency will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)
    • @dependabot use these labels will set the current labels as the default for future PRs for this repo and language
    • @dependabot use these reviewers will set the current reviewers as the default for future PRs for this repo and language
    • @dependabot use these assignees will set the current assignees as the default for future PRs for this repo and language
    • @dependabot use this milestone will set the current milestone as the default for future PRs for this repo and language

    You can disable automated security fix PRs for this repo from the Security Alerts page.

    dependencies 
    opened by dependabot[bot] 0
  • Bump pillow from 9.1.1 to 9.3.0

    Bump pillow from 9.1.1 to 9.3.0

    Bumps pillow from 9.1.1 to 9.3.0.

    Release notes

    Sourced from pillow's releases.

    9.3.0

    https://pillow.readthedocs.io/en/stable/releasenotes/9.3.0.html

    Changes

    ... (truncated)

    Changelog

    Sourced from pillow's changelog.

    9.3.0 (2022-10-29)

    • Limit SAMPLESPERPIXEL to avoid runtime DOS #6700 [wiredfool]

    • Initialize libtiff buffer when saving #6699 [radarhere]

    • Inline fname2char to fix memory leak #6329 [nulano]

    • Fix memory leaks related to text features #6330 [nulano]

    • Use double quotes for version check on old CPython on Windows #6695 [hugovk]

    • Remove backup implementation of Round for Windows platforms #6693 [cgohlke]

    • Fixed set_variation_by_name offset #6445 [radarhere]

    • Fix malloc in _imagingft.c:font_setvaraxes #6690 [cgohlke]

    • Release Python GIL when converting images using matrix operations #6418 [hmaarrfk]

    • Added ExifTags enums #6630 [radarhere]

    • Do not modify previous frame when calculating delta in PNG #6683 [radarhere]

    • Added support for reading BMP images with RLE4 compression #6674 [npjg, radarhere]

    • Decode JPEG compressed BLP1 data in original mode #6678 [radarhere]

    • Added GPS TIFF tag info #6661 [radarhere]

    • Added conversion between RGB/RGBA/RGBX and LAB #6647 [radarhere]

    • Do not attempt normalization if mode is already normal #6644 [radarhere]

    ... (truncated)

    Commits

    Dependabot compatibility score

    Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting @dependabot rebase.


    Dependabot commands and options

    You can trigger Dependabot actions by commenting on this PR:

    • @dependabot rebase will rebase this PR
    • @dependabot recreate will recreate this PR, overwriting any edits that have been made to it
    • @dependabot merge will merge this PR after your CI passes on it
    • @dependabot squash and merge will squash and merge this PR after your CI passes on it
    • @dependabot cancel merge will cancel a previously requested merge and block automerging
    • @dependabot reopen will reopen this PR if it is closed
    • @dependabot close will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually
    • @dependabot ignore this major version will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)
    • @dependabot ignore this minor version will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)
    • @dependabot ignore this dependency will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)
    • @dependabot use these labels will set the current labels as the default for future PRs for this repo and language
    • @dependabot use these reviewers will set the current reviewers as the default for future PRs for this repo and language
    • @dependabot use these assignees will set the current assignees as the default for future PRs for this repo and language
    • @dependabot use this milestone will set the current milestone as the default for future PRs for this repo and language

    You can disable automated security fix PRs for this repo from the Security Alerts page.

    dependencies 
    opened by dependabot[bot] 0
  • installation issue the sequel

    installation issue the sequel

    I get this error when installing with pip3 install auto-pi-lot:

    Building wheels for collected packages: blosc Building wheel for blosc (pyproject.toml) ... error error: subprocess-exited-with-error

    × Building wheel for blosc (pyproject.toml) did not run successfully. │ exit code: 1 ╰─> [9 lines of output] File "/tmp/pip-build-env-ayfnen4r/overlay/local/lib/python3.10/dist-packages/skbuild/setuptools_wrap.py", line 613, in setup cmkr = cmaker.CMaker(cmake_executable) File "/tmp/pip-build-env-ayfnen4r/overlay/local/lib/python3.10/dist-packages/skbuild/cmaker.py", line 141, in init self.cmake_version = get_cmake_version(self.cmake_executable) File "/tmp/pip-build-env-ayfnen4r/overlay/local/lib/python3.10/dist-packages/skbuild/cmaker.py", line 95, in get_cmake_version raise SKBuildError( Traceback (most recent call last):

      Problem with the CMake installation, aborting build. CMake executable is cmake
      [end of output]
    

    note: This error originates from a subprocess, and is likely not a problem with pip. ERROR: Failed building wheel for blosc Failed to build blosc ERROR: Could not build wheels for blosc, which is required to install pyproject.toml-based projects

    opened by lucasogit 1
  • problems opening hdf5 file

    problems opening hdf5 file

    I am somewhat frequently getting HDF5 errors when starting a session.

                        DEBUG    [registry.get] Found <class 'autopilot.tasks.paft.PAFT'> as subclass of <class 'autopilot.tasks.task.Task'> in module                registry.py:111
                                 autopilot.tasks.paft                                                                                                                                
                        DEBUG    [data.subject.Subject.DistractorTaskHigh220726_v8] error opening /home/mouse/autopilot/data/DistractorTaskHigh220726_v8.h5            subject.py:186
    Exception in thread Thread-7:
    Traceback (most recent call last):
      File "/home/mouse/miniconda3/envs/autopilot/lib/python3.9/threading.py", line 950, in _bootstrap_inner
                        DEBUG    [data.subject.Subject.DistractorTaskHigh220726_v8] error opening /home/mouse/autopilot/data/DistractorTaskHigh220726_v8.h5            subject.py:210
    Traceback (most recent call last):
        self.run()
      File "/home/mouse/dev/autopilot/autopilot/gui/widgets/terminal.py", line 442, in toggle_start
      File "/home/mouse/miniconda3/envs/autopilot/lib/python3.9/threading.py", line 888, in run
        self.start_fn(True, self.pilot, current_subject)
      File "/home/mouse/dev/autopilot/autopilot/agents/terminal.py", line 548, in toggle_start
        self._target(*self._args, **self._kwargs)
      File "/home/mouse/dev/autopilot/autopilot/data/subject.py", line 907, in _data_thread
        task = self.subjects[subject].prepare_run()
      File "/home/mouse/dev/autopilot/autopilot/data/subject.py", line 744, in prepare_run
        with self._h5f() as h5f:
      File "/home/mouse/miniconda3/envs/autopilot/lib/python3.9/contextlib.py", line 117, in __enter__
        task_params['step'] = int(self.step)
      File "/home/mouse/dev/autopilot/autopilot/data/subject.py", line 366, in step
        return next(self.gen)
      File "/home/mouse/dev/autopilot/autopilot/data/subject.py", line 183, in _h5f
        return self.protocol.step
      File "/home/mouse/dev/autopilot/autopilot/data/subject.py", line 257, in protocol
        h5f = tables.open_file(str(self.file), mode="r+")
        protocoldict['protocol'] = json.loads(protocol_node.readall())
      File "/home/mouse/miniconda3/envs/autopilot/lib/python3.9/site-packages/tables/nodes/filenode.py", line 246, in readinto
      File "/home/mouse/miniconda3/envs/autopilot/lib/python3.9/site-packages/tables/file.py", line 300, in open_file
        b[:n] = self._node.read(start, stop).tobytes()
      File "/home/mouse/miniconda3/envs/autopilot/lib/python3.9/site-packages/tables/array.py", line 866, in read
        return File(filename, mode, title, root_uep, filters, **kwargs)
      File "/home/mouse/miniconda3/envs/autopilot/lib/python3.9/site-packages/tables/file.py", line 750, in __init__
        arr = self._read(start, stop, step, out)
      File "/home/mouse/miniconda3/envs/autopilot/lib/python3.9/site-packages/tables/array.py", line 823, in _read
        self._g_new(filename, mode, **params)
      File "tables/hdf5extension.pyx", line 486, in tables.hdf5extension.File._g_new
        self._read_array(start, stop, step, arr)
    tables.exceptions.HDF5ExtError: HDF5 error back trace
    
      File "H5F.c", line 620, in H5Fopen
        unable to open file
      File "H5Iint.c", line 1104, in H5I_inc_ref
        can't locate ID
      File "H5PLpath.c", line 578, in H5PL__path_table_iterate
        can't iterate over plugins in plugin path '(null)'
      File "H5PLpath.c", line 620, in H5PL__path_table_iterate_process_path
        can't open directory: /usr/local/hdf5/lib/plugin
      File "H5Iint.c", line 1104, in H5I_inc_ref
        can't locate ID
      File "H5VLnative_file.c", line 97, in H5VL__native_file_open
        unable to open file
      File "H5Fint.c", line 1862, in H5F_open
        file is already open for read-only
    
    End of HDF5 error back trace
    
    Unable to open/create file '/home/mouse/autopilot/data/DistractorTaskHigh220726_v8.h5'
      File "tables/hdf5extension.pyx", line 1537, in tables.hdf5extension.Array._read_array
    tables.exceptions.HDF5ExtError: Problems reading the array data.
    

    I feel like the problem might be here: https://github.com/auto-pi-lot/autopilot/blob/04b5968ba02c8a1413a27eb6a138b6a186b130f1/autopilot/data/subject.py#L189 If one thread has the file open for read/write, and another one tries to open for read, then the try/except here (lines 188-194) will have the second thread open it for read/write as well. But that's not going to work, right, because only one thread can have write access at a time?

    bug 
    opened by cxrodgers 8
Releases(v0.5.0a0)
  • v0.5.0a0(Jun 2, 2022)

    v0.5.0a0 - ~ The Data Modeling Edition ~

    A lot. Namely a whole rewriting of the {mod}autopilot.data module.

    This is being released as an alpha version while we finish working out a few of the kinks in the data modeling system because Jonny is finishing their dissertation and has a deadline determined by the viscera of institutional science rather than software development!

    Upgrading From <v0.5.0

    • The subject class will attempt to update any of your existing .h5 files to the new format -- it should be nondestructive (making a backup of the whole file first and then making backups up any tables that it can't create a new version of that fully preserves the data from the previous version), but don't be alarmed when your subject files start updating! We advise, as we always do, making a full backup of your data directory before making this update.
    • Change references in launch_autopilot.sh scripts from autopilot.core.terminal or autopilot.core.pilot to autopilot.agents.terminal or autopilot.agents.pilot, respectively
    • Update any TrialData declarations in plugin tasks to use the new {class}.Trial_Data model. See {attr}.Nafc.TrialData for an example. The old pytables IsDescriptor style descriptions will still work, but you will be given a warning every time.
    • We advise updating pigpio to at least 3c23715 to avoid warnings and make use of some new functionality.

    Package Structure

    • The autopilot.core module was removed entirely, and was split into
      • {mod}autopilot.agent: which contains the {class}.Terminal and {class}.Pilot classes and a stub metaclass
      • {mod}autopilot.utils.loggers: formerly autopilot.core.loggers
      • {mod}autopilot.gui (described in module documentation) now has all the GUI modules, split out by type rather than all in one file.
      • {mod}autopilot.data was created from autopilot.core.subject, see below.
    • {mod}autopilot.root contains root classes:
      • {class}.Autopilot_Type - types and data models for handling and saving live data
      • {class}.Autopilot_Object - General root metaclass, beneath which will be top-level metaclasses for each of the major class hierarchies that are currently separate.
      • {class}.Autopilot_Pref - Stub for later converting prefs and environmental configuration parameters from their scattered state.
    • {mod}autopilot.exceptions will be used for more explicit exception handling.
    • Code files were given 755 permissions to allow execution (as opposed to 644)
    • The unused git submodules for pigpio and an old temperature sensor were removed.

    Major updates

    • {mod}autopilot.data is the major change in this release! See the module-level doc page for more details
      • autopilot.core.subject was moved to {mod}autopilot.data.subject and remains the main interface to autopilot data. It was effectively rewritten to accomodate the use of data models, and many of its public methods were made private to make its use more obvious.
        • The subject structure that determines which things are located where has been made into its own class {class}.Subject_Structure
        • Making new subjects is now done with the {meth}.Subject.new class method rather than from the __init__ method
        • The open_hdf method has been replaced with the _h5f private method that is a more explicit context manager allowing locking read/write access and nonlocking read access
        • The subject file now explicitly handles updates between different versions of the Subject class.
        • Subject will check if the protocol .json file has been updated from when it was assigned and automatically update to the new version, logging the change.
        • All trial data is now saved with a session_uuid unique identifier column that is an always-unique key in case of any overlapping session IDs from reassignments, etc.
        • Many attributes were replaced by properties that return data models:
          • {attr}.Subject.bio and {attr}.Subject.info -> {class}.Biography
          • {attr}.Subject.protocol -> {class}.Protocol_Status, which manages the current trial, session, step, etc. rather than being treated separately
      • {mod}autopilot.data.interfaces contains classes to relate abstract data models to different representations and formats
        • {mod}.interfaces.datajoint allows creating datajoint schema definitions from autopilot models using the companion datajoint-babel project
        • {mod}.interfaces.tables translates pydantic models to HDF5 files
      • {mod}autopilot.data.modeling has basic types for use in data models
      • {mod}autopilot.data.models has the models themselves
      • {mod}autopilot.data.units stub module for using explicit units instead of anonymous floats
    • A new set of general model filling widgets - {mod}.widgets.input and {mod}.widgets.model - were created that will eventually replace much of the existing GUI which suffers from code duplication problems from minor variations between representations of parameters, etc.
    • {class}~.loggers.Log, {class}.Log_Format, {class}.LogEntry, {class}.ParseError, {data}.MESSAGE_FORMATS, {data}.LOG_FORMATS were added to allow programmatic loading of logfiles, and {mod}.utils.log_parsers was added to allow recovery of structured data (eg. from logged trial data). Loggers now use the rich logging handler for much more readable logs in stderr.
    • {class}.Jack_Sound classes now put all frames at once into the processing queue, rather than one at a time for much less variability in sound onset jitter. The {class}.JackClient class was updated to reflect that as well by adding a play_q_size argument that controls the size of the deque used to store frames to play (rather than pulling each from a Queue as was done before). {class}.JackClient also has an optional disable_gc (False by default) parameter that allows the garbage collector to be disabled in the process for further (unquantified) jitter reduction.

    Minor updates

    • The bandwidth test ({class}.Bandwidth_Test widget and {meth}.Pilot.l_bandwidth listen) was updated to allow selecting compression, use of a random array, or preserialized messages
    • The {mod}.Terminal now closes cleanly (still with a warning of leaked semaphors) without hanging.
    • External processes {func}.external.start_pigpiod and {func}.external.start_jackd first check if the process is already started
    • {meth}.Digital_Out.set now has a result argument for faster setting that doesn't confirm the result when False
    • {class}.Message s now provide clearer support for compression in serialization, and automatically detect and handle a compressed message when deserializing. This is also carried through to the {meth}.Net_Node.send method.
    • {meth}.Station.l_kill was moved up to the top level {class}.Station class, rather than being something the Terminal station owns.
    • The {mod}~.setup.run_script functions no longer uses the totally unreadable white on green ANSI code combo that I have no idea why i ever used in the first place.
    • A picamera_legacy script was added for enabling the picamera on bullseye.
    • The {class}.sounds.Gammatone sound now accepts a filter_kwargs dictionary that is passed on to the {class}.timeseries.Gammatone filter.
    • The {class}.Task and {class}.Nafc classes use the new {class}.Trial_Data style data declarations.
    • Two utility functions {func}.common.walk_dicts and {func}.common.flatten_dicts were added to make handling nested dictionaries a bit easier.

    Bugfixes

    • The Subject class would incorrectly overwrite data given a mismatch in system times between the Terminal and Pilot. The subject class should now ideally no longer overwrite anything ever.
    • The subject class would drop trial data silently if it was not in the TrialData description. En route to making the table automatically expand to accomodate unexpected data, dropped data is now logged as an exception.
    • The prefs manager handles being launched from within ipython and other processes better, but is still a bit buggy. Now it uses the check used internally in the multiprocessing module to see if a manager can be launched, and falls back to using a standard dictionary if not.
    • jackd_source script uses correct https:// rather than git:// protocol.
    • A _TASK_LIST was added to {mod}.utils.registry to support deprecated task specifications.

    Regressions

    • With additional checking for monotonic increases in trial_num and checks that prevent data overwriting, individual writes of trial data are now a bit slower, which should be optimized for when we complete the transition to uniform data models throughout the library.

    Prefs

    • PIGPIOD - bool - if True, start pigpiod on pilot start
    • Stubs were created for converting the prefs to using data models, but they have not been filled yet.
    • A AUTOPILOT_NO_PREFS_MANAGER environment variable now controls the use of a multiprocessing manager explicit. Documentation for environmental variables is forthcoming.
    • A AUTOPILOT_WARN_DEFAULTS environment variable controls whether warnings should be printed for when a default pref value is retrieved, because that warning is a good one but can be really annoying.

    Packaging & Dependencies

    • Autopilot is now packaged with Poetry! This allows for fully deterministic installation with the poetry.lock file and updates from the old setuptools style of dependency specification.
    • The source repository has moved from https://github.com/wehr-lab/autopilot to https://github.com/auto-pi-lot/autopilot
    • MANIFEST.in has been replaced by the include field in pyproject.toml
    • autopilot.__version__ is now determined by importlib.metadata and specified in the pyproject.toml file rather than in the __init__.py file
    • blosc was replaced with blosc2
    • New dependencies
      • global
        • pydantic (^1.9.0)
        • parse (^1.19.0)
        • rich (^11.2.0)
        • validators (^0.18.2)
      • docs
        • autodoc_pydantic (^1.7.0)
        • myst_parser (^0.17.2)
    • Version Bumps
      • pyzmq 18.1.* -> ^22.3.0
      • tornado >=5.0.0 -> ^6.1.0
      • numpy >=1.16.5 -> ^1.20.0
      • scipy >=1.6.0 -> ^1.7.0
      • pandas >=0.19.2 -> ^1.3.0 on python 3.7 and ^1.4.0 on >=3.8
      • tables >=3.4.2 -> ^3.7.0
      • Sphinx >=3.1.2 -> ^4.3.1
      • A lot more dependencies were taken from being implicit versions to explicit by the conversion to using Poetry...

    Docs

    • Configuration was moved to its own page, documenting setting up the system as well as the contents of the user directory.
    • A faq page was stubbed out (but is still pretty skeletal)
    • The overview was updated with some more information in the module tour
    • Virtual environment usage was moved from the setup page to its own subpage linked from the FAQ.
    • A make serve option was added to the docs makefile that makes use of sphinx-autobuild to livereload docs while editing them.
    • autopilot_theme.css was updated to be compatible with the new version of sphinx-rtd-theme that apparently changed the way that TOC buttons were made, as well as remove incorrect references to fonts no longer packaged.
    • The autodoc_pydantic and myst_parser extensions were added -- and we will be moving towards using MyST rather than hellish ReST for future narrative docs!
    • Private methods and functions are now no longer rendered in the main documentation, and the library will over time use the public/private distinction more systematically to make it more understandable.
    • Examples was split off into its own folder and links to wiki plugins. Blink was moved with it

    Tests

    Source code(tar.gz)
    Source code(zip)
  • v0.4.4(Feb 3, 2022)

    Several parts to this update!

    • See https://github.com/wehr-lab/autopilot/pull/146 for details about improvements to jackd sound timing! In short:
    • Changed the way that continuous sounds work. Rather than cycling through an array, which was easy to drop, now pass a sound object that can generate its own samples on the fly using the hydration module.
    • More accurate timing of sound ending callbacks. Before, the event would be called immediately on buffering the sounds into the jack ports, but that was systematically too early. Instead, use jack timing methods to account for delay from blocksize and n_periods to wait_until a certain delay to set() the event. See _wait_for_end

    Other stuff:

    New

    • hydration module for creating and storing autopilot objects between processes and computers!
    • @Introspect made and added to sound classes. Will be moved to root class. Allows storing the parameters given on instantiation.
    • requires module for more explicit declarations of by-object dependencies to resolve lots of the little fragile checks throughout the package, as well as make it easier for plugins :)
    • minor - added exceptions module, just stubs for now
    • Made dummy sound class to just use sounds without needing a running sound server
    • New transformations! The Prasad line fitting algorithm as Linefit_Prasad and ordering points in a line from, eg. edge detection in Order_Points

    Improvements

    • Only warn once for returning a default pref value, and make its own warning class so that it can be filtered.
    • Cleaning up the base sound classes and moved them to their own module because sounds was very cumbersome and hard to reason about. Now use get_sound_class instead of declaring within the module.
    • Made optional install packages as extras_require so now can install with pip install auto-pi-lot -E pilot rather than autodetecting based on architecture. Further improvements (moving to poetry) will be in v0.5.0

    Bugfixes

    • Correctly identify filenames in logging, before the last module name was treated as a suffix on the path and removed, and so only the most recent logger created would actually log to disk. Logging now works across threads and processes.
    • Fall back to a non-multiprocessing-based prefs if for some reason we can't use a mp.Manager in the given context (eg. ipython) - Still need to figure out a way to not print the exception because it is thrown asynchronously.
    • as much as i love it, the splash screen being absent for whatever reason shouldn't crash the program.
    • Raise an exception when instantiating a picamera without having picamera installed, re: https://github.com/wehr-lab/autopilot/issues/142
    • Raise ImportError when ffmpeg is not present and trying to use a videowriter class
    • Use a deque rather than an infinitely growing list to store GPIO events.

    Docs

    • Documenting the scripts module a bit better.
    • Lots more docs on jack_server
    Source code(tar.gz)
    Source code(zip)
  • v0.4.3(Oct 20, 2021)

    New Features

    • timeseries.Gammatone filter and sounds.Gammatone filtered noise classes! Thank you scipy team for making this simple!

    Minor Improvements

    • https://github.com/wehr-lab/autopilot/commit/579ef1a41518f57721decd7ecfc289f2b358b356b - En route to implementing universal calibrations, load and save them in a specified place for each hardware object instead of the horrific olde way which was built into prefs for some reason
    • prefs attempts to make directories if they don't exist
    • plenty of new debugging flags!

    Bugfixes

    • https://github.com/wehr-lab/autopilot/commit/a775723acc66e327492145066eea0e7dff59331e - Sleep before graduating tasks, lateral fix until we rework the task initiation ritual
    • https://github.com/wehr-lab/autopilot/commit/360062d3ad4cd30cdba1c53eebe5ff7d7698ffad - pad sounds with silence or continuous sounds if they aren't a full period length
    • https://github.com/wehr-lab/autopilot/commit/6614c808ec2f3fc86e01df17b78aa976e8843b5a - Revert to old way of making chunks to make it work with both padded and unpadded sounds
    • Import sounds module directly instead of referring from the package root in tests
    • Terminal node pings pilots instead of an erroneous reference to a nonexistent Terminal.send method
    • https://github.com/wehr-lab/autopilot/commit/47dd4c2345df081fb5f3ca1c1705d2f696fc62c9 - Fix pinging by passing pilot id, and handle pressing start/stop button when subject not selected
    • Fixed some GUI exceptions from trying to make blank lines in reassign window, improperly handling the Subject class.
    Source code(tar.gz)
    Source code(zip)
  • v0.4.2(Aug 25, 2021)

  • v0.4.1(Aug 18, 2021)

    • The :class:autopilot.setup.forms.HARDWARE_FORM would incorrectly use the class object itself rather than the class name in a few places which caused hardware names to incorrectly display and be impossible to add!
    • Correctly handle module name in loggers when running interactively
    • Use accelerometer calibration when computing :meth:~autopilot.hardware.i2c.I2C_9DOF.rotation
    • Use :func:autopilot.get in :func:autopilot.transform.make_transform
    Source code(tar.gz)
    Source code(zip)
  • v0.4.0(Aug 4, 2021)

    See the changelog! https://docs.auto-pi-lot.com/en/latest/changelog/v0.4.0.html

    v0.4.0 - Become Multifarious (August 3rd, 2021)

    This release is primarily to introduce the new plugin system, the autopilot wiki, and their integration as a way of starting the transformation of Autopilot into a tool with decentralized development and governance (as well as make using the tool a whole lot easier and more powerful).

    With humble thanks to Lucas Ott, Tillie Morris, Chris Rodgers <https://github.com/cxrodgers/>, Arne Meyer <https://github.com/arnefmeyer> , Mikkel Roald-Arbøl <https://github.com/roaldarbol>_ , David Robbe <https://github.com/neurodavidus>_ , and an anonymous discussion board poster for being part of this release.

    New Features

    • Registries & Plugins <https://github.com/wehr-lab/autopilot/pull/109>_ - Autopilot now supports users writing their code outside of the library as plugins! To support this, a registry system was implemented throughout the program. Plugin objects can be developed as objects that inherit from the Autopilot object tree -- eg. implementing a GPIO object by subclassing :class:.hardware.gpio.GPIO , or a new task by subclassing :class:~.tasks.task.Task . This system is flexible enough to allow any lineage of objects to be included as a plugin -- stimuli, tasks, and so on -- and we will be working to expand registries to every object in Autopilot, including the ability for plugins to replace core modules to make Autopilot's flexibility verge on ludicrous. The basic syntax of the registry system is simple and doesn't require any additional logic beyond inheritance to be implemented on plugin objects -- autopilot.get('object_type', 'object_name') is the basic method, with a few aliases for specific object types like autopilot.get_hardware(). Also thanks to Arne Meyer <https://github.com/arnefmeyer>_ for submitting an early draft of the registry system and Mikkel Roald-Arbøl <https://github.com/roaldarbol>_ for raising the issue.
    • At long last, the Autopilot Wiki is alive!!!! - https://wiki.auto-pi-lot.com/ - The wiki is the place for communal preservation of technical knowledge about using Autopilot, like hardware designs, build guides, parameter sets, and beyond! This isn't any ordinary wiki, though, we got ourselves a semantic wiki which augments traditional wikis with a rich system of human and computer-readable linked attributes: a particular type of page will have some set of attributes, like a page about a 3D printed part will have an associated .stl file, but rather than having these be in plaintext they are specified in a format that is queryable, extensible, and infinitely mutable. The vision for the wiki is much grander (but not speculative! very concrete!) than just a place to take notes, but is intended to blend the use of Autopilot as an experimental tool with body of knowledge that supports it. Autopilot can query the wiki with the wiki module like wiki.ask('[[Category:3D_CAD]]', 'Has STL') to get links to all .stl files for all 3D parts on the wiki. The integration between the two makes using and submitting information trivial, but also makes designing whole new types of community interfaces completely trivial. As a first pass, the Wiki will be the place to index plugins, the system for submitting them, querying them, and downloading them only took a few hours and few dozen lines of code to implement. The wiki is infinitely malleable -- that's the point -- and I am very excited to see how people use it.
    • Tests & Continuous Integration with Travis! We are on the board with having nonzero tests! The travis page is here: https://travis-ci.com/github/wehr-lab/autopilot and the coveralls page is here: https://coveralls.io/github/wehr-lab/autopilot . At the moment we have a whopping 27% coverage, but as we build out our testing suite we hope that it will become much easier for people to contribute to Autopilot and be confident that it works!
    • New Hardware Objects
      • :class:.cameras.PiCamera - A fast interface to the PiCamera, wrapping the picamera library, and using tips from its developer to juice every bit of speed i could!
      • The I2C_9DOF object was massively improved to take better advantage of its onboard DSP and expose more of its i2c commands.
    • New Transforms
      • :class:.timeseries.Kalman - adapted a Kalman filter from the wonderful filterpy package! it's in the new timeseries transform module
      • :class:.geometry.IMU_Orientation - IMU_Orientation performs a sensor fusion algorithm with the Kalman Filter class to combine gyroscope and accelerometer measurements into a better estimate of earth-centric roll and pitch. This is used by the IMU class, but is made independent so it can be used without an Autopilot hardware object/post-facto/etc.
      • :class:.timeseries.Filter_IIR - Filter_IIR implements scipy's IIR filter as a transform object.
      • :class:.timeseries.Integrate - Integrate adds successive numbers together (scaled by dt if requested). not much by itself, but when used with a kalman filter very useful :)
      • :class:.geometry.Rotate - use scipy to rotate a vector by some angle in x, y, and/or z
      • :class:.geometry.Spheroid - fit and transform 3d coordinates according to some spheroid - used in the IMU's accelerometer calibration method: given some target spheroid, and some deformed spheroid (eg. a miscalibrated accelerometer might have the x, y, or z axis scaled or offset) either explicitly set or estimated from a series of point measurements, transform future input given that transformation to correct for the deformed source spheroid.
    • New Prefs
      • 'AUTOPLUGIN' - Attempt to import the contents of the plugin directory,
      • 'PLUGIN_DB' - filename to use for the .json plugin_db that keeps track of installed plugins',
      • 'PING_INTERVAL' - How many seconds should pilots wait in between pinging the Terminal?',
      • 'TERMINAL_SETTINGS_FN' - filename to store QSettings file for Terminal',
      • 'TERMINAL_WINSIZE_BEHAVIOR' - Strategy for resizing terminal window on opening',
      • 'TERMINAL_CUSTOM_SIZE' - Custom size for window, specified as [px from left, px from top, width, height]',

    Major Improvements

    • Stereo Sound (Thank you Chris Rodgers! <https://github.com/cxrodgers/>_) - https://github.com/wehr-lab/autopilot/pull/102
    • Multihop messages & direct messaging - https://github.com/wehr-lab/autopilot/pull/99 - it is now possible to send multihop messages through multiple Station objects, as well as easier to send messages directly between net nodes. See the examples in the network tests section of the docs.
    • Multiple Children (Thank you Chris Rodgers! <https://github.com/cxrodgers/>_) - https://github.com/wehr-lab/autopilot/pull/103 - the CHILDID field now accepts a list, allowing a Pilot to initialize child tasks on multiple children. (this syntax and the hierarchical nature of pilots and children will be deprecated as we refactor the networking modules into a general mesh system, but this is lovely to have for now :)
    • Programmatic Setup - https://github.com/wehr-lab/autopilot/issues/33 - noninteractive setup of prefs and scripts by using autopilot.setup -f prefs.json -p PREFNAME=VALUE -s scriptname1 -s scriptname2
    • Widget to stream video, en route to more widgets for direct GUI control of hardware objects connected to pilots
    • Support python 3.8 and 3.9 essentially by not insisting that the spinnaker SDK be installable by all users (which at the time was only available for 3.7)

    Minor Improvements

    • Terminal can be opened maximized, or have its size and position set explicitly, preserve between launches (Thank you Chris Rodgers! <https://github.com/cxrodgers/>_) - https://github.com/wehr-lab/autopilot/pull/70
    • Pilots will periodically ping the Terminal again, Terminal can manually ping Pilots that may have gone silent - https://github.com/wehr-lab/autopilot/pull/91
    • Pilots share their prefs with the Terminal in their initial handshake - https://github.com/wehr-lab/autopilot/pull/91
    • Reintroduce router ports for net-nodes to allow them to bind a port to receive messages - https://github.com/wehr-lab/autopilot/pull/115/commits/35be5d634d98a7983ec3d3d6c5b94da6965a2579
    • Listen methods are now optional for net_nodes
    • Allowed the creation of dataless tasks - https://github.com/wehr-lab/autopilot/pull/115/commits/628e1fb9c8fcd15399b19b351fed87e4826bc9ab
    • Allowed the creation of plotless tasks - https://github.com/wehr-lab/autopilot/pull/115/commits/08d99d55a32b45f54e3853813c7c71ea230b25dc
    • The I2C_9DOF clas uses memoryviews rather than buffers for a small performance boost - https://github.com/wehr-lab/autopilot/pull/115/commits/890f2c500df8010b50d61f64e2755cd2c7a8aeed
    • Phasing out using Queue s in favor of collections.deque for applications that only need thread and not process safety because they are way faster and what we wanted in the first place anyway.
    • New Scripts - i2c, picamera, env_terminal
    • utils.NumpyEncoder and decoder to allow numpy arrays to be json serialized
    • calibrations are now loaded by hardware objects themselves instead of the extraordinarily convoluted system in prefs -- though some zombie code still remains there.
    • Net nodes know their ip now, but this is a lateral improvement pending a reworking of the networking modules.
    • performance script now sets swappiness = 10 to discourage the use of swapfiles - see https://www.raspberrypi.org/forums/viewtopic.php?t=198765
    • Setting a string in the deprecation field of a pref in _DEFAULTS prints it as a warning to start actually deprecating responsibly.
    • Logging in more places like Subject creation, manipulation, protocol assignation.

    Bugfixes

    • Loggers would only work for the last object that was instantiated, which was really embarassing. fixed - https://github.com/wehr-lab/autopilot/pull/91
    • Graduation criteria were calculated incorrectly when subjects were demoted in stages of a protocol - https://github.com/wehr-lab/autopilot/pull/91
    • fix durations in solenoid class (Thank you Chris Rodgers! <https://github.com/cxrodgers/>_) - https://github.com/wehr-lab/autopilot/pull/63
    • LED_RGB ignores zero - https://github.com/wehr-lab/autopilot/pull/98
    • Fix batch assignment window crashing when there are subjects that are unassigned to a task - https://github.com/wehr-lab/autopilot/pull/115/commits/e42fc5802792822ff5a53a2379041a4a8b301e9e
    • Catch malformed protocols in batch assignment widget - https://github.com/wehr-lab/autopilot/pull/115/commits/2cc8508a4bf3a6d49512197dc72433c60d0c656e
    • Remove broken Terminal.reset_ui method and made control panel better at adding/removing pilots - https://github.com/wehr-lab/autopilot/pull/91
    • Subject class handles unexpected state a lot better (eg. no task assigned, no step assigned, tasks with no data.) but is still an absolute travesty that needs to be refactored badly.
    • The jackclient would crash with long-running continuous sounds as the thread feeding it samples eventually hiccuped. Made more robust by having jackclient store samples locally int he sound server rather than being continuously streamed from the queue.
    • PySide2 references still incorrectly used QtGui rather than QtWidgets
    • pigpio scripts would not be stopped and removed when a task was stopped, the :func:.gpio.clear_scripts function now handles that.
    • xcb was removed from PySide2 distributions, so it's now listed in the requirements for the Terminal and made available in the env_terminal script.
    • LED_RGB didn't appropriately raise a ValueError when called with a single pin - https://github.com/wehr-lab/autopilot/issues/117
    • A fistful of lingering Python 2 artifacts

    Code Structure

    • continuing to split out modules in :mod:autopilot.core - networking this time
    • utils is now a separate module instead of being in multiple places
    • the npyscreen forms in setup_autopilot were moved to a separate module
    • setup_autopilot was broken into functions instead of a very long and impenetrable script. still a bit of cleaning to do there.
    • autopilot.setup.setup_autopilot was always extremely awkward, so it's now been aliased as autopilot.setup
    • the docs have now been split into subfolders rather than period separated names to make urls nicer -- eg /dev/hardware/cameras.htm rather than /dev/hardware.cameras.html . this should break some links when switching between versions on readthedocs but other than that be nondestructive.

    Docs

    • new :ref:quickstart documentation with lots of quick examples!

    Regressions

    • Removed the check_compatible method in the Transforms class. We will want to make a call at some point if we want to implement a full realtime pipelining framework or if we want to use something like luigi or joblib or etc. for now this is an admission that type and shape checking was never really implemented but it does raise some exceptions sometimes.
    Source code(tar.gz)
    Source code(zip)
  • v0.3.5(Feb 23, 2021)

    https://github.com/wehr-lab/autopilot/pull/79

    A very minor bugfix in response to https://github.com/wehr-lab/autopilot/discussions/72#discussioncomment-386330

    • Very minor one, fixes to the way :class:.Terminal accesses the pilot_db.json file to use :attr:.Terminal.pilots property that makes a new pilot_db.json file if one doesn't exist, but otherwise loads the one that is found in prefs.get('PILOT_DB')
    • Reorganized :class:.Terminal source to group properties together & minor additions of type hinting
    • Fixed some bad fallback behavior looking for files in old hardcoded default directories, eg. in the ye olde :func:.utils.get_pilotdb
    • bump version to 0.3.5
    Source code(tar.gz)
    Source code(zip)
  • v0.3.4(Dec 14, 2020)

    Improvements:

    • Unify the creation of loggers!!!! See the docs ;) https://docs.auto-pi-lot.com/en/latest/autopilot.core.loggers.html : https://github.com/wehr-lab/autopilot/pull/52/commits/d55638f985ab38044fc95ffeff5945021c2e198e https://github.com/wehr-lab/autopilot/issues/38
    • Unify prefs, including sensible defaults, refactoring of scripts into a reasonable format, multiprocess-safety, and just generally a big weight off my mind. Note that this is a breaking change to the way prefs are accessed. Previously one would do prefs.PREF_NAME, but that made it very difficult to provide default values or handle missing prefs. the new syntax is prefs.get('PREF_NAME') which returns defaults with a warning and None if the pref is not set: https://github.com/wehr-lab/autopilot/pull/52/commits/c40a212bcaf5f184f2a6a606027fe15b1b4df59c https://github.com/wehr-lab/autopilot/issues/38
    • completely clean up scripts, and together that opened the path to clean up setup as well. so all things configuration got a major promotion
    • We're on the board with CI and automated testing with a positively massive 3% code coverage!!! https://github.com/wehr-lab/autopilot/pull/52/commits/743bb8fe67a69fcc556fa76e81f72f97f510dff7
    • new scripts to eg. create autopilot alias: https://github.com/wehr-lab/autopilot/pull/52/commits/211919b05922e18a85d8ef6216973f4000fd32c5

    Bugfixes:

    • cleanup scripts on object deletion: https://github.com/wehr-lab/autopilot/pull/52/commits/e8218304bd7ef2e13d2adfc236f3e781abea5f78 https://github.com/wehr-lab/autopilot/issues/41
    • don't drop 'floats' from gui when we say we can use them...: https://github.com/wehr-lab/autopilot/pull/52/commits/743bb8fe67a69fcc556fa76e81f72f97f510dff7
    • pigpio scripts dont like floats: https://github.com/wehr-lab/autopilot/pull/52/commits/9f939cd78a5296db3bf318115bee0213bcd1afc0

    Docs:

    • Clarification of supported systems: https://github.com/wehr-lab/autopilot/pull/52/commits/ce0ddf78b7f59f5487fec2ca7e8fb3c0ad162051
    • Solved an ancient sphinx riddle of how to get data objects/constants to pretty-print: https://github.com/wehr-lab/autopilot/pull/52/commits/ec6d5a75dada05688b6bd3c1a53b3d9e5923870f
    • Clarify hardware prefs https://github.com/wehr-lab/autopilot/pull/52/commits/f3a7609995c84848004891a0f41c7847cb754aae
    • what numbering system do we use: https://github.com/wehr-lab/autopilot/pull/52/commits/64267249d7b1ec1040b522308cd60f928f2b2ee6

    Logging:

    • catch pigpio script init exception: https://github.com/wehr-lab/autopilot/pull/52/commits/3743f8abde7bbd3ed7766bdd75aee52afedf47e2
    • more of it idk https://github.com/wehr-lab/autopilot/pull/52/commits/b682d088dbad0f206c3630543e96a5a00ceabe25
    Source code(tar.gz)
    Source code(zip)
  • v0.3.3(Oct 25, 2020)

    Bugfixes

    • Fix layout in batch reassign from python 3 float division
    • Cleaner close by catching KeyboardInterrupt in networking modules
    • Fixing audioserver boot options -- if 'AUDIOSERVER' is set even if 'AUDIO' isn't set in prefs, should still start server. Not full fixed, need to make single plugin handler, single point of enabling/disabling optional services like audio server
    • Fix conflict between polarity and pull in initializing pulls in pilot
    • Catch tables.HDF5ExtError if local .h5 file corrupt in pilot
    • For some reason 'fs' wasn't being replaced in the jackd string, reinstated.
    • Fix comparison in LED_RGB that caused '0' to turn on full becuse 'value' was being checked for its truth value (0 is false) rather than checking if value is None.
    • obj.next() to next(obj) in jackdserver

    Improvements

    • Better internal handling of pigpiod -- you're now able to import and use hardware modules without needing to explicitly start pigpiod!!
    • Hopefully better killing of processes on exit, though still should work into unified process manager so don't need to reimplement everything (eg. as is done with launching pigpiod and jackd)
    • Environment scripts have been split out into setup/scripts.py and you can now run them with python -m autopilot.setup.run_script (use --help to see how!)
    • Informative error when setup is run with too narrow terminal: https://github.com/wehr-lab/autopilot/issues/23
    • More loggers, but increased need to unify logger creation!!!

    Cleanup

    • remove unused imports in main __init__.py that made cyclical imports happen more frequently than necessary
    • single-sourcing version number from __init__.py
    • more cleanup of unnecessary meta and header stuff left from early days
    • more debugging flags
    • filter NaturalNameWarning from pytables
    • quieter cleanups for hardware objects
    Source code(tar.gz)
    Source code(zip)
  • v0.3.2(Sep 29, 2020)

    bugfixes

    • https://github.com/wehr-lab/autopilot/issues/19 - previously, I attempted to package binaries for the lightly modified pigpio and for jackd (the apt binary used to not work), but after realizing that was the worst possible way of going about it I changed install strategies, but didn't entirely remove the vestiges of the prior attempt. The installation expected certain directories to exist (in autopilot/external) that didn't, which crashed and choked install. Still need to formalize a configuration and plugin system, but getting there.
    • https://github.com/wehr-lab/autopilot/issues/20 - the jackd binary in the apt repos for the raspi used to not work, so i was in the habit of compiling jackd audio from source. I had build that into the install routine, but something about that now causes the JACK-Client python interface to throw segfaults. Somewhere along the line someone fixed the apt repo version of jackd so we use that now.
    • previously I had only tested in a virtual environment, but now the installation routine properly handles not being in a venv.

    cleanup

    • remove bulky static files like fonts and css from /docs/ where they were never needed and god knows how they got there
    • use a forked sphinx-sass when building docs that doesn't specify a required sphinx version (which breaks sphinx)
    • removed skbuild requirements from install
    • fixed pigpio install requirement in requirements_pilot.txt
    • included various previously missed files in MANIFEST.in
    • added installation of system libraries to the pilot configuration menu
    Source code(tar.gz)
    Source code(zip)
  • v0.3.0(Aug 5, 2020)

    Autopilot is growing, it is learning to build strong APIs and has gained a lot of new features along the way :)

    • Autopilot has moved to Python 3!! (Tested on 3.6-3.8)
    • Capturing video with OpenCV and the Spinnaker SDK is now supported (See :mod:autopilot.hardware.cameras)
    • An :class:~hardware.i2c.I2C_9DOF motion sensor and the :class:~hardware.i2c.MLX90640 temperature sensor are now supported.
    • Timestamps from GPIO events are now microsecond-precise thanks to some modifications to the pigpio library
    • GPIO output timing is also microsecond-precise thanks to the use of pigpio scripts, so you can deliver exactly the reward volumes you intend <3
    • Hardware modules have been refactored into their own module, and have been almost wholly rebuilt to have sensible inheritance structure.
    • Networking modules are more efficient and automatically compress arrays (like video frames!) on transmission. Streaming is also easier now, check out :meth:.Net_Node.get_stream !
    • We now have a detailed :ref:development roadmap <todo> , so you can see the magnificent future we have planned.
    • We have created the autopilot-users <https://groups.google.com/forum/#!forum/autopilot-users>_ discussion board for troubleshooting & coordinating community development :)
    Source code(tar.gz)
    Source code(zip)
Unified Distributed Execution

Unified Distributed Execution The framework supports multiple execution backends: Ray, Dask, MPI and MultiProcessing. To run tests you need to install

null 17 Dec 25, 2022
Demo code for "Logs in distributed systems" webinar

Hexlet Logs Demo Пререквизиты docker-compose python3 Учетка в DataDog Базовое понимание, что такое логи (можно почитать гайд

Anton Markelov 1 Dec 1, 2021
PsychoPy is an open-source package for creating experiments in behavioral science.

PsychoPy is an open-source package for creating experiments in behavioral science. It aims to provide a single package that is: precise enoug

PsychoPy 1.3k Dec 31, 2022
Retentioneering 581 Jan 7, 2023
Behavioral Testing of Clinical NLP Models

Behavioral Testing of Clinical NLP Models This repository contains code for testing the behavior of clinical prediction models based on patient letter

Betty van Aken 2 Sep 20, 2022
Python package for analyzing behavioral data for Brain Observatory: Visual Behavior

Allen Institute Visual Behavior Analysis package This repository contains code for analyzing behavioral data from the Allen Brain Observatory: Visual

Allen Institute 16 Nov 4, 2022
XGBoost-Ray is a distributed backend for XGBoost, built on top of distributed computing framework Ray.

XGBoost-Ray is a distributed backend for XGBoost, built on top of distributed computing framework Ray.

null 92 Dec 14, 2022
Distributed-systems-algos - Distributed Systems Algorithms For Python

Distributed Systems Algorithms ISIS algorithm In an asynchronous system that kee

Tony Joo 2 Nov 30, 2022
SciKit-Learn Laboratory (SKLL) makes it easy to run machine learning experiments.

SciKit-Learn Laboratory This Python package provides command-line utilities to make it easier to run machine learning experiments with scikit-learn. O

ETS 528 Nov 25, 2022
Applications using the GTN library and code to reproduce experiments in "Differentiable Weighted Finite-State Transducers"

gtn_applications An applications library using GTN. Current examples include: Offline handwriting recognition Automatic speech recognition Installing

Facebook Research 68 Dec 29, 2022
Simple reimplemetation experiments about FcaNet

FcaNet-CIFAR An implementation of the paper FcaNet: Frequency Channel Attention Networks on CIFAR10/CIFAR100 dataset. how to run Code: python Cifar.py

null 76 Feb 4, 2021
ReproZip is a tool that simplifies the process of creating reproducible experiments from command-line executions, a frequently-used common denominator in computational science.

ReproZip ReproZip is a tool aimed at simplifying the process of creating reproducible experiments from command-line executions, a frequently-used comm

null 267 Jan 1, 2023
L2X - Code for replicating the experiments in the paper Learning to Explain: An Information-Theoretic Perspective on Model Interpretation.

L2X Code for replicating the experiments in the paper Learning to Explain: An Information-Theoretic Perspective on Model Interpretation at ICML 2018,

Jianbo Chen 113 Sep 6, 2022
Image augmentation for machine learning experiments.

imgaug This python library helps you with augmenting images for your machine learning projects. It converts a set of input images into a new, much lar

Alexander Jung 13.2k Jan 2, 2023
Code to reproduce the experiments in the paper "Transformer Based Multi-Source Domain Adaptation" (EMNLP 2020)

Transformer Based Multi-Source Domain Adaptation Dustin Wright and Isabelle Augenstein To appear in EMNLP 2020. Read the preprint: https://arxiv.org/a

CopeNLU 36 Dec 5, 2022
Algorithmic trading with deep learning experiments

Deep-Trading Algorithmic trading with deep learning experiments. Now released part one - simple time series forecasting. I plan to implement more soph

Alex Honchar 1.4k Jan 2, 2023
Code to run experiments in SLOE: A Faster Method for Statistical Inference in High-Dimensional Logistic Regression.

Code to run experiments in SLOE: A Faster Method for Statistical Inference in High-Dimensional Logistic Regression. Not an official Google product. Me

Google Research 27 Dec 12, 2022
This repository contains the implementations related to the experiments of a set of publicly available datasets that are used in the time series forecasting research space.

TSForecasting This repository contains the implementations related to the experiments of a set of publicly available datasets that are used in the tim

Rakshitha Godahewa 80 Dec 30, 2022
Implementation of experiments in the paper Clockwork Variational Autoencoders (project website) using JAX and Flax

Clockwork VAEs in JAX/Flax Implementation of experiments in the paper Clockwork Variational Autoencoders (project website) using JAX and Flax, ported

Julius Kunze 26 Oct 5, 2022
Neural implicit reconstruction experiments for the Vector Neuron paper

Neural Implicit Reconstruction with Vector Neurons This repository contains code for the neural implicit reconstruction experiments in the paper Vecto

Congyue Deng 35 Jan 2, 2023