Python tools for querying and manipulating BIDS datasets.

Overview

PyBIDS

DOI DOI

Build Status (Linux and OS X) Binder PyPI version

PyBIDS is a Python library to centralize interactions with datasets conforming BIDS (Brain Imaging Data Structure) format. For more information about BIDS visit http://bids.neuroimaging.io.

PyBIDS currently supports Python 3 on POSIX operating systems (including Mac OS). Windows is not officially supported, though most PyBIDS functionality will probably work fine.

Installation

PyBIDS is most easily installed from pip. To install the latest official release:

pip install pybids

If you want to live on the bleeding edge, you can install from master:

pip install git+https://github.com/bids-standard/pybids.git

Dependencies

PyBIDS has a number of dependencies. The core querying functionality requires only the BIDS-Validator package. However, most other modules require the core Python neuroimaging stack: numpy, scipy, pandas, and nibabel. The reports module additionally requires num2words. By default, all dependencies will be installed with pybids (if they aren't already available).

Usage

Get started by checking out the documentation!

Or you can start at our tutorial! You can run it interactively without installing anything via binder. Click on the link and then navigate to examples/pybids_tutorial.ipynb to explore.

How to cite

To credit PyBIDS in your work, please cite both the JOSS paper and the Zenodo archive. The former provides a high level description of the package, and the latter points to a permanent record of all PyBIDS versions (we encourage you to cite the specific version you used). Example citations (for PyBIDS 0.9.3):

Yarkoni et al., (2019). PyBIDS: Python tools for BIDS datasets. Journal of Open Source Software, 4(40), 1294, https://doi.org/10.21105/joss.01294

Yarkoni, Tal, Markiewicz, Christopher J., de la Vega, Alejandro, Gorgolewski, Krzysztof J., Halchenko, Yaroslav O., Salo, Taylor, … Blair, Ross. (2019, August 8). bids-standard/pybids: 0.9.3 (Version 0.9.3). Zenodo. http://doi.org/10.5281/zenodo.3363985

Community guidelines

Bug reports, feedback, and other contributions to PyBIDS are welcome. To report a bug, request a feature, or raise a technical (non-usage-related) question for discussion, please open an issue on GitHub. Requests for support and questions about usage (where no bug is suspected) should be posted on the NeuroStars forum. For questions about the BIDS specification or BIDS-Validator, please see the corresponding repositories. For general BIDS-related discussion, see the bids-discussion mailing list.

Pull requests are always welcome. If you have any doubts about the suitability of a PR, or want to discuss implementation-related issues, feel free to open an issue for discussion.

Comments
  • get_collections fails when meta-data includes lists

    get_collections fails when meta-data includes lists

    For me this error occurs when:

    bids_analysis = Analysis(bids_layout, model)
    bids_analysis.setup(**entities)
    

    TypeError: unhashable type: 'list'

    Looking at the traceback:

    collections = self.layout.get_collections(self.level, drop_na=drop_na, **kwargs)
    

    seems to be what's failing because in:

    > /celery_worker/src/pybids/bids/variables/entities.py(161)get_nodes()
        159         sort_cols = [sc for sc in sort_cols if sc in set(rows.columns)]
        160         sort_cols += list(set(rows.columns) - set(sort_cols))
    --> 161         rows = rows.sort_values(sort_cols)
        162         inds = rows['node_index'].astype(int)
        163         return [self.nodes[i] for i in inds]
    

    In this dataaset, rows contains some values that are lists because of structured meta-data.

    For example:

    rows['ImageType'] 
    0    [ORIGINAL, PRIMARY, FMRI, NONE, ND, NORM, MOSA]
    Name: ImageType, dtype: object
    

    I can't really make sense of why that requires sorting (if its not being returned), and if only node_index is being accesed. But either way, I think we need to add meta-data with lists to the tests, as that's probably why we didn't catch this.

    bug 
    opened by adelavega 38
  • REFACTOR: 0.8 [WIP]

    REFACTOR: 0.8 [WIP]

    This is a near-total refactoring of the core BIDSLayout object. The main point of this exercise is to increase maintainability (I don't want to be in charge of this forever) by removing the grabbit dependency and increasing code clarity. With a few very minor exceptions, the user-facing API remains unperturbed (as evidence of that, the tests for most modules passed with virtually no modification). Two side benefits of the overhaul are that (a) indexing should be substantially more efficient, and (b) there's a new (currently undocumented) object-oriented API that allows one to walk down a BIDS project (each BIDSLayout has a root_node attribute, which is a BIDSNode object that has helpful attributes like .children, .parent, .path, entities, etc.).

    API changes: I believe the only common case where this PR breaks the public 0.7 API is in the handling of derivatives in get() calls. Previously, get took a derivatives argument that indicates whether to search derivatives. This argument has been replaced with a new scope argument. The behavior of scope is quite similar (see docstring), but I think a name change is in order because the scope argument now allows one to scan only derivatives folders, which was previously impossible.

    There are quite a few changes that could potentially break things for people who were already reaching into the internals. Most notably, _get_nearest_helper is gone. If you were using this, you need to switch to get_nearest, and you'll need to update the calling arguments (see examples throughout the codebase, but most likely you need to set all_=True and possibly pass ignore_strict_entities). Also, the Domain class is gone. Every Entity must now be unique across the entire BIDSLayout and any derivative layouts. This has some downsides, but I decided we could live with them, as it simplifies the code quite a bit. The upshot is that if the word 'domain' occurs anywhere in a workflow, you'll probably need to make minor changes to get things to work again (in most function calls, you may just be able to delete the domain argument without any problems).

    Beyond that, it's hard for me to anticipate where else changes with downstream impacts might have happened, so @effigies and @adelavega, maybe try running with this branch and let me know if anything breaks in a way that isn't trivially fixable. Keep in mind that I'm still porting over peripheral functionality from grabbit.

    Remaining items:

    • [x] Add back all functionality from the old Layout (currently the BIDSLayout is stripped down to bare essentials)
    • [x] Port the extensions in grabbit (e.g., file-writing and path-building)
    • [x] Port grabbit tests to pybids (basically, all functionality that isn't currently covered by the pybids tests)
    • [x] Add docstrings and update docs to reflect the changes
    • [x] Add back support for include and exclude directives to allow file filtration at a more fine-grained level than entire derivatives pipelines.

    Closes #350. Closes #285.

    opened by tyarkoni 36
  • preparation for JOSS submission

    preparation for JOSS submission

    Adds files needed for submission to JOSS (see #394). All: please look over paper.md and feel free to provide comments/edits. Keep in mind that this is meant to be "a summary describing the high-level functionality and purpose of the software for a diverse, non-specialist audience" (see author guidelines); it is not a tutorial on how to use the software. It's supposed to be 250 - 1,000 words (we're currently around 600).

    I'll give this a few days, then will probably submit early next week. Please provide comments before then.

    opened by tyarkoni 32
  • WIP: Refactoring of analysis module

    WIP: Refactoring of analysis module

    This is a complete refactoring of the analysis module. Nearly all of the change are internal, so I won't go into detail, but I'll summarize some of the key changes, plus the (fairly small) changes in the API:

    • The analysis module has been split up into separate variables and analysis modules. variables contains functionality for working with variables read from BIDSLayout objects. No analysis is implied by anything here; it's just a set of tools for loading, representing, and accessing variables from BIDS projects.
    • The variables module is structured into entities, variables, collections, and io submodules:
      • entities contains classes that try to mirror the BIDS structure relatively closely. Basically, it's a big tree that begins with a single Dataset node and then branches into Subject, Session, and Run levels. Users will rarely if ever have any reason to use anything inside entities directly.
      • variables contains classes that represent individual variables.
      • collections contains classes that represent collections of variables.
      • io contains a single load_variables function (plus some private handlers) that is the workhorse for extracting variables from a BIDSLayout. However, it doesn't need to be called directly very often, because...
    • The BIDSLayout class now implements a get_collections class method that returns BIDSVariableCollections containing all kinds of variable goodies. For most users, interactions with variables can probably be accomplished exclusively through this function.
    • There are actually some tests this time. Not a lot, but some. This is progress.
    • The Analysis API has been simultaneously simplified and made more flexible. The minimum needed to run an analysis is now:
    layout = BIDSLayout('./ds000114')
    analysis = Analysis(layout, 'model.json')
    analysis.setup(task='fingerfootlips')
    
    # Then we can do things like...
    dm = analysis[0].get_design_matrix(format='wide', entities=False, subject='01', run=1)[0]
    cm = analysis[0].get_contrast_matrix(subject='01', run=1)[0]
    

    The problems with generating contrast matrices are now solved; the returned contrast matrix should have contrasts in rows and images in columns, with appropriate coding.

    • The Analysis interface is also now more efficient in some ways. In particular, variable loading is now done on a per-node basis, so that passing **selector constraints won't require loading of all variables. This means that the following works efficiently now:
    layout = BIDSLayout('./ds000114')
    analysis = Analysis(layout, 'model.json')
    analysis.setup(task='fingerfootlips', subject='01')
    ...
    

    Because variable loading is now deferred to the Block-level setup, the above snippet would only read variables for subject '01'. This makes it efficient to loop over subjects, sessions, runs, etc., which is probably a more natural way to interact with the Analysis object in many cases.

    That all said, this isn't quite finished. Things left to do (I should be able to get to these on Wednesday):

    • [x] More testing (@effigies, please flag problems you find)
    • [x] I haven't tested reading from scans.tsv, sessions.tsv, or participants.tsv at all. This will be much easier with the new synthetic dataset.
    • [x] Propagating the results of one Block onto the next isn't working yet. This should be a quick fix though. For now, the first level should work fine, but the rest won't.
    • [x] Cleanup and harmonization of the entire codebase, which probably has some jagged edges after this major refactoring (again, @effigies, feel free to take a pass through)
    • [ ] Reimplement aggregation of Collections into a higher level.
    • [ ] Merging Collections at different levels doesn't work yet. This is not a huge deal since it's not necessary for standard summary statistics-style workflows. It will be necessary however for more complex analyses (e.g., fitting one big model for all subjects at once).
    opened by tyarkoni 30
  • adds get_nearest() functionality via new grabbit version

    adds get_nearest() functionality via new grabbit version

    This replaces much of the code in get_metadata() with a cleaner approach that leverages the latest version of grabbit. Also provides a generic get_nearest() method that can be easily used to walk up the hierarchy and get the appropriate file for any given input file. E.g.,

    layout = BIDSLayout(path)
    event_file = layout.get_nearest('path/to/sub-03_ses-1_run-1_bold.nii.gz', type='event', all_=True)
    

    ...would return all matching event files in the hierarchy for the given functional.

    opened by tyarkoni 30
  • Allow wildcard names for model variables

    Allow wildcard names for model variables

    I'm trying to pass variable names to a bids model with wildcards like so:

    "Model": { "X": [ 'stimulus_type.neutral_anticipation', 'stimulus_type.neutral_feedback', 'stimulus_type.loss_anticipation', 'stimulus_type.avoided_loss_feedback', 'stimulus_type.gain_anticipation', 'stimulus_type.gain_feedback', 'stimulus_type.missed_gain_feedback', 'stimulus_type.loss_feedback', 'trans_', 'rot_', 'cosine*', 'a_comp_cor_', 't_comp_cor_', 'aroma_motion_', 'censor_' ], },

    This is throwing a key error trans_*

    Traceback (most recent call last):
      File "/data/nielsond/python/envs/fitlins_dev/lib/python3.6/site-packages/nipype/pipeline/plugins/multiproc.py", line 69, in run_node
        result['result'] = node.run(updatehash=updatehash)
      File "/data/nielsond/python/envs/fitlins_dev/lib/python3.6/site-packages/nipype/pipeline/engine/nodes.py", line 473, in run
        result = self._run_interface(execute=True)
      File "/data/nielsond/python/envs/fitlins_dev/lib/python3.6/site-packages/nipype/pipeline/engine/nodes.py", line 564, in _run_interface
        return self._run_command(execute)
      File "/data/nielsond/python/envs/fitlins_dev/lib/python3.6/site-packages/nipype/pipeline/engine/nodes.py", line 649, in _run_command
        result = self._interface.run(cwd=outdir)
      File "/data/nielsond/python/envs/fitlins_dev/lib/python3.6/site-packages/nipype/interfaces/base/core.py", line 376, in run
        runtime = self._run_interface(runtime)
      File "/gpfs/gsfs11/users/MBDU/midla/notebooks/code/fitlins/fitlins/interfaces/bids.py", line 235, in _run_interface
        analysis.setup(drop_na=False, desc='preproc', **selectors)
      File "/gpfs/gsfs11/users/MBDU/midla/notebooks/code/pybids/bids/analysis/analysis.py", line 89, in setup
        b.setup(input_nodes, drop_na=drop_na, **selectors)
      File "/gpfs/gsfs11/users/MBDU/midla/notebooks/code/pybids/bids/analysis/analysis.py", line 213, in setup
        transform.Select(coll, X)
      File "/gpfs/gsfs11/users/MBDU/midla/notebooks/code/pybids/bids/analysis/transformations/base.py", line 87, in __new__
        return t.transform()
      File "/gpfs/gsfs11/users/MBDU/midla/notebooks/code/pybids/bids/analysis/transformations/base.py", line 223, in transform
        self._clone_variables()
      File "/gpfs/gsfs11/users/MBDU/midla/notebooks/code/pybids/bids/analysis/transformations/base.py", line 123, in _clone_variables
        for v in self.variables}
      File "/gpfs/gsfs11/users/MBDU/midla/notebooks/code/pybids/bids/analysis/transformations/base.py", line 123, in <dictcomp>
        for v in self.variables}
      File "/gpfs/gsfs11/users/MBDU/midla/notebooks/code/pybids/bids/variables/kollekshuns.py", line 184, in __getitem__
        return self.variables[var]
    KeyError: 'trans_*'
    

    I get the same thing if i use trans_.* I'm pretty sure this is happening because no matching is being done on X here: https://github.com/bids-standard/pybids/blob/15aa08fb2b146e3fcc62f359ea4a4d810d38591a/bids/analysis/analysis.py#L212-L213

    opened by Shotgunosine 28
  • Replace each nested list comprehension with a single DB query in BIDSLayout.__repr__

    Replace each nested list comprehension with a single DB query in BIDSLayout.__repr__

    I think there is currently some serious performance issues with BIDSLayout. Using a somewhat average database of 132 subjects (1 session and 1 run per subject), it needs about 1:15 minute to get a layout object. Using the following code:

    from bids import BIDSLayout
    %lprun -f BIDSLayout.__init__ BIDSLayout("/media/christian/ElementsSE/MPI-Leipzig_Mind-Brain-Body-LEMON/BIDS_LEMON/")
    

    I get the following profiling report :

    Total time: 76.4714 s
    File: /home/christian/pybids/bids/layout/layout.py
    Function: __init__ at line 196
    
    Line #      Hits         Time  Per Hit   % Time  Line Contents
    ==============================================================
       196                                               def __init__(self, root, validate=True, absolute_paths=True,
       197                                                            derivatives=False, config=None, sources=None, ignore=None,
       198                                                            force_index=None, config_filename='layout_config.json',
       199                                                            regex_search=False, database_path=None, database_file=None,
       200                                                            reset_database=False, index_metadata=True):
       201                                                   """Initialize BIDSLayout."""
       202         1          4.0      4.0      0.0          self.root = str(root)
       203         1          2.0      2.0      0.0          self.validate = validate
       204         1          1.0      1.0      0.0          self.absolute_paths = absolute_paths
       205         1          2.0      2.0      0.0          self.derivatives = {}
       206         1          2.0      2.0      0.0          self.sources = sources
       207         1          3.0      3.0      0.0          self.regex_search = regex_search
       208         1          2.0      2.0      0.0          self.config_filename = config_filename
       209                                                   # Store original init arguments as dictionary
       210         1          3.0      3.0      0.0          self._init_args = self._sanitize_init_args(
       211         1          2.0      2.0      0.0              root=root, validate=validate, absolute_paths=absolute_paths,
       212         1          2.0      2.0      0.0              derivatives=derivatives, ignore=ignore, force_index=force_index,
       213         1         91.0     91.0      0.0              index_metadata=index_metadata, config=config)
       214                                           
       215         1          4.0      4.0      0.0          if database_path is None and database_file is not None:
       216                                                       database_path = database_file
       217                                                       warnings.warn(
       218                                                           'In pybids 0.10 database_file argument was deprecated in favor'
       219                                                           ' of database_path, and will be removed in 0.12. '
       220                                                           'For now, treating database_file as a directory.',
       221                                                           DeprecationWarning)
       222         1          4.0      4.0      0.0          if database_path:
       223                                                       database_path = str(Path(database_path).absolute())
       224                                           
       225         1         47.0     47.0      0.0          self.session = None
       226                                           
       227         1      25891.0  25891.0      0.0          index_dataset = self._init_db(database_path, reset_database)
       228                                           
       229                                                   # Do basic BIDS validation on root directory
       230         1        488.0    488.0      0.0          self._validate_root()
       231                                           
       232         1          4.0      4.0      0.0          if ignore is None:
       233         1          3.0      3.0      0.0              ignore = self._default_ignore
       234                                           
       235                                                   # Instantiate after root validation to ensure os.path.join works
       236         1          3.0      3.0      0.0          self.ignore = [os.path.abspath(os.path.join(self.root, patt))
       237                                                                  if isinstance(patt, str) else patt
       238         1        102.0    102.0      0.0                         for patt in listify(ignore or [])]
       239         1          3.0      3.0      0.0          self.force_index = [os.path.abspath(os.path.join(self.root, patt))
       240                                                                       if isinstance(patt, str) else patt
       241         1          4.0      4.0      0.0                              for patt in listify(force_index or [])]
       242                                           
       243                                                   # Initialize the BIDS validator and examine ignore/force_index args
       244         1          4.0      4.0      0.0          self._validate_force_index()
       245                                           
       246         1          1.0      1.0      0.0          if index_dataset:
       247                                                       # Create Config objects
       248         1          2.0      2.0      0.0              if config is None:
       249         1          2.0      2.0      0.0                  config = 'bids'
       250         1          1.0      1.0      0.0              config = [Config.load(c, session=self.session)
       251         1      62271.0  62271.0      0.1                        for c in listify(config)]
       252         1         15.0     15.0      0.0              self.config = {c.name: c for c in config}
       253                                                       # Missing persistence of configs to the database
       254         2          6.0      3.0      0.0              for config_obj in self.config.values():
       255         1        308.0    308.0      0.0                  self.session.add(config_obj)
       256         1      27372.0  27372.0      0.0                  self.session.commit()
       257                                           
       258                                                       # Index files and (optionally) metadata
       259         1         35.0     35.0      0.0              indexer = BIDSLayoutIndexer(self)
       260         1   28127988.0 28127988.0     36.8              indexer.index_files()
       261         1          3.0      3.0      0.0              if index_metadata:
       262         1   48226769.0 48226769.0     63.1                  indexer.index_metadata()
       263                                                   else:
       264                                                       # Load Configs from DB
       265                                                       self.config = {c.name: c for c in self.session.query(Config).all()}
       266                                           
       267                                                   # Add derivatives if any are found
       268         1          3.0      3.0      0.0          if derivatives:
       269                                                       if derivatives is True:
       270                                                           derivatives = os.path.join(root, 'derivatives')
       271                                                       self.add_derivatives(
       272                                                           derivatives, parent_database_path=database_path,
       273                                                           validate=validate, absolute_paths=absolute_paths,
       274                                                           derivatives=None, sources=self, ignore=ignore,  config=None,
       275                                                           force_index=force_index, config_filename=config_filename,
       276                                                           regex_search=regex_search, index_metadata=index_metadata,
       277                                                           reset_database=index_dataset or reset_database
       278                                                           )
    

    For day-to-day interaction with a dataset, development tests, etc., this kind of delay seems prohibitive to me...

    opened by christian-oreilly 27
  • Derivatives indexing fails silently for deriv directories that lack a `dataset_description.json` file

    Derivatives indexing fails silently for deriv directories that lack a `dataset_description.json` file

    I have a set of derivatives files that look like this:

    derivatives/rt/sub-02/func/sub-02_task-dts_meanRT.tsv

    they seem to pass the validator fine, but I get nothing when I use .get(scope='derivatives'):

    In [35]: layout2 = BIDSLayout(args.bids_dir, derivatives=True)
    
    In [36]: layout2.get(scope='derivatives')
    Out[36]: []
    
    
    opened by poldrack 27
  • resample: ValueError: x and y arrays must be equal in length along interpolation axis.

    resample: ValueError: x and y arrays must be equal in length along interpolation axis.

    When trying to resample a dense variable (physio) with a sampling rate of 0.49999984750004656 the length of the x given to interp1d is one longer than values. My guess is that this has to do with a rounding error with the funky sampling_rate.

    bug 
    opened by adelavega 27
  • Renaming ce to contrast precludes the use of contrast in statistical derivatives

    Renaming ce to contrast precludes the use of contrast in statistical derivatives

    #464 breaks FitLins. I recognize that these derivatives aren't standardized, but neither is "contrast" a standardized name for the "ce" entity. As it is, trying to parse a FitLins derivatives structure yields:

    *** sqlalchemy.orm.exc.FlushError: New instance <Entity at 0x2b8e1e860c18> with identity key (<class 'bids.layout.models.Entity'>, ('contrast',), None) conflicts with persistent instance <Entity at 0x2b8e1d8da208>
    

    @tyarkoni @oesteban any objections to reverting this?

    opened by effigies 24
  • Improve  BIDSLayout performance on very large datasets

    Improve BIDSLayout performance on very large datasets

    Hey - I was trying to use pybids in a pipeline of mine, but found that it takes a very long time to create a layout on large datasets. I tested this using the public NKI-RS dataset from FCP-INDI in BIDS format, and ran using subsets of the dataset of various size: 2 subjects, 90, and the full thing, 963.

    My script is the following:

    from bids.layout import BIDSLayout
    import time
    
    start1 = time.time()
    bl = BIDSLayout('/project/6008063/gkiar/data/smallRS')
    dur1 = time.time() - start1
    print("{0}:{1}".format(len(bl.get_subjects()), dur1))
    
    
    start2 = time.time()
    bl = BIDSLayout('/project/6008063/gkiar/data/medRS/')
    dur2 = time.time() - start2
    print("{0}:{1}".format(len(bl.get_subjects()), dur2))
    
    start3 = time.time()
    bl = BIDSLayout('/project/6008063/gkiar/data/RocklandSample/')
    dur3 = time.time() - start3
    print("{0}:{1}".format(len(bl.get_subjects()), dur3))
    

    The output, in the form of {n subs}:{time elapsed in seconds}, is:

    $ date
    Wed Nov 14 15:40:58 EST 2018
    $ python profilebl.py
    2:0.39104771614074707
    90:14.413928270339966
    963:631.153669834137
    

    With this being the first step of my pipeline, and I'm almost always specifying a --participant_label or --session_label for a bunch of these launched in parallel, it would be great if it didn't take upwards of 10 minutes per task to find the data. I'd still like to have the BIDSLayout as part of the pipeline itself so I can grab various pieces of metadata as I need them.

    Any ideas why this gets so slow as sample size increases, or places you suggest I could make a PR to speed things up (likely in grabbit)?

    enhancement 
    opened by gkiar 24
  • [ENH] add visualize module to plot events file

    [ENH] add visualize module to plot events file

    • closes #661

    Summary

    Add en EventPlotter class to handle plotting of events file with plotly.

    • plots time course of each trial type
    • plots inter stimulus interval histogram
    • plots responses and response time histogram if there is a response_time column
    • allows to select which column should define the events to include

    Possible addition

    • [x] duration histogram
    • [x] pass list of trial types to include in plot
    • [ ] accept dataframe and BIDSDataFile as input ?

    TODO

    • documentation ?
    opened by Remi-Gau 2
  • Problem with Google Collab: ModuleNotFoundError

    Problem with Google Collab: ModuleNotFoundError

    Hello: I'm trying to use PyBIDS in Google Collab. However, after installing it:

    !pip3 install git+https://github.com/bids-standard/pybids.git

    Looking in indexes: https://pypi.org/simple, https://us-python.pkg.dev/colab-wheels/public/simple/ Collecting git+https://github.com/bids-standard/pybids.git Cloning https://github.com/bids-standard/pybids.git to /tmp/pip-req-build-ho9uyjz5 Running command git clone --filter=blob:none --quiet https://github.com/bids-standard/pybids.git /tmp/pip-req-build-ho9uyjz5 Resolved https://github.com/bids-standard/pybids.git to commit 18230cfe0facbc3b21f978613e20e5c53f6fd5f3 Running command git submodule update --init --recursive -q Installing build dependencies ... done Getting requirements to build wheel ... done Installing backend dependencies ... done Preparing metadata (pyproject.toml) ... done Requirement already satisfied: num2words in /usr/local/lib/python3.8/site-packages (from pybids==0.15.5.post0.dev16) (0.5.12) Requirement already satisfied: sqlalchemy<1.4.0.dev0 in /usr/local/lib/python3.8/site-packages (from pybids==0.15.5.post0.dev16) (1.3.24) Requirement already satisfied: scipy in /usr/local/lib/python3.8/site-packages (from pybids==0.15.5.post0.dev16) (1.9.3) Requirement already satisfied: nibabel>=2.1 in /usr/local/lib/python3.8/site-packages (from pybids==0.15.5.post0.dev16) (4.0.2) Requirement already satisfied: formulaic<0.6,>=0.2.4 in /usr/local/lib/python3.8/site-packages (from pybids==0.15.5.post0.dev16) (0.2.4) Requirement already satisfied: click>=8.0 in /usr/local/lib/python3.8/site-packages (from pybids==0.15.5.post0.dev16) (8.1.3) Requirement already satisfied: numpy in /usr/local/lib/python3.8/site-packages (from pybids==0.15.5.post0.dev16) (1.24.1) Requirement already satisfied: bids-validator in /usr/local/lib/python3.8/site-packages (from pybids==0.15.5.post0.dev16) (1.9.9) Requirement already satisfied: pandas>=0.23 in /usr/local/lib/python3.8/site-packages (from pybids==0.15.5.post0.dev16) (1.5.2) Requirement already satisfied: interface-meta>=1.2 in /usr/local/lib/python3.8/site-packages (from formulaic<0.6,>=0.2.4->pybids==0.15.5.post0.dev16) (1.3.0) Requirement already satisfied: wrapt in /usr/local/lib/python3.8/site-packages (from formulaic<0.6,>=0.2.4->pybids==0.15.5.post0.dev16) (1.14.1) Requirement already satisfied: astor in /usr/local/lib/python3.8/site-packages (from formulaic<0.6,>=0.2.4->pybids==0.15.5.post0.dev16) (0.8.1) Requirement already satisfied: packaging>=17.0 in /usr/local/lib/python3.8/site-packages (from nibabel>=2.1->pybids==0.15.5.post0.dev16) (22.0) Requirement already satisfied: setuptools in /usr/local/lib/python3.8/site-packages (from nibabel>=2.1->pybids==0.15.5.post0.dev16) (65.5.0) Requirement already satisfied: python-dateutil>=2.8.1 in /usr/local/lib/python3.8/site-packages (from pandas>=0.23->pybids==0.15.5.post0.dev16) (2.8.2) Requirement already satisfied: pytz>=2020.1 in /usr/local/lib/python3.8/site-packages (from pandas>=0.23->pybids==0.15.5.post0.dev16) (2022.7) Requirement already satisfied: docopt>=0.6.2 in /usr/local/lib/python3.8/site-packages (from num2words->pybids==0.15.5.post0.dev16) (0.6.2) Requirement already satisfied: six>=1.5 in /usr/local/lib/python3.8/site-packages (from python-dateutil>=2.8.1->pandas>=0.23->pybids==0.15.5.post0.dev16) (1.16.0)


    • If I try to import:

    from bids import BIDSLayout

    I get module not found error:

    ModuleNotFoundError: No module named 'bids'

    • Also, in the next case:

    from pybids import BIDSLayout

    ModuleNotFoundError: No module named 'pybids'

    Please, could you clarify how to do the import? Thank you in advance, María

    opened by mjmoron 1
  • fix: convolve and scale transformations

    fix: convolve and scale transformations

    It looks like newer version of pandas and pybids are having issues with some syntax in the Scale and Convolve transformations.

    In particular, Scale has an issue when performing a boolean operation on a Series

    Convolve requires the oversampling parameter to be coerced to an int

    opened by adelavega 1
  • PyBIDs query

    PyBIDs query

    Screenshot from 2022-12-06 17-20-15

    Dear developers,

    Is it possible to select a specific session and a specific run for a specific subject? Could it be possible to input 3 lists with the desired combinations (participant_id, session_id, run_id) and obtain the selected files (e.g T1w) ? See an example in the attached csv file.

    Best and many thanks in advance for your help and your advice Yasser Alemán Gómez

    opened by yasseraleman 1
  • Handle empty directories more gracefully

    Handle empty directories more gracefully

    If presented with a subject with an empty anat/ directory, PyBIDS will raise an unhandled StopIteration:

      File "/opt/conda/lib/python3.9/site-packages/fmriprep/config.py", line 472, in init
        cls._layout = BIDSLayout(
      File "/opt/conda/lib/python3.9/site-packages/bids/layout/layout.py", line 152, in __init__
        indexer(self)
      File "/opt/conda/lib/python3.9/site-packages/bids/layout/index.py", line 109, in __call__
        self._index_dir(self._layout._root, self._config)
      File "/opt/conda/lib/python3.9/site-packages/bids/layout/index.py", line 193, in _index_dir
        self._index_dir(d, list(config), default_action=default)
      File "/opt/conda/lib/python3.9/site-packages/bids/layout/index.py", line 193, in _index_dir
        self._index_dir(d, list(config), default_action=default)
      File "/opt/conda/lib/python3.9/site-packages/bids/layout/index.py", line 173, in _index_dir
        _, dirnames, filenames = next(os.walk(path))
    

    Running through pdb:

    StopIteration
    Uncaught exception. Entering post mortem debugging
    Running 'cont' or 'step' will restart the program
    > /opt/conda/lib/python3.9/site-packages/bids/layout/index.py(173)_index_dir()
    -> _, dirnames, filenames = next(os.walk(path))
    (Pdb) print(path)
    /data/sub-050202002B/anat
    (Pdb) up
    > /opt/conda/lib/python3.9/site-packages/bids/layout/index.py(193)_index_dir()
    -> self._index_dir(d, list(config), default_action=default)
    (Pdb) print(d)
    /data/sub-050202002B/anat
    (Pdb) up
    > /opt/conda/lib/python3.9/site-packages/bids/layout/index.py(193)_index_dir()
    -> self._index_dir(d, list(config), default_action=default)
    (Pdb) print(d)
    /data/sub-050202002B
    

    Initially reported in https://github.com/nipreps/fmriprep/issues/2902

    bug 
    opened by effigies 0
Releases(0.15.5)
  • 0.15.5(Nov 8, 2022)

    Release Notes

    Bug-fix release in 0.15.x series.

    What's Changed

    • FIX: Use kwargs for DataFrame.pivot() (#913)
    • ENH: Add transformation history to BIDSStatsModelsNodeOutput (#905)
    • ENH: Update examples, add to sphinx, and convert to .md (#908)
    • ENH: More helpful __repr__ in modeling (#906)
    • MNT: Update git submodules (#911)
    • MNT: Bump peter-evans/create-pull-request from 3 to 4 (#904)
    • CI: Automatically update GitHub Actions in the future (#903)

    Full Changelog: https://github.com/bids-standard/pybids/compare/0.15.4...0.15.5

    Source code(tar.gz)
    Source code(zip)
  • 0.15.3(Sep 12, 2022)

    DOI

    What's Changed

    • FIX: Test and update sync_kwargs class variable on transformations by @jdkent in (https://github.com/bids-standard/pybids/pull/891)
    • FIX: Filtering of DM columns by @adelavega in (https://github.com/bids-standard/pybids/pull/892)
    • ENH: Enable using participants.tsv, scans.tsv & session.tsv values as grouping variables by @adelavega in (https://github.com/bids-standard/pybids/pull/894)
    • REF: Rename incoming single contrast column to intercept by @adelavega in (https://github.com/bids-standard/pybids/pull/886)
    • DOC: Update references to nistats with references to nilearn in docs by @alexisthual in (https://github.com/bids-standard/pybids/pull/882)
    • TST: some sort of QA for derivatives' default_path_patterns by @GalBenZvi in (https://github.com/bids-standard/pybids/pull/879)
    • MNT: Codespell errors by @effigies in (https://github.com/bids-standard/pybids/pull/885)
    • CI: Update actions versions by @effigies in (https://github.com/bids-standard/pybids/pull/897)

    New Contributors

    • @GalBenZvi made their first contribution in https://github.com/bids-standard/pybids/pull/879
    • @alexisthual made their first contribution in https://github.com/bids-standard/pybids/pull/882

    Full Changelog: https://github.com/bids-standard/pybids/compare/0.15.2...0.15.3

    Source code(tar.gz)
    Source code(zip)
  • 0.15.2(Jul 25, 2022)

    DOI

    Release Notes

    Bug-fix release in 0.15.x series.

    Changes

    • FIX: Synchronize kwargs in transformations, e.g., in Assign (#836)
    • FIX: Add HRF derivatives to variable collection when created (#838)
    • FIX: Interpolate derivative pipeline name in error message (#847)
    • FIX: Match only within relative path when indexer is validating (#859)
    • FIX: Rename intercept in DummyContrast (#866)
    • FIX: Add variables to correct Node in load_tsv_variables (#872)
    • FIX: Repair variable IO so scans.tsv is found (#869)
    • FIX: Restore automodel functionality (#853)
    • FIX: Allow pybids layout --derivatives to be a boolean flag or accept a path (#848)
    • ENH: Add default path patterns to derivatives.json (#605)
    • ENH: Rename statsmodels contrasts if they cannot be uniquely identified downstream (#861)
    • ENH: Make intercept-only first level models throw a NotImplementedError (#854)
    • ENH: Update config to support microscopy, qMRI, PET, ASL (#840)
    • TEST: Add bids-examples submodule and bids_examples pytest fixture (#842)
    • TEST: Add tests for default path patterns (#846)
    • TEST: Update bids-examples and remove expected failing tests (#845)
    • MNT: Update git submodules (#874)
    • MNT: Add workflow top automatically update submodule (#870)
    • MNT: Remove deprecated no dot config (#841)
    • CI: Upgrade several GitHub actions to v3 (#867)
    • CI: Add cron jobs to avoid sneaky failures during low activity periods (#860)

    New Contributors

    • @jmumford made their first contribution in https://github.com/bids-standard/pybids/pull/869
    • @arovai made their first contribution in https://github.com/bids-standard/pybids/pull/847

    Full Changelog: https://github.com/bids-standard/pybids/compare/0.15.1...0.15.2

    Source code(tar.gz)
    Source code(zip)
  • 0.15.1(Apr 4, 2022)

    DOI

    Release Notes

    Bug-fix release in the 0.15.x series.

    Changes

    • RF/FIX: Decompose filter construction for special queries and lists (#826)
    • ENH: Relax group_by rules, allowing any entity to be used (#829)
    • MNT: Replace deprecated DataFrame.append call (#833)
    • DOC: Fresh rerun of entire pybids_tutorial.ipynb (#832)

    Full Changelog: https://github.com/bids-standard/pybids/compare/0.15.0...0.15.1

    Source code(tar.gz)
    Source code(zip)
  • 0.14.1(Apr 4, 2022)

    DOI

    Release Notes

    Bug-fix release in the 0.14.x series.

    Changes

    • RF/FIX: Decompose filter construction for special queries and lists by @effigies in https://github.com/bids-standard/pybids/pull/826

    Includes the following back-ports from 0.15.0:

    • FIX: Clarify exception message (#806)
    • FIX: Catch UnicodeDecodeErrors along with JSONDecodeErrors for better reporting (#796)
    • FIX: Accept paths/strings for layout configuration files (#799)
    • ENH: Add __main__ to allow python -m bids to run CLI (#794)

    Full Changelog: https://github.com/bids-standard/pybids/compare/0.14.0...0.14.1

    Source code(tar.gz)
    Source code(zip)
  • 0.15.0(Mar 28, 2022)

    DOI

    Release Notes

    New feature release in the 0.15.x series.

    Downstream tools should be aware of a potentially breaking, albeit long-demanded, change introduced in #819. Run indices are now stored so that the integers that come out retain any zero-padding that was found during parsing.

    This release also introduces the bids.ext namespace package that allows independent packages to install modules in this namespace. This is an infrastructural change that will allow some components to be separately managed and follow a different development pace.

    Changes

    • FIX: Allow grouping by run and session when entities are undefined (#822)
    • FIX: Clarify exception message (#806)
    • FIX: Catch UnicodeDecodeErrors along with JSONDecodeErrors for better reporting (#796)
    • FIX: Accept paths/strings for layout configuration files (#799)
    • FIX: Small typo: repeated word in docstring (#793)
    • ENH: Retain zero-padding in run entities while preserving integer queries and comparisons (#819)
    • ENH: Add bids.ext namespace package for subpackages (#820)
    • ENH: Handle wildcards in model X (#810)
    • ENH: Implement automatic detection of derivative data (#805)
    • ENH: Add new Query for optional entities (#809)
    • ENH: Add main to allow python -m bids to run CLI (#794)
    • REF: Improve modularization of bids.reports (#617)
    • DOC: Link from sphinx documentation to notebook tutorials. (#797)
    • MNT: Test on Python 3.10, various CI updates (#824)
    • MNT: Avoid jinja2 v3 until nbconvert handles breakages (#823)

    New Contributors

    • @kousu made their first contribution in https://github.com/bids-standard/pybids/pull/791
    • @jstaph made their first contribution in https://github.com/bids-standard/pybids/pull/806
    • @pvandyken made their first contribution in https://github.com/bids-standard/pybids/pull/805

    Full Changelog: https://github.com/bids-standard/pybids/compare/0.14.0...0.15.0

    Source code(tar.gz)
    Source code(zip)
  • 0.14.0(Nov 9, 2021)

    DOI

    Release Notes

    New feature release in the 0.14.x series.

    This release includes a significant refactor of BIDS Statistical Models, replacing the bids.analysis module with bids.modeling.

    Changes to bids.layout are minimal, and we do not anticipate API breakage.

    Changes

    • FIX: LGTM.com warning: Implicit string concatenation in a list (#785)
    • FIX: Take the intersection of variables and Model.X, ignoring missing variables (usually contrasts) (#764)
    • FIX: Associate "is_metadata" with Tag, not Entity; and only return non-metadata entries for core Entities in get(return_type='id') (#749)
    • FIX: Only include regressors if they are TSV (#752)
    • FIX: ensure force_dense=True runs to_dense only on sparse variables (#745)
    • FIX: get unique, with conflicting meta-data (#748)
    • FIX: Clean up some deprecation and syntax warnings (#738)
    • ENH: Add pybids upgrade command (#654)
    • ENH: Add Lag transformation (#759)
    • ENH: Use indirect transformations structure (#737)
    • ENH: Add visualization for statsmodel graph (#742)
    • ENH: Permit explicit intercept (1) in Contrasts and DummyContrasts (#743)
    • ENH: Add meta-analysis model type (#731)
    • ENH: Contrast type is now test (#733)
    • REF: Use pathlib.Path internally when possible (#746)
    • REF: Remove group_by from edges and add filter (#734)
    • REF: Improved/refactored StatsModels module (#722)
    • MNT: Make sure codespell skips .git when run locally (#787)
    • MNT: LGTM.com recommendations (#786)
    • MNT: Better codespell configuration (#782)
    • MNT: Constrain formulaic version to 0.2.x . (#784)
    • MNT: Update versioneer: 0.18 → 0.20 (#778)
    • MNT: Add "codespell" tool to CI checks to catch typos sooner (#776)
    • MNT: Disable bids-nodot mode (#769)
    • MNT: Send codecov reports again (#766)
    • MNT: Set minimum version to Python 3.6 (#739)
    Source code(tar.gz)
    Source code(zip)
  • 0.13.2(Aug 27, 2021)

    DOI

    Release Notes

    Bug-fix release in the 0.13 series.

    CHANGES

    • FIX/TEST: gunzip regressors.tsv.gz, allow timeseries.tsv as well (#767)
    • FIX: run is required (#762)
    • MNT: Patch 0.13.x maint branch (#763)
    Source code(tar.gz)
    Source code(zip)
  • 0.13.1(May 21, 2021)

    DOI

    Release Notes

    Bug-fix release in the 0.13 series. With thanks to Lea Waller for contributions.

    CHANGES

    • ENH: Improve get performance (#723)
    • STY: Fix typos identified by codespell (#720)
    • TEST: dataset-level model spec retrieval (#693)
    Source code(tar.gz)
    Source code(zip)
  • 0.13(Apr 16, 2021)

    Release Notes

    Minor new feature and bugfix release in the 0.13 series.

    CHANGES

    Version 0.13 (April 14, 2021)

    FIX: Resample to n_vols for sampling_rate == 'TR' (#713)
    FIX: Lazily load metadata, skip when missing data files are missing sidecars (#711)
    FIX: Ensure indicator matrix is boolean when indexing in Split transformation (#710)
    FIX: Correctly pair metadata file when creating association records (#699)
    FIX: Resolve side-effects of new testfile in #682 (#695)
    FIX: BIDSLayout -- TypeError: unhashable type: 'dict' (#682)
    ENH: Add res/den entities to derivatives.json (#709)
    ENH: Update datatypes (#708)
    ENH: add more informative validation error message for dataset_description.json (#705)
    ENH: Add flip, inv, mt, and part entities (#688)
    CI: Run packaging tests on main repository only (#696)
    CI: Migrate to GH actions (#691)
    
    Source code(tar.gz)
    Source code(zip)
  • 0.12.3(Oct 24, 2020)

    Release Notes

    Minor new feature and bugfix release in the 0.12.x series.

    CHANGES

    • FIX: Require aligned input for logical operations (#649) @adelavega
    • ENH: Incremental variable loading in Step setup (#672) @tyarkoni
    Source code(tar.gz)
    Source code(zip)
  • 0.12.2(Oct 9, 2020)

    DOI

    Release Notes

    Bug-fix release in 0.12.x series.

    Changes

    • FIX: Support nibabel < 3 when calculating time series length (#669)
    • FIX: Sanitize single derivative Path passed to BIDSLayout (#665)
    • FIX: Force UTF-8 encoding in _index_metadata (#663)
    • FIX: Explicitly convert to HTML when testing tutorial (nbconvert 6.0 breakage) (#664)
    Source code(tar.gz)
    Source code(zip)
  • 0.12.0(Aug 5, 2020)

    DOI

    Release Notes

    New feature release for the 0.12.x series.

    This release includes significant speedups for BIDSLayout creation and improves handling of GIFTI and CIFTI-2 derivative files.

    Changes

    • FIX: Remove need to eager load associations (#642)
    • ENH: Fetch number of time points from NIfTI, GIFTI or CIFTI-2 (#637)
    • ENH: Catch any NIfTI/GIFTI (incl. CIFTI-2) files as BIDSImageFiles (#638)
    Source code(tar.gz)
    Source code(zip)
  • 0.11.1(Jul 2, 2020)

    DOI

    Release Notes

    Bug-fix release in the 0.11.x series

    With thanks to Mathias Goncalves for finding and fixing the issue.

    Changes

    • FIX: Preserve get logic when using custom config (#636)
    Source code(tar.gz)
    Source code(zip)
  • 0.11.0(Jun 29, 2020)

    DOI

    Release Notes

    New feature release in the 0.11.x series.

    One significant change in this PR is to add the configuration option extension_initial_dot. Setting to True will adopt the future default behavior of returning extension entities with an initial dot (.).

    Other notable changes include a significant refactoring of the Analysis module, and a number of small improvements to error reporting that should add up to simpler debugging for users and developers.

    Changes

    • FIX: Post-fix And and Or transformations with underscore (#628)
    • FIX: made _validate_file work on Windows (#627)
    • FIX: Scale transformation fails if passed constant input. (#614)
    • FIX: Certain queries involving multiple layouts are very slow (#616)
    • FIX: Bug in get() when passing enums as extensions (#612)
    • FIX: Bug in BIDSDataFile.get_df() (#611)
    • FIX: Make entity assertions Python 3.5-safe (#606)
    • FIX: BIDSLayout.build_path to respect absolute_paths. (#580)
    • ENH: Adds OS-level file copying instead of reading/writing via Python (#613)
    • ENH: Add explicit warning message when users pass in filters as a dictionary keyword (#623)
    • ENH: Introduce PyBIDS exceptions (#615)
    • ENH: Add example bids and derivatives dataset_description.json strings to error messages (#620)
    • ENH: Improved handling of invalid filters (#610)
    • ENH: Add method to generate report from list of files (#607)
    • ENH: Pass kwargs from BIDSImageFile.get_image() to nibabel.load (#601)
    • ENH: Model spec module and associated utilities (#548)
    • ENH: Add BIDSMetadata dictionary to report file with missing metadata (#593)
    • RF: Add extension_initial_dot config option to transition to extension entity with initial dot (#629)
    • MNT: Automatically deploy docs (#598)
    • CI: Drop --pre check for Python 3.5 (#621)
    • CI: Test on Python 3.8 (#594)
    Source code(tar.gz)
    Source code(zip)
  • 0.10.2(Feb 26, 2020)

  • 0.10.1(Feb 15, 2020)

    DOI

    Release Notes

    Bug fix release in the 0.10.x series.

    This release just makes available some of the latest minor fixes and improvements.

    Changes

    • FIX: Replace os.path.sep with fwdslash because bids validator hardcodes fwd (#582)
    • FIX: Refactor of build_path and inner machinery (#574)
    • FIX: Domain entity, and slow __repr__ (#569)
    • FIX: "strict" helptext in BIDSLayout.write_contents_to_file (#566)
    • FIX: typos in helpstrings and comments (#564)
    • FIX: Correct term "caret" to "angle bracket" in helpstrings (#565)
    • ENH: Extend build_path to generate lists of files (#576)
    • ENH: Add one parametric test of BIDSLayout.build_path (#577)
    • ENH: Enable partial metadata indexing (#560)
    • ENH: Upscale to collection sampling rate prior to resampling (#568)
    • ENH: Calculate default sampling rate for SparseRunVariable.to_dense (#571)
    • MNT: Add .vscode (for Visual Studio Code) to .gitignore (#562)
    • MNT: Ignore pip-wheel-metadata (#581)
    • DOC: Remove Python 2 support statement, now that v0.10.0 has dropped it (#561)
    Source code(tar.gz)
    Source code(zip)
  • 0.10.0(Dec 3, 2019)

    DOI

    Release Notes

    New feature release in the 0.10.x series.

    This release removes Python 2 support.

    Changes

    • ENH: Helpful error for db argument mismatch, and add classmethod load_from_db (#547)
    • ENH: Add Resample transformation (#373)
    • ENH: Save BIDSLayout + Derivatives to folder (with init arguments) (#540)
    • ENH: Adds support for NONE and ANY query Enums (#542)
    • ENH: Transformation-related improvements (#541)
    • ENH: FEMA contrasts (#520)
    • STY: Minor PEP8 Fixes (#545)
    • MNT: Various (#543)
    • MNT: Remove Python 2.7 support (#524)
    • CI: Configure Circle Artifact Redirector (#544)
    Source code(tar.gz)
    Source code(zip)
  • 0.9.5(Nov 11, 2019)

    DOI

    Release Notes

    Bug fix release in the 0.9.x series.

    Final planned release with Python 2 support.

    With thanks to new contributors Ashley Gillman, Satra Ghosh and Hans Johnson.

    Changes

    • FIX: Filter before downsampling (#529)
    • FIX: Copy input dict in replace_entities; Typos in default_path_patterns (#517)
    • FIX: Use string dtype for all entities when using regex search (#511)
    • FIX: Update Sphinx docs for 2.2 (#507)
    • ENH: Enable automatic derivative database caching (#523)
    • ENH: Raise ValueError if BIDSLayout.build_path fails to match any pattern (#508)
    • RF: Subclass analysis Node from object (#528)
    • DOC: Unify docstring format (#499)
    • DOC: Auto-generate stubs (#513)
    • STY: .sql is a misleading extension to sqlite files (#531)
    • STY: General cleanups (#526, #527)
    Source code(tar.gz)
    Source code(zip)
  • 0.9.4(Sep 23, 2019)

    DOI

    Release Notes

    Bug fix release in the 0.9.x series.

    With thanks to new contributor Jon Haitz Legarreta.

    Changes

    • FIX: Ignore default_ignore paths by default (#495)
    • FIX: Filter and sort on scalar metadata in get_nodes() (#488)
    • FIX: Automatically sanitize dtype of get() arguments (#492)
    • FIX: Check that default_path_patterns is not None before using in build_path (#485)
    • FIX: Add CBV and phase modalities to func datatype path pattern (#479)
    • FIX: Drop bold suffix constraint from echo entity (#477)
    • ENH: Enforce dtypes on spec-defined columns when reading in DFs (#494)
    • ENH: Validate built paths (#496)
    • ENH: Update contrast enhancing agent query name (ceagent) (#497)
    • DOC: Add citation information to README (#493)
    • DOC: Improve wording in Python notebook example comment (#484)
    • DOC: Finish sentence in Python Notebook example documentation (#483)
    • DOC: Add JOSS Badge (#472)
    • STY: Apply syntax highlight to Python notebook example doc (#482)
    • MAINT: Move setup configuration to setup.cfg (#448)
    • MAINT: Additional Zenodo metadata (#470)
    • MAINT/CI: Use extras_require to declare dependencies for tests (#471)
    Source code(tar.gz)
    Source code(zip)
  • 0.9.3(Aug 8, 2019)

    DOI

    Release Notes

    This version includes a number of minor fixes and improvements, one of which breaks the existing API (by renaming two entities; see #464).

    With thanks to new contributor Remi Gau.

    Changes

    • FIX: Avoid DB collisions for redundant entities (#468)
    • FIX: Minor changes to entity names in core spec (#464)
    • FIX: Make bids.reports work properly with .nii images (#463)
    • CI: Execute notebook in Travis (#461)
    • ENH: More sensible __repr__ for Tag model (#467)
    Source code(tar.gz)
    Source code(zip)
  • 0.9.2(Jul 12, 2019)

    DOI

    Release Notes

    This version includes a number of minor fixes and improvements. EEG files are better handled, and BIDSLayout and BIDSFile play more nicely with Path-like objects.

    With thanks to new contributor Cecile Madjar.

    Changes

    • FIX: Instantiate ignore/force_index after root validation (#457)
    • FIX: Restore <entity>=None query returning files lacking the entity (#458)
    • ENH: Add BIDSJSONFile (#444)
    • ENH: Add BIDSFile.__fspath__ to work with pathlib (#449)
    • ENH: Add eeg datatype to layout config (#455)
    • RF: Remove unused kwargs to BIDSFile (#443)
    • DOC: Improve docstring consistency, style (#443)
    • DOC: Address final JOSS review (#453)
    • STY: PEP8 Fixes (#456)
    • MAINT: Set name explicitly in setup.py (#450)
    Source code(tar.gz)
    Source code(zip)
  • 0.9.1(May 24, 2019)

    DOI

    Release notes

    Hotfix release:

    • Fixed package deployment issues (#446)
    • Updated author list (#447)

    Thanks to new contributors Erin Dickie, Chadwick Boulay and Johannes Wennberg.

    Source code(tar.gz)
    Source code(zip)
  • 0.9.0(May 21, 2019)

    DOI

    Release notes

    Version 0.9 replaces the native Python backend with a SQLite database managed via SQLAlchemy. The layout module has been refactored (again), but API changes are minimal. This release also adds many new features and closes a number of open issues.

    API CHANGES/DEPRECATIONS:

    • The extensions argument has now been banished forever; instead, use extension, which is now defined as a first-class entity. The former will continue to work until at least the 0.11 release (closes #404).
    • Relatedly, values for extension should no longer include a leading ., though this should also continue to work for the time being.
    • The BIDSLayout init argument index_associated has been removed as the various other filtering/indexing options mean there is longer a good reason for users to manipulate this.
    • bids.layout.MetadataIndex no longer exists. It's unlikely that anyone will notice this.
    • BIDSLayout.get_metadata() no longer takes additional entities as optional keyword arguments (they weren't necessary for anything).
    • Direct access to most BIDSFile properties is discouraged, and in one case is broken in 0.9 (for .metadata, which was unavoidable, because it's reserved by SQLAlchemy). Instead, users should use getters (get_metadata, get_image, get_df, etc.) whenever possible.

    NEW FUNCTIONALITY:

    • All file and metadata indexing and querying is now supported by a relational (SQLite) database (see #422). While this has few API implications, the efficiency of many operations is improved, and complex user-generated queries can now be performed via the SQLAlchemy .session stored in each BIDSLayout.
    • Adds .save() method to the BIDSLayout that saves the current SQLite DB to the specified location. Conversely, passing a filename as database_file at init will use the specified store instead of re-indexing all files. This eliminates the need for a pickling strategy (#435).
    • Related to the above, the BIDSLayout init adds a reset_database argument that forces reindexing even if a database_file is specified.
    • The BIDSLayout has a new index_metadata flag that controls whether or not the contents of JSON metadata files are indexed.
    • Added metadata flag to BIDSLayout.to_df() that controls whether or not metadata columns are included in the returned pandas DataFrame (#232).
    • Added get_entities() method to BIDSLayout that allows retrieval of all Entity instances available within a specified scope (#346).
    • Adds drop_invalid_filters argument to BIDSLayout.get(), enabling users to (optionally) ensure that invalid filters don't clobber all search results (#402).
    • BIDSFile instances now have a get_associations() method that returns associated files (see #431).
    • The BIDSFile class has been split into a hierarchy, with BIDSImageFile and BIDSDataFile subclasses. The former adds a get_image() method (returns a NiBabel image); the latter adds a get_df() method (returns a pandas DF). All BIDSFile instances now also have a get_entities() method.

    BUG FIXES AND OTHER MINOR CHANGES:

    • Metadata key/value pairs and file entities are now treated identically, eliminating a source of ambiguity in search (see #398).
    • Metadata no longer bleeds between raw and derivatives directories unless explicitly specified (see #383).
    • BIDSLayout.get_collections() no longer drops user-added columns (#273).
    • Various minor fixes/improvements/changes to tests.
    • The tutorial Jupyter notebook has been fixed and updated to reflect the latest changes.
    Source code(tar.gz)
    Source code(zip)
  • 0.8.0(Feb 15, 2019)

    DOI

    Release notes

    Version 0.8 refactors much of the layout module. It drops the grabbit dependency, overhauls the file indexing process, and features a number of other improvements. However, changes to the public API are very minimal, and in the vast majority of cases, 0.8 should be a drop-in replacement for 0.7.*.

    API-breaking changes

    • Changes to (rarely-used) BIDSLayout initialization arguments:
      • include and exclude have been replaced with ignore and force_index. Paths passed to ignore will be ignored from indexing; paths passed to force_index will be forcibly indexed even if they are otherwise BIDS-non-compliant. force_index takes precedence over ignore.
    • Most querying/selection methods add a new scope argument that controls scope of querying (e.g., 'raw', 'derivatives', 'all', etc.). In some cases this replaces the more limited derivatives argument.
    • No more domains: with the grabbit removal (see below), the notion of a 'domain' has been removed. This should impact few users, but those who need to restrict indexing or querying to specific parts of a BIDS project should be able to use the scope argument more effectively.

    Other changes

    • FIX: Path indexing issues in get_file() (#379)
    • FIX: Duplicate file returns under certain conditions (#350)
    • FIX: Pass new variable args as kwargs in split() (#386) @effigies
    • TEST: Update naming conventions for synthetic dataset (#385) @effigies
    • REF: The grabbit package is no longer a dependency; as a result, much of the functionality from grabbit has been ported over to pybids.
    • REF: Required functionality from six and inflect is now bundled with pybids in bids.external, minimizing external dependencies.
    • REF: Core modules have been reorganized. Key data structures and containers (e.g., BIDSFile, Entity, etc.) are now in a new bids.layout.core module.
    • REF: A new Config class has been introduced to house the information found in bids.json and other layout configuration files.
    • REF: The file-indexing process has been completely refactored. A new hierarchy of BIDSNode objects has been introduced. While this has no real impact on the public API, and isn't really intended for public consumption yet, it will in future make it easier for users to work with BIDS projects in a tree-like way, while also laying the basis for a more sensible approach to reading and accessing associated BIDS data (e.g., .tsv files).
    • MNT: All invocations of pd.read_table have been replaced with read_csv.
    Source code(tar.gz)
    Source code(zip)
  • 0.7.1(Feb 1, 2019)

    DOI

    Release Notes

    This is a bug fix release in the 0.7 series. The primary API change is improved handling of Path objects.

    CHANGES

    • FIX: Path validation (#342)
    • FIX: Ensure consistent entities at all levels (#326)
    • FIX: Edge case where a resampled column was too-long-by-one (#365)
    • FIX: Use BIDS metadata for TR over nii header (#357)
    • FIX: Add check for run_info to be a list, pass run_info in correct position. (#353)
    • FIX: If sampling_rate is 'auto', set to first rate of DenseRunVariables (#351)
    • FIX: Get the absolute path of the test data directory (#347)
    • FIX: Update reports to be 0.7-compatible (#341)
    • ENH: Rename sr variable to more intuitive interval (#366)
    • ENH: Support pathlib.Path and other str-castable types (#307)
    • MNT: Updates link to derivative config file in notebook (#344)
    • MNT: Add bids-validator dependency (#363)
    • MNT: Require pandas >= 0.23.0 (#348)
    • MNT: Bump grabbit version (#338)
    • CI: Ignore OSX Python 3.5 failures (#372)
    • CI: Build with Python 3.7 on Travis, deploy on 3.6 (#337)
    Source code(tar.gz)
    Source code(zip)
  • 0.7.0(Jan 10, 2019)

    DOI

    Release notes

    This is a major, API-breaking release. It introduces a large number of new features, bug fixes, and improvements.

    API-BREAKING CHANGES:

    • A number of entities (or keywords) have been renamed to align more closely with the BIDS specification documents:
      • 'type' becomes 'suffix'
      • 'modality' becomes 'datatype'
      • 'acq' is removed (use 'acquisition')
      • 'mod' becomes 'modality'
    • The following directories are no longer indexed by default: derivatives/, code/, stimuli/, models/, sourcedata/. They must be explicitly included using the 'include' initialization argument.
    • The grabbids module has been renamed to layout and BIDSLayout.py and BIDSvalidator.py are now layout.py and validation.py, respectively.
    • The BIDS validator is now enabled by default at layout initialization (i.e., validate=True)
    • The exclude initialization argument has been removed.
    • BIDSLayout.parse_entities utility has been removed (use the more flexible parse_file_entities).
    • Calls to .get() now return BIDSFile objects, rather than namedtuples, by default (#281). The BIDSFile API has been tweaked to ensure backwards incompatibility in nearly all cases.
    • Naming conventions throughout the codebase have been updated to ensure consistency with the BIDS specs. This is most salient in the analysis module, where snake_case has been replaced with CamelCase throughout.

    NEW FEATURES:

    • File metadata (i.e., in JSON sidecars) is now searchable by default, and behaves just like native BIDS entities (e.g., metadata keys can be passed as arguments to .get() calls)
    • A new BIDSFile wrapper provides easy access to .metadata and .image
    • HRF convolution is now supported via bundling of nistats' hemodynamic_models module; convolution is handled via the convolve_HRF transformation.
    • Named config paths that customize how projects are processed can be added at run-time (#313)
    • Preliminary support for BIDS-Derivatives RC1 (mainly core keywords)

    MINOR IMPROVEMENTS AND BUG FIXES:

    • Specifying 'derivatives' in a path specification now automatically includes 'bids' (#246)
    • Zenodo DOIs are now minted with new releases (#308)
    • Variable loading via load_variables can now be done incrementally
    • Expanded and improved path-building via layout.build_path()
    • get_collections no longer breaks when merge=True and the list is empty (#202)
    • Layout initialization no longer fails when validate=True (#222)
    • The auto_contrasts field in the modeling tools now complies with the BIDS-Model spec (#234)
    • Printing a BIDSFile now provides more useful information, including path (#298)
    • Resample design matrix to 1/TR by default (#309)
    • Fix the Sum transformation
    • Ensure that resampling works properly when a sampling rate is passed to get_design_matrix (#297)
    • Propagate derivative entities into top-level dynamic getters (#306)
    • Deprecated get_header call in nibabel removed (#300)
    • Fix bug in entity indexing for BIDSVariableCollection (#319)
    • Exclude modules with heavy dependencies from root namespace for performance reasons (#321)
    • Fix bug that caused in-place updating of input selectors in Analysis objects (#323)
    • Add a DropNA transformation (#325)
    • Add a get_tr() method to BIDSLayout (#327)
    • Add entity hints when calling get() with a target argument (#328)
    • Improved test coverage
    Source code(tar.gz)
    Source code(zip)
  • 0.6.5(Aug 21, 2018)

    Release Notes

    This release includes a number of documentation improvements, BIDS-MEG support, as well as some bug fixes.

    CHANGES

    • FIX: Do not drop rows of NaNs (#217) @adelavega
    • FIX: Declare run as having integer type (#236) @effigies
    • ENH: MEG support (#229) @jasmainak
    • REF: rename grabbids to layout, closes #228 (#230) @ltirrell
    • DOC: add .get_collection examples to tutorial (#219) @Shotgunosine
    • DOC: Fix link in README to point to documentation (#223) @KirstieJane
    • DOC: Add binder link for tutorial (#225) @KirstieJane
    • MAINT: Restore "analysis" installation extra (#218) @yarikoptic
    • MAINT: Do not import tests in __init__.py (#226) @tyarkoni
    Source code(tar.gz)
    Source code(zip)
  • 0.6.4(Jul 24, 2018)

  • 0.6.3(Jun 19, 2018)

Owner
Brain Imaging Data Structure
Brain Imaging Data Structure
A Pythonic introduction to methods for scaling your data science and machine learning work to larger datasets and larger models, using the tools and APIs you know and love from the PyData stack (such as numpy, pandas, and scikit-learn).

This tutorial's purpose is to introduce Pythonistas to methods for scaling their data science and machine learning work to larger datasets and larger models, using the tools and APIs they know and love from the PyData stack (such as numpy, pandas, and scikit-learn).

Coiled 102 Nov 10, 2022
Python Package for DataHerb: create, search, and load datasets.

The Python Package for DataHerb A DataHerb Core Service to Create and Load Datasets.

DataHerb 4 Feb 11, 2022
Python dataset creator to construct datasets composed of OpenFace extracted features and Shimmer3 GSR+ Sensor datas

Python dataset creator to construct datasets composed of OpenFace extracted features and Shimmer3 GSR+ Sensor datas

Gabriele 3 Jul 5, 2022
CleanX is an open source python library for exploring, cleaning and augmenting large datasets of X-rays, or certain other types of radiological images.

cleanX CleanX is an open source python library for exploring, cleaning and augmenting large datasets of X-rays, or certain other types of radiological

Candace Makeda Moore, MD 20 Jan 5, 2023
HyperSpy is an open source Python library for the interactive analysis of multidimensional datasets

HyperSpy is an open source Python library for the interactive analysis of multidimensional datasets that can be described as multidimensional arrays o

HyperSpy 411 Dec 27, 2022
Instant search for and access to many datasets in Pyspark.

SparkDataset Provides instant access to many datasets right from Pyspark (in Spark DataFrame structure). Drop a star if you like the project. ?? Motiv

Souvik Pratiher 31 Dec 16, 2022
VHub - An API that permits uploading of vulnerability datasets and return of the serialized data

VHub - An API that permits uploading of vulnerability datasets and return of the serialized data

André Rodrigues 2 Feb 14, 2022
🧪 Panel-Chemistry - exploratory data analysis and build powerful data and viz tools within the domain of Chemistry using Python and HoloViz Panel.

???? ??. The purpose of the panel-chemistry project is to make it really easy for you to do DATA ANALYSIS and build powerful DATA AND VIZ APPLICATIONS within the domain of Chemistry using using Python and HoloViz Panel.

Marc Skov Madsen 97 Dec 8, 2022
Active Learning demo using two small datasets

ActiveLearningDemo How to run step one put the dataset folder and use command below to split the dataset to the required structure run utils.py For ea

null 3 Nov 10, 2021
MetPy is a collection of tools in Python for reading, visualizing and performing calculations with weather data.

MetPy MetPy is a collection of tools in Python for reading, visualizing and performing calculations with weather data. MetPy follows semantic versioni

Unidata 971 Dec 25, 2022
A collection of robust and fast processing tools for parsing and analyzing web archive data.

ChatNoir Resiliparse A collection of robust and fast processing tools for parsing and analyzing web archive data. Resiliparse is part of the ChatNoir

ChatNoir 24 Nov 29, 2022
GWpy is a collaboration-driven Python package providing tools for studying data from ground-based gravitational-wave detectors

GWpy is a collaboration-driven Python package providing tools for studying data from ground-based gravitational-wave detectors. GWpy provides a user-f

GWpy 342 Jan 7, 2023
A Python Tools to imaging the shallow seismic structure

ShallowSeismicImaging Tools to imaging the shallow seismic structure, above 10 km, based on the ZH ratio measured from the ambient seismic noise, and

Xiao Xiao 9 Aug 9, 2022
Additional tools for particle accelerator data analysis and machine information

PyLHC Tools This package is a collection of useful scripts and tools for the Optics Measurements and Corrections group (OMC) at CERN. Documentation Au

PyLHC 3 Apr 13, 2022
Tools for the analysis, simulation, and presentation of Lorentz TEM data.

ltempy ltempy is a set of tools for Lorentz TEM data analysis, simulation, and presentation. Features Single Image Transport of Intensity Equation (SI

McMorran Lab 1 Dec 26, 2022
Visions provides an extensible suite of tools to support common data analysis operations

Visions And these visions of data types, they kept us up past the dawn. Visions provides an extensible suite of tools to support common data analysis

null 168 Dec 28, 2022
Helper tools to construct probability distributions built from expert elicited data for use in monte carlo simulations.

Elicited Helper tools to construct probability distributions built from expert elicited data for use in monte carlo simulations. Credit to Brett Hoove

Ryan McGeehan 3 Nov 4, 2022
Universal data analysis tools for atmospheric sciences

U_analysis Universal data analysis tools for atmospheric sciences Script written in python 3. This file defines multiple functions that can be used fo

Luis Ackermann 1 Oct 10, 2021
Tools for working with MARC data in Catalogue Bridge.

catbridge_tools Tools for working with MARC data in Catalogue Bridge. Borrows heavily from PyMarc

null 1 Nov 11, 2021