This python module allows to extract data from the RAW-file-format produces by devices from Thermo Fisher Scientific.

Overview

fisher_py

This Python module allows access to Thermo Orbitrap raw mass spectrometer files. Using this library makes it possible to automate the analysis of mass spectra, without having to export the data first with another tool. This module is a wrapper that builds uppon the RawFileReader project which is a library developed for C#. Structures have been implemented to make processing data more convenient for Python users.

Installation

fisher_py can be installed via the package repository PyPi:

pip install fisher_py

System Requirements

fishery_py shoud work on any modern desktop operating system (Linux, Windows, Mac OS) with Python 3.6 (or higher) installed.

  • Windows: Tested on Windows 10 Pro x64
  • Linux: Tested on Ubuntu 20.04 LTS x64
  • Mac OS: Not tested

The module relies on the RawFileReader DLLs (Dynamic-Linked-Libraries) to be loaded at runtime (using pythonnet). Since Microsoft introduced .NET Standard it is possible to load DLLs compiled with this framework to be loaded on non-Windows systems (such as Mac OS and Linux). However, systems other than Windows may require additional setup steps in order for fisher_py to work. If you have trouble problems installing fisher_py it is probably because of pythonnet not being able to compile. To resolve this the usualy path is to install mono (https://www.mono-project.com/). There are several guides online to do this but one that was tested can be found here.

Examples

The following example demonstrates how to extract and plot data from a raw-file:

import matplotlib.pyplot as plt
from fisher_py import RawFile
from fisher_py.data.business import TraceType
raw_file = RawFile('my_file.raw')

target_mass = 848.36862
mass_tolerance_ppm = 10
rt, i = raw_file.get_chromatogram(target_mass, mass_tolerance_ppm, TraceType.MassRange)
mz, i2, charges, real_rt = raw_file.get_scan_ms1(1)
print(real_rt)

plt.figure()
plt.plot(rt, i)

plt.figure()
plt.plot(mz, i2)

plt.show()

This example may be fine for some use-cases but the RawFile class only provides limited access to all the functionalities and can serve as an example how to use the module wihtin a project. For an example that uses more of the modules capabilites have a look at raw_file_reader_examle.py.

License and copyright

fisher_py (Copyright 2021 ethz-institute-of-microbiology) is licensed under the MIT license.

Third-party licenses and copyright

RawFileReader reading tool. Copyright © 2016 by Thermo Fisher Scientific, Inc. All rights reserved. See RawFileReaderLicense.md for licensing information. Note: anyone recieving RawFileReader as part of a larger software distribution (in the current context, as part of fisher_py) is considered an "end user" under section 3.3 of the RawFileReader License, and is not granted rights to redistribute RawFileReader.

Comments
  • Missing fields like

    Missing fields like "UserLabel" and "Sequence Row Level Name"

    Hello, I am using this nice Python API to access raw file metadata. Although I can easily obtain sample_information.user_text, I cannot extract "UserLabel" or "Sequence Row Level Name". Do I need to add these .NET <=> Python wrappers myself?

    opened by keesh0 7
  • Problem with select_instrument()

    Problem with select_instrument()

    I have problems with the raw_file.select_instrument(Device.MS, 1) line using Python 3.10 on Windows 10.

    File loads properly and lines such as raw_file.instrument_count and other sample-related functionality such as raw_file.sample_information.vial and others from https://github.com/ethz-institute-of-microbiology/fisher_py/blob/main/examples/raw_file_reader_example.py work well, but not those that directly refer to the instrument access, like raw_file.get_instrument_data().name

    Instead of the intended functionality, there is an AssertionError for line 893 in https://github.com/ethz-institute-of-microbiology/fisher_py/blob/main/fisher_py/raw_file_reader/raw_file_access.py

    The file I try to open is attached blank219.raw

    blank219.zip

    I just cannot figure out if the problem is with my installation or there is something else wrong?!?!

    opened by schatzsc 5
  • How to access full profile spectrum?

    How to access full profile spectrum?

    Sorry, more a question than an issue - I can access the mass spectra in two ways:

    average_scan = raw_file.average_scans_in_scan_range(first_scan_number, last_scan_number, scan_filter, options)
    spectrum = average_scan.segmented_scan
    for i in range(len(spectrum.positions)):
        print(f'  {i} - {spectrum.positions[i]}, {spectrum.intensities[i]}')
    for i in range(average_scan.centroid_scan.length):
        print(f'  {average_scan.centroid_scan.masses[i]} {average_scan.centroid_scan.intensities[i]}')
    

    and there is also a function that Converts the segmented scan to centroid scan. Used to centroid profile data.

    But how do I access the raw profile data?

    The segmented_scan seems to set values which are below a certain threshold to zero respectively does not show m/z ranges outside the identified peaks but I want the full intensity vs. m/z date pairs for the whole scan range - or is this more of a nomenclature problem and the "segmented scan" is actually the "profile"?!?

    opened by schatzsc 4
  • GetMassListFromScanNum equivalent in Python / dotNET

    GetMassListFromScanNum equivalent in Python / dotNET

    Hello, Not really an issue, but wondering the equivalent call from older C++ interface XRawfile::IXRawfile2Ptr GetMassListFromScanNum() in the new dotNet / Python world? thx e.-

    opened by keesh0 4
  • Cannot run multiprocessed on linux

    Cannot run multiprocessed on linux

    When I run a script multiprocessed on Linux I get this telemetry dumper:

    • Assertion at mono-threads.c:499, condition `result' not met ================================================================= Native Crash Reporting ================================================================= Got a SIGABRT while executing native code. This usually indicates a fatal error in the mono runtime or one of the native libraries used by your application. ================================================================= ================================================================= Native stacktrace: ================================================================= 0x7f3f1127665d - /home/tochr/miniconda3/lib/python3.8/site-packages/../../libmonosgen-2.0.so.1 : 0x7f3f112769e9 - /home/tochr/miniconda3/lib/python3.8/site-packages/../../libmonosgen-2.0.so.1 : 0x7f3f11207bc2 - /home/tochr/miniconda3/lib/python3.8/site-packages/../../libmonosgen-2.0.so.1 : 0x7f3f11275ba2 - /home/tochr/miniconda3/lib/python3.8/site-packages/../../libmonosgen-2.0.so.1 : 0x7f3f11f7e3c0 - /lib/x86_64-linux-gnu/libpthread.so.0 : 0x7f3f11c5703b - /lib/x86_64-linux-gnu/libc.so.6 : gsignal 0x7f3f11c36859 - /lib/x86_64-linux-gnu/libc.so.6 : abort 0x7f3f1116fc2f - /home/tochr/miniconda3/lib/python3.8/site-packages/../../libmonosgen-2.0.so.1 : 0x7f3f1144b376 - /home/tochr/miniconda3/lib/python3.8/site-packages/../../libmonosgen-2.0.so.1 : 0x7f3f11465aaf - /home/tochr/miniconda3/lib/python3.8/site-packages/../../libmonosgen-2.0.so.1 : 0x7f3f1146613d - /home/tochr/miniconda3/lib/python3.8/site-packages/../../libmonosgen-2.0.so.1 : monoeg_assertion_message 0x7f3f1146617b - /home/tochr/miniconda3/lib/python3.8/site-packages/../../libmonosgen-2.0.so.1 : 0x7f3f1145ada9 - /home/tochr/miniconda3/lib/python3.8/site-packages/../../libmonosgen-2.0.so.1 : mono_thread_info_attach 0x7f3f113ab1e3 - /home/tochr/miniconda3/lib/python3.8/site-packages/../../libmonosgen-2.0.so.1 : 0x7f3f11f72609 - /lib/x86_64-linux-gnu/libpthread.so.0 : 0x7f3f11d33163 - /lib/x86_64-linux-gnu/libc.so.6 : clone ================================================================= Telemetry Dumper: =================================================================

    This only happens on linux, and happens on multiple independent linux systems. I installed fisher-py with 'conda install -c conda-forge pythonnet' and 'pip install fisher-py' and to repoduce the error:

    from time import sleep from fisher_py import RawFile from multiprocessing import Pool def read_file(file): RawFile(file) sleep(10)

    files = ['/path/to/file1.raw', '/path/to/file2.raw'] with Pool(2) as ps: list(ps.map(read_file, files))

    opened by TjobberTjob 2
  • Disable native thermo peak picking

    Disable native thermo peak picking

    Is there a way to disable the native Thermo peak picking using fisher-py? I know very little of the way this peak picking is implemented natively into the raw files, but looking at the ThermoRawFileParser (https://github.com/compomics/ThermoRawFileParser) an option -p is possible to disable this peak picking, returning, on average, 10+ times more peaks per spectra.

    I would love if there was a way to achieve this using fisher-py, but from looking through the RawFile class and the documentation I can find no such option, is this manageable or completely not feasible with the current fisher-py? image

    opened by TjobberTjob 2
  • Issues with instrument methods when using Mono

    Issues with instrument methods when using Mono

    I tried newer DLLs (OpenMcdf, ThermoFisher.CommonCore.Data, ThermoFisher.CommonCore.RawFileReader) in https://github.com/compomics/ThermoRawFileParser/releases/tag/v1.3.4 and the issue has been fixed. However, looks like the .pdbs are missing...

    opened by abdelq 2
  • Error accessing ScanEvent properties on macOS/Linux

    Error accessing ScanEvent properties on macOS/Linux

    Description

    I encountered a couple of errors when accessing MS/MS RAW files from our proteomics lab using fisher-py 1.0.17 on our CentOS 7.5 cluster, and also on my MacBook. Both of the errors were part of scan_event.py.

    1. When the ms_order property was accessed, we got an error about a missing attribute. The relevant portion of the stack trace is:

        File "/Users/dpsmca/MprcExtractRaw/venv/lib/python3.9/site-packages/fisher_py/raw_file.py", line 71, in __init__
          scan_numbers, rt = self._get_ms_scan_numbers_and_retention_times_(MsOrderType.Ms)
        File "/Users/dpsmca/MprcExtractRaw/venv/lib/python3.9/site-packages/fisher_py/raw_file.py", line 126, in _get_ms_scan_numbers_and_retention_times_
          if scan_event.ms_order != ms_order:
        File "/Users/dpsmca/MprcExtractRaw/venv/lib/python3.9/site-packages/fisher_py/data/scan_event.py", line 296, in ms_order
          return MsOrderType(self._get_wrapped_object_().MsOrder)
      AttributeError: 'IScanEvent' object has no attribute 'MsOrder'
      
    2. When get_reaction() was called, we got a similar error about the Reactions attribute. Stack trace:

        File "/Users/dpsmca/MprcExtractRaw/venv/lib/python3.9/site-packages/fisher_py/raw_file.py", line 81, in __init__
          scan_numbers, filter_masses = self._get_ms2_scan_numbers_and_masses_()
        File "/Users/dpsmca/MprcExtractRaw/venv/lib/python3.9/site-packages/fisher_py/raw_file.py", line 109, in _get_ms2_scan_numbers_and_masses_
          precursor_mass = self._get_scan_filter_precursor_mass_(scan_number)
        File "/Users/dpsmca/MprcExtractRaw/venv/lib/python3.9/site-packages/fisher_py/raw_file.py", line 138, in _get_scan_filter_precursor_mass_
          return scan_event.get_reaction(0).precursor_mass
        File "/Users/dpsmca/MprcExtractRaw/venv/lib/python3.9/site-packages/fisher_py/data/scan_event.py", line 599, in get_reaction
          return Reaction(self._get_wrapped_object_().Reactions[index])
      AttributeError: 'IScanEvent' object has no attribute 'Reactions'
      

    Issues

    • (1) is just a typo, I think -- looking at ThermoFisher.CommonCore.Data.xml, it appears that this should be MSOrder, with a capital S.

    • (2) seems to work if you replace the direct access to Reactions[index] with a call to GetReaction(index).

    I've created a pull request for this: #13

    Thanks!

    opened by dpsmca 1
  • Support for Python.NET 3

    Support for Python.NET 3

    Implicit int-enum conversion is no longer a default. Considering how this project is set up, I've added the necessary codec to do the conversion.

    However, as of this PR's creation, there is a bug upstream in that codec waiting to be fixed and v3 is still in alpha.

    opened by abdelq 1
  • Bug/scan event

    Bug/scan event

    Two fixes for for problems encountered in scan_event.py:

    • get_reaction(index) replaced direct access to Reactions[index] with call to GetReaction(index)
    • ms_order() fixed capitalization typo: was MsOrder, should be MSOrder
    opened by dpsmca 0
  • Update raw_file_reader_example.py

    Update raw_file_reader_example.py

    Currently the two for loops

    for i in range(average_scan.centroid_scan.length):
        print(f'  {average_scan.centroid_scan.masses[0]} {average_scan.centroid_scan.intensities[i]}')
    

    have average_scan.centroid_scan.masses[0] but shouldn't that also be iterated as average_scan.centroid_scan.masses[i] since otherwise for all intensities only the mass of the first scan is shown?!?

    opened by schatzsc 0
Owner
null
JD-backup is an advanced Python script, that will extract all links from a jDownloader 2 file list and export them to a text file.

JD-backup is an advanced Python script, that will extract all links from a jDownloader 2 file list and export them to a text file.

Kraken.snv 3 Jun 7, 2022
EasyBuild is a software build and installation framework that allows you to manage (scientific) software on High Performance Computing (HPC) systems in an efficient way.

EasyBuild is a software build and installation framework that allows you to manage (scientific) software on High Performance Computing (HPC) systems in an efficient way.

EasyBuild community 87 Dec 27, 2022
App and Python library for parsing, writing, and validation of the STAND013 file format.

python-stand013 python-stand013 is a Python app and library for parsing, writing, and validation of the STAND013 file format. Features The following i

Oda 3 Nov 9, 2022
A python script for combining multiple native SU2 format meshes into one mesh file for multi-zone simulations.

A python script for combining multiple native SU2 format meshes into one mesh file for multi-zone simulations.

MKursatUzuner 1 Jan 20, 2022
A code to clean and extract a bib file based on keywords.

These are two scripts I use to generate clean bib files. clean_bibfile.py: Removes superfluous fields (which are not included in fields_to_keep.json)

Antoine Allard 4 May 16, 2022
Python with the scientific stack, compiled to WebAssembly.

Pyodide may be used in any context where you want to run Python inside a web browser.

null 9.5k Jan 9, 2023
freeCodeCamp Scientific Computing with Python Project for Certification.

Time_Calculator_freeCodeCamp freeCodeCamp Scientific Computing with Python Project for Certification. Write a function named add_time that takes in tw

Rajdeep Mondal 1 Dec 23, 2021
Fetch data from an excel file and create HTML file

excel-to-html Problem Statement! - Fetch data from excel file and create html file Excel.xlsx file contain the information.in multiple rows that is ne

Vivek Kashyap 1 Oct 25, 2021
Scientific color maps and standardization tools

Scicomap is a package that provides scientific color maps and tools to standardize your favourite color maps if you don't like the built-in ones. Scicomap currently provides sequential, bi-sequential, diverging, circular, qualitative and miscellaneous color maps. You can easily draw examples, compare the rendering, see how colorblind people will perceive the color maps. I will illustrate the scicomap capabilities below.

Thomas Bury 14 Nov 30, 2022
Scitizen - Help scientific research for the benefit of mankind and humanity 🔬

Scitizen - Help scientific research for the benefit of mankind and humanity ?? Scitizen has been built from the ground up to give everyone the possibi

Pierre CORBEL 21 Mar 8, 2022
Module for remote in-memory Python package/module loading through HTTP/S

httpimport Python's missing feature! The feature has been suggested in Python Mailing List Remote, in-memory Python package/module importing through H

John Torakis 220 Dec 17, 2022
Viewflow is an Airflow-based framework that allows data scientists to create data models without writing Airflow code.

Viewflow Viewflow is a framework built on the top of Airflow that enables data scientists to create materialized views. It allows data scientists to f

DataCamp 114 Oct 12, 2022
Python-geoarrow - Storing geometry data in Apache Arrow format

geoarrow Storing geometry data in Apache Arrow format Installation $ pip install

Joris Van den Bossche 11 Mar 3, 2022
MiniJVM is simple java virtual machine written by python language, it can load class file from file system and run it.

MiniJVM MiniJVM是一款使用python编写的简易JVM,能够从本地加载class文件并且执行绝大多数指令。 支持的功能 1.从本地磁盘加载class并解析 2.支持绝大多数指令集的执行 3.支持虚拟机内存分区以及对象的创建 4.支持方法的调用和参数传递 5.支持静态代码块的初始化 不支

keguoyu 60 Apr 1, 2022
About A python based Apple Quicktime protocol,you can record audio and video from real iOS devices

介绍 本应用程序使用 python 实现,可以通过 USB 连接 iOS 设备进行屏幕共享 高帧率(30〜60fps) 高画质 低延迟(<200ms) 非侵入性 支持多设备并行 Mac OSX 安装 python >=3.7 brew install libusb pkg-config 如需使用 g

YueC 124 Nov 30, 2022
Junos PyEZ is a Python library to remotely manage/automate Junos devices.

The repo is under active development. If you take a clone, you are getting the latest, and perhaps not entirely stable code. DOCUMENTATION Official Do

Juniper Networks 623 Dec 10, 2022
Simple control of Thorlabs Elliptec devices from Python.

Elliptec Simple control of Thorlabs Elliptec devices. No docs yet » Get started · Report a bug · Request a feature About The Project ThorLabs Elliptec

David Roesel 8 Sep 22, 2022
A program made in PYTHON🐍 that automatically performs data insertions into a POSTGRES database 🐘 , using as base a .CSV file 📁 , useful in mass data insertions

A program made in PYTHON?? that automatically performs data insertions into a POSTGRES database ?? , using as base a .CSV file ?? , useful in mass data insertions.

Davi Galdino 1 Oct 17, 2022
A Python3 script to decode an encoded VBScript file, often seen with a .vbe file extension

vbe-decoder.py Decode one or multiple encoded VBScript files, often seen with a .vbe file extension. Usage usage: vbe-decoder.py [-h] [-o output] file

John Hammond 147 Nov 15, 2022