Pandas DataFrames and Series as Interactive Tables in Jupyter

Overview

Pandas DataFrames and Series as Interactive Tables in Jupyter

Pypi CI codecov.io Language grade: Python Code style: black Binder Star

Turn pandas DataFrames and Series into interactive datatables in both your notebooks and their HTML representation with import itables.interactive:

Quick start

Install the package with

pip install itables

Activate the interactive mode for all series and dataframes with

from itables import init_notebook_mode
init_notebook_mode(all_interactive=True)
import world_bank_data as wb

df = wb.get_countries()
df

You don't see any table above? Please either open the HTML export of this notebook, or run this README on Binder!

Or display just one series or dataframe as an interactive table with the show function.

from itables import show

x = wb.get_series("SP.POP.TOTL", mrv=1, simplify_index=True)
show(x)

Advanced usage

Pagination

How many rows per page

Select how many entries should appear at once in the table with either the lengthMenu argument of the show function, or with the global option itables.options.lengthMenu:

import itables.options as opt

opt.lengthMenu = [2, 5, 10, 20, 50, 100, 200, 500]
df

Show the table in full

Show the table in full with the paging argument, either in the show method, or in the options:

show(df.head(), paging=False)

Scroll

If you prefer to replace the pagination with a vertical scroll, use for instance

show(df, scrollY="200px", scrollCollapse=True, paging=False)

Table and cell style

Select how your table should look like with the classes argument of the show function, or by changing itables.options.classes. For the list of possible values, see datatables' default style and the style examples.

opt.classes = ["display", "nowrap"]
df
opt.classes = ["display", "cell-border"]
df

Float precision

Floats are rounded using pd.options.display.float_format. Please change that format according to your preference.

import math
import pandas as pd

with pd.option_context("display.float_format", "{:,.2f}".format):
    show(pd.Series([i * math.pi for i in range(1, 6)]))

You may also choose to convert floating numbers to strings:

with pd.option_context("display.float_format", "${:,.2f}".format):
    show(pd.Series([i * math.pi for i in range(1, 6)]))

Advanced cell formatting

Datatables allows to set the cell or row style depending on the cell content, with either the createdRow or createdCell callback. For instance, if we want the cells with negative numbers to be colored in red, we can use the columnDefs.createdCell argument as follows:

show(
    pd.DataFrame([[-1, 2, -3, 4, -5], [6, -7, 8, -9, 10]], columns=list("abcde")),
    columnDefs=[
        {
            "targets": "_all",
            "createdCell": """function (td, cellData, rowData, row, col) {
      if ( cellData < 0 ) {
        $(td).css('color', 'red')
      }
    }""",
        }
    ],
)

Column width

For tables that are larger than the notebook, the columnDefs argument allows to specify the desired width. If you wish you can also change the default in itables.options.

show(x.to_frame().T, columnDefs=[{"width": "120px", "targets": "_all"}])

Cell alignment

You can use the datatables.net cell classes like dt-left, dt-center, dt-right etc to set the cell alignment. Specify it for one table by using the columnDefs argument of show

show(df, columnDefs=[{"className":"dt-center",  "targets": "_all"}])

or globally by setting opt.columnDefs:

opt.columnDefs = [{"className":"dt-center", "targets": "_all"}]
df
del opt.columnDefs

HTML in cells

bold", "italic", 'link', ], name="HTML", ), paging=False, ) ">
import pandas as pd

show(
    pd.Series(
        [
            "bold",
            "italic",
            'link',
        ],
        name="HTML",
    ),
    paging=False,
)

Select rows

Not currently implemented. May be made available at a later stage using the select extension for datatables.

Copy, CSV, PDF and Excel buttons

Not currently implemented. May be made available at a later stage thanks to the buttons extension for datatable.

Downsampling

When the data in a table is larger than maxBytes, which is equal to 64KB by default, itables will display only a subset of the table - one that fits into maxBytes. If you wish, you can deactivate the limit with maxBytes=0, change the value of maxBytes, or similarly set a limit on the number of rows (maxRows, defaults to 0) or columns (maxColumns, defaults to pd.get_option('display.max_columns')).

Note that datatables support server-side processing. At a later stage we may implement support for larger tables using this feature.

df = wb.get_indicators().head(500)
opt.maxBytes = 10000
df.values.nbytes
df

To show the table in full, we can modify the value of maxBytes either locally:

show(df, maxBytes=0)

or globally:

opt.maxBytes = 2**20
df

References

DataTables

  • DataTables is a plug-in for the jQuery Javascript library. It has a great documentation, and a large set of examples.
  • The R package DT uses datatables.net as the underlying library for both R notebooks and Shiny applications. In addition to the standard functionalities of the library (display, sort, filtering and row selection), RStudio seems to have implemented cell edition.
  • Marek Cermak has an interesting tutorial on how to use datatables within Jupyter. He also published jupyter-datatables, with a focus on numerical data and distribution plots.

Alternatives

ITables uses basic Javascript, and because of this it will only work in Jupyter Notebook, not in JupyterLab. It is not a Jupyter widget, which means that it does not allows you to edit the content of the dataframe.

If you are looking for Jupyter widgets, have a look at

If you are looking for a table component that will fit in Dash applications, see datatable by Dash.

Contributing

I think it would be very helpful to have an identical table component for both Jupyter and Dash. Please let us know if you are interested in drafting a new table component based on an existing Javascript library for Dash.

Also, if you happen to prefer another Javascript table library (say, ag-grid), and you would like to see it supported in itables, please open either an issue or a PR, and let us know what is the minimal code to display a table in Jupyter using your library.

" rel="nofollow">https://buttons.github.io/buttons.js">
Comments
  • Offline mode

    Offline mode

    At the moment itables does not have an offline mode. While the table data is embedded in the notebook, the jquery and datatables.net are loaded from a CDN, see require.config and table template, so an internet connection is required to display the tables.

    Is there a way to add offline usage?

    opened by BBassi 52
  • first very naive version of integrating buttons

    first very naive version of integrating buttons

    This is a very naive version that integrates basic functionalities for buttons' extentions. I don't expect this to be merged, just give me guidance for getting this integrated.

    • experiments/buttons/buttons.ipynb contains an example of using buttons.
    • itables/init.py is currently responsible of calling load_datatables() with the new "required" modules/css. This should be located elsewhere for the user to specify. Where do you want me to put that?
    opened by jpbarrette 10
  • Reinventing the wheel: jupyter-datatables package

    Reinventing the wheel: jupyter-datatables package

    Hello there,

    I like that there is a demand for this feature and that I am not the only one who sees that :). However, the functionality is really being duplicated here, which is perhaps shame and unnecessary reinventing of the wheel. Also, it is a great display of my failure in marketing the package properly :)

    Feel free to check out the existing Jupyter DataTables

    Jupyter DataTables preview

    I'd very much appreciate if you fed back to me about the features that you miss in the aforementioned package and/or contributed yourself! :)

    Also, Jupyter DataTables only supports Jupyter Notebooks at the moment, so it might be worthwhile to focus the attention to JupyterLab (or any other media you wish to be supported).

    Cheers! Marek

    opened by CermakM 10
  • Offline mode by default

    Offline mode by default

    This is the result of a pair coding session with @fwouts .

    We'd like to

    • Use inline javascript by default (i.e. the jQuery and datatables libraries are injected in the notebook, so there's no need to be online)
    • Allow the user to use his own urls if they prefers to do so (the inline mode increases the size of each notebook by about 700kB)

    TODO

    • [x] Make sure this works in all environments (Jupyter Lab, Jupyter Notebook, NBconvert, Jupyter Book, VS Code, PyCharm) for both inline=True/False
    • [x] Make sure this works reliably (new notebook, notebook executed and reloaded, notebook executed, reloaded and re-executed)
    opened by mwouts 9
  • request an easy way to enable individual column searching

    request an easy way to enable individual column searching

    datatables has support for individual column searching: https://datatables.net/examples/api/multi_filter.html

    I'm not sure if it's possible with itables currently to add the appropriate html and javascript to make this happen. It would be nice if it was an option (enable_column_search).

    opened by shewless 9
  • Column width stopped working?

    Column width stopped working?

    Hi, and thanks for the great project! When I try to set the column width (as documented here), it doesn't seem to work for me. I tried it with code similar to what you wrote in https://github.com/mwouts/itables/issues/5#issuecomment-549679169 but it seems to have no effect:

    image

    I got the same result in both jupyter lab and the classic notebook. I'm using Python 3.8.15 and the following:

    itables                  1.4.1
    jupyter_client           7.4.8
    jupyter_core             5.1.0
    jupyter-events           0.5.0
    jupyter_server           2.0.0
    jupyter_server_terminals 0.4.2
    jupyterlab               3.5.1
    jupyterlab-pygments      0.2.2
    jupyterlab_server        2.16.3
    jupyterlab-widgets       1.1.1
    

    I hope this is enough details to reproduce, but if you can't reproduce it, please let me know and I'll try to reproduce it in docker.

    Thanks!

    opened by infokiller 8
  • Leveraging panda's to_html() function

    Leveraging panda's to_html() function

    Hello it seems that itable doesn't support displaying dataframes with multiindex, whereas datatables could support it. This seems to come from the fact that _make_header function only works for non-multiindex columns. However it seems that Pandas's dataframe to_html() function is able to correctly render multiindex columns, using much more complex logic. It also seems that using the raw datatables lib over the html pandas dataframe will indeed correctly convert the dataframe.

    So, my question is the following: would it be possible to add a functionality so that the "base table HTML" for the dataframe is not generated with itables code but with pandas's to_html code ? This would also allow that styling defined in pandas is preserved

    thanks!

    opened by vincent-grosbois 8
  • It only works on a brand new notebook

    It only works on a brand new notebook

    When I test following codes on a new created jupyternote book, it works. however, once the note book is saved and reopen again, it no longer works. (the df table not shown, only the column headers are displayed).

    Tried clear browser cache, restart kernal, restart pc, cant solve the problem. Any one has any idea on this?

    Test code: import itables.interactive import world_bank_data as wb

    df = wb.get_countries() df

    opened by GZR5876 7
  • Column width were not right and strings were overlapping

    Column width were not right and strings were overlapping

    Thank you for working on this package.

    I could successfully render the breast_cancer data frame on jupyter notebook and I could successfully convert data tables to static html.

    However, after convert to static html, the column width were not right and the column names were overlapping to each other.

    Here are my print screens. Screen Shot 2019-06-03 at 13 47 50

    Save Notebook Widget State Screen Shot 2019-06-03 at 11 18 25

    Download as HTML(.html) Screen Shot 2019-06-03 at 11 18 35

    It's gone Screen Shot 2019-06-03 at 13 48 29

    opened by orcahmlee 7
  • itables with Colab dark theme

    itables with Colab dark theme

    Thanks so much for the Colab support! Works great.

    One issue I have is that in Colab dark theme (Tools -> Settings -> Theme: dark) the itable does not fully adapt. Its background remains gray but the text color inherits the (very similar) gray text color from the Colab theme, so the numbers are hard to see. Additionally, the page buttons on the bottom only show up for real when you hover over them (they somehow keep the black text color against the dark theme background). I'm guessing for the same reason. Here's a screenshot: screenshot_itables

    Is there a way to change the theme of itables to dark? Or maybe manually changing the displayed text color to black, so even if the tables are light theme the data is visible?

    Thanks!

    opened by ofir-reich 6
  • Export itables rendering

    Export itables rendering

    Hi, Is it possible to export itables rendering to an html file? If so, how?

    The main application for me would be to integrate it in Reveal.js presentations.

    opened by adrienpacifico 5
  • ITables 2.0 - update datatables.net to 1.13.1 and provide support for buttons

    ITables 2.0 - update datatables.net to 1.13.1 and provide support for buttons

    @AllanJard has released a new version of datatables.net==1.13.1 that provides support for the datables plugins in the ESM format.

    I will report here on the progress on transitioning itables to that new release of datatables.net.

    opened by mwouts 4
  • [Feature] Dark Mode support for itables

    [Feature] Dark Mode support for itables

    Thx for itables

    In the case of using itables with dark themes, like VSCode + Dracula, it is rather difficult to see the buttons.

    image

    I am wondering if there is a way to change the buttons and text to white for example?

    opened by Anselmoo 27
  • Numeric > search

    Numeric > search

    Hi @joseberlines,

    Great work, simple and effective!

    I have two requests (happy to work on them)…

    1. I was wondering if it would be possible/easy to implement regex search for numeric? It “great than x”, “less than y” etc?

    2. is it possible to color each cell as a function of it values heat map to highlight extremes.

    Thanks again for all your work.

    opened by beebeed 1
  • jQuery import

    jQuery import

    Hi,

    I try to integrate your awesome package into Pretty Jupyter. I noticed that itables loads the jQuery without a possibility to overload this settings. This is a bit problematic for me because in the exported HTML file I use another jQuery, which is overwritten by this import. I hotfixed it with jQuery.noConflict() for now, however a better solution is needed.

    Some possible solutions that occur to me:

    • Allow to load not load jQuery. Here might be a problem with different IDEs (vscode,...) having or not having jQuery already imported.
    • Load the jQuery only if there is none yet loaded.
    • Avoid global variable $ and use your own named version. It might also be necessary to force this onto DataTables.

    I would like also to ask if there is a reliable way to detect whether the jQuery was imported. I hacked some selector ($(".output_subarea.output_javascript"), but i don't know if this will hold in future releases. Some IDd element would be nice, but I didn't find any.

    opened by JanPalasek 5
  • WIP Support DataTables plugins

    WIP Support DataTables plugins

    It would be great to support

    • [X] Search highlighting
    • [ ] Buttons
    • [ ] And the select extension.

    Surprisingly, the search highlighting works! Thank you @qqdaiyu55 for suggesting that extension at #63, if you'd like to you can try the development version (documented at datatables_plugins.md with

    pip install git+https://github.com/mwouts/itables.git@datatables_plugins
    
    opened by mwouts 2
Releases(v1.4.2)
  • v1.4.2(Dec 23, 2022)

  • v1.4.1(Dec 4, 2022)

    1.4.1 (2022-12-04)

    Fixed

    • We have added setuptools.build_meta as the build backend in pyproject.toml (#142)
    • We have fixed a typo in itables.options.style

    Changed

    • We have updated the development status of the project to Production/Stable
    Source code(tar.gz)
    Source code(zip)
  • v1.4.0(Dec 4, 2022)

    Fixed

    • We have improved the support for dark themes by using the CSS from datatables.net in version 1.13.1 (#103)
    • We have fixed a compatibility issue with old versions of pandas
    • We have added a test to make sure that timezones are preserved
    • requests was added as a build dependency (#123)
    • and the flake8 pre-commit hook was fixed (#124) - thanks to Anselm Hahn for these two contributions!
    • Duplicated column and index names are supported (#134)

    Added

    • The examples in the documentation are now executed as part of the test suite to increase the coverage.
    • We have added a new caption argument to the show function to make it easier to add captions on tables.

    Changed

    • We have changed the default table to style = "table-layout:auto;width:auto;margin:auto" to fix an issue on the width of index columns (default style was width:auto previously) (#130)
    • The default classes applied to datatables are now ["display", "nowrap"]
    • We have changed the default order to order = [] i.e. we don't sort anymore the table, even when the index is monotonic, to fix an issue in the order of categories (#135)
    • We have set an explicit maxRows = 0 and also increased maxColumns to 200 (instead of Pandas' default at 20).
    Source code(tar.gz)
    Source code(zip)
  • v1.3.5(Nov 12, 2022)

    Fixed

    • We use pandas.io.formats.format.format_array to format non-trivial dtypes (as in itables<=1.3.1) (#112)
    • The downsampling of large tables is faster. We have also added a new function generate_random_df to generate large tables on demand (#113)
    • We don't raise a warning anymore when a table is downsampled. Instead, we add this information to the table summary (#114)

    Added

    • We have added support for Python 2 (#115).
    Source code(tar.gz)
    Source code(zip)
  • v1.3.4(Nov 7, 2022)

  • v1.3.3(Nov 6, 2022)

  • v1.3.2(Nov 6, 2022)

  • v1.3.1(Nov 5, 2022)

  • v1.3.0(Sep 3, 2022)

  • v1.2.0(Aug 14, 2022)

  • v1.1.3(Aug 11, 2022)

  • v1.1.2(Jul 2, 2022)

  • v1.1.1(Jun 23, 2022)

  • v1.1.0(Jun 23, 2022)

    1.1.0 (2022-06-23)

    Added

    • itables.options and the show function have a new column_filters argument to display individual column search boxes (#69)
    • We have documented DataTables' dom option.
    • We have introduced a new class JavascriptFunction to limit the evaluation of Javascript function to selected ones.
    • The documentation is formatted with black thanks to a Jupytext & Black pre-commit hook.
    Source code(tar.gz)
    Source code(zip)
  • v1.0.0.b(Jun 22, 2022)

  • v1.0.0(Jun 22, 2022)

  • v0.4.7(Apr 13, 2022)

  • v0.4.6(Mar 29, 2022)

    0.4.6 (2022-03-29)

    Changed

    • We have removed the default column width at 70 pixels (#61, #62, #66)
    • We now use pyupgrade in our pre-commit hooks

    Fixed

    • We have improved the rendering of multiindex columns (#63)
    Source code(tar.gz)
    Source code(zip)
  • v0.4.5(Jan 24, 2022)

    Changed

    • The itables documentation now uses Jupyter Book (#56)
    • We have added a new style option in itables.options and in show, with a default value equal to max-width:100%.
    Source code(tar.gz)
    Source code(zip)
  • v0.4.4(Jan 10, 2022)

  • v0.4.3(Jan 8, 2022)

    Changed

    • When a JS function is created on the Python side, we export it as-is (without quotes) in the HTML file and don't use JS eval anymore.
    Source code(tar.gz)
    Source code(zip)
  • v0.4.2(Jan 7, 2022)

    Fixed

    • Fix the HTML output when eval_functions=True
    • Display "Loading..." under the table header until the table is displayed with datatables.net
    • init_notebook_mode(all_interactive=False) restores the original Pandas HTML representation
    Source code(tar.gz)
    Source code(zip)
  • v0.4.1(Jan 6, 2022)

  • v0.4.0(Jan 6, 2022)

    Fixed

    • Now itables also works in Jupyter Lab, Colab, Nteract, VS Code and PyCharm (#3, #4, #26, #40), as we load the datatables.net library with an ES import when require.js is not available. Many thanks to François Wouts for his precious help!

    Changed

    • The show function (and itables.options) has a new argument eval_functions. When set to True, the nested strings passed to datatables.net that start with function are converted to Javascript functions.
    • The HTML code for the datatables.net representation of the table is generated with an HTML template.
    • We use f-strings and thus require Python >= 3.6
    Source code(tar.gz)
    Source code(zip)
  • v0.3.1(Dec 24, 2021)

    Fixed

    • We fixed an issue (jquery not found) with the HTML export when using nbconvert>=6.0 (#21)
    • We documented how to change the default ordering of rows - with the order option (#30)
    • We documented how to load require in Jupyter Lab (#3)

    Changed

    • The main branch for the project is main rather than master
    • Updated datatables to 1.11.3 and jquery to 3.5.1
    Source code(tar.gz)
    Source code(zip)
  • v0.3.0(Dec 14, 2020)

    Fixed

    • itables now has an explicit init_notebook_mode function, which inserts the datatables.net library in the notebook. Use init_notebook_mode(all_interactive=True) to display all the pandas object as interactive tables. This fixes (#6) and (#17).

    Changed

    • itables uses GitHub Actions for the CI.

    Added

    • itables is tested with Python 3.9 as well.
    Source code(tar.gz)
    Source code(zip)
  • v0.2.2(Oct 1, 2020)

  • 0.2.1(Nov 21, 2019)

  • 0.2.0(Nov 20, 2019)

    0.2.0 (2019-11-20)

    Added

    • Large tables are downsampled (#2)

    Changed

    • Javascript code moved to Javascript files

    Fixed

    • Tables with many columns are now well rendered (#1, #5)
    Source code(tar.gz)
    Source code(zip)
  • 0.1.0(Apr 23, 2019)

Owner
Marc Wouts
Research Manager at Capital Fund Management
Marc Wouts
Visualize classified time series data with interactive Sankey plots in Google Earth Engine

sankee Visualize changes in classified time series data with interactive Sankey plots in Google Earth Engine Contents Description Installation Using P

Aaron Zuspan 76 Dec 15, 2022
Kats is a toolkit to analyze time series data, a lightweight, easy-to-use, and generalizable framework to perform time series analysis.

Kats, a kit to analyze time series data, a lightweight, easy-to-use, generalizable, and extendable framework to perform time series analysis, from understanding the key statistics and characteristics, detecting change points and anomalies, to forecasting future trends.

Facebook Research 4.1k Dec 29, 2022
Pandas Machine Learning and Quant Finance Library Collection

Pandas Machine Learning and Quant Finance Library Collection

null 148 Dec 7, 2022
Turns your machine learning code into microservices with web API, interactive GUI, and more.

Turns your machine learning code into microservices with web API, interactive GUI, and more.

Machine Learning Tooling 2.8k Jan 2, 2023
Interactive Web App with Streamlit and Scikit-learn that applies different Classification algorithms to popular datasets

Interactive Web App with Streamlit and Scikit-learn that applies different Classification algorithms to popular datasets Datasets Used: Iris dataset,

Samrat Mitra 2 Nov 18, 2021
Interactive Parallel Computing in Python

Interactive Parallel Computing with IPython ipyparallel is the new home of IPython.parallel. ipyparallel is a Python package and collection of CLI scr

IPython 2.3k Dec 30, 2022
A collection of interactive machine-learning experiments: 🏋️models training + 🎨models demo

?? Interactive Machine Learning experiments: ??️models training + ??models demo

Oleksii Trekhleb 1.4k Jan 6, 2023
A python library for easy manipulation and forecasting of time series.

Time Series Made Easy in Python darts is a python library for easy manipulation and forecasting of time series. It contains a variety of models, from

Unit8 5.2k Jan 4, 2023
STUMPY is a powerful and scalable Python library for computing a Matrix Profile, which can be used for a variety of time series data mining tasks

STUMPY STUMPY is a powerful and scalable library that efficiently computes something called the matrix profile, which can be used for a variety of tim

TD Ameritrade 2.5k Jan 6, 2023
Automatically build ARIMA, SARIMAX, VAR, FB Prophet and XGBoost Models on Time Series data sets with a Single Line of Code. Now updated with Dask to handle millions of rows.

Auto_TS: Auto_TimeSeries Automatically build multiple Time Series models using a Single Line of Code. Now updated with Dask. Auto_timeseries is a comp

AutoViz and Auto_ViML 519 Jan 3, 2023
Examples and code for the Practical Machine Learning workshop series

Practical Machine Learning Workshop Series Practical Machine Learning for Quantitative Finance Post conference workshop at the WBS Spring Conference D

CompatibL 21 Jun 25, 2022
An open-source library of algorithms to analyse time series in GPU and CPU.

An open-source library of algorithms to analyse time series in GPU and CPU.

Shapelets 216 Dec 30, 2022
A data preprocessing package for time series data. Design for machine learning and deep learning.

A data preprocessing package for time series data. Design for machine learning and deep learning.

Allen Chiang 152 Jan 7, 2023
MaD GUI is a basis for graphical annotation and computational analysis of time series data.

MaD GUI Machine Learning and Data Analytics Graphical User Interface MaD GUI is a basis for graphical annotation and computational analysis of time se

Machine Learning and Data Analytics Lab FAU 10 Dec 19, 2022
A collection of Scikit-Learn compatible time series transformers and tools.

tsfeast A collection of Scikit-Learn compatible time series transformers and tools. Installation Create a virtual environment and install: From PyPi p

Chris Santiago 0 Mar 30, 2022
A machine learning toolkit dedicated to time-series data

tslearn The machine learning toolkit for time series analysis in Python Section Description Installation Installing the dependencies and tslearn Getti

null 2.3k Jan 5, 2023
Tool for producing high quality forecasts for time series data that has multiple seasonality with linear or non-linear growth.

Prophet: Automatic Forecasting Procedure Prophet is a procedure for forecasting time series data based on an additive model where non-linear trends ar

Facebook 15.4k Jan 7, 2023
Open source time series library for Python

PyFlux PyFlux is an open source time series library for Python. The library has a good array of modern time series models, as well as a flexible array

Ross Taylor 2k Jan 2, 2023
Automatic extraction of relevant features from time series:

tsfresh This repository contains the TSFRESH python package. The abbreviation stands for "Time Series Feature extraction based on scalable hypothesis

Blue Yonder GmbH 7k Jan 6, 2023