Evidence enables analysts to deliver a polished business intelligence system using SQL and markdown.

Overview

evidence-logo

License: MIT

Business Intelligence for Modern Data Teams

Evidence enables analysts to deliver a polished business intelligence system using SQL and markdown.

It offers a modern development workflow for analysts and a polished user experience for decision-makers.

How It Works

An Evidence project is a collection of markdown documents. Whenever you write SQL queries in those documents, Evidence runs that SQL against your data warehouse.

You can reference the results of those queries directly in your markdown document's text, or you can use them to make charts, graphs, and tables using our built-in component library.

Evidence compiles your markdown documents into a fast and reliable website. It tests your queries and keeps the site up-to-date with your data warehouse.

Evidence supports loops, conditionals, parameterized pages, tests and more. It's easy to get started and you can build data products which are both more sophisticated and dramatically easier to use than what you can build with a typical BI tool. how-it-works

Getting Started

Evidence is in alpha. A lot of things will change between now and our stable release.

Installation

Check out the documentation for a complete walk through.

npx degit evidence-dev/template my-project
cd my-project 
npm install 
npm run dev 

Join the Evidence Community

Join our Slack channel to get involved in the community's discussion, share what you're working on, or request features.

Follow us on Twitter to receive the latest updates on Evidence.

Contributing

If you are interested in contributing, please join us on our slack channel, open an issue, or contribute a pull request.

License

Evidence is licensed under the MIT license. See the LICENSE file for licensing information.

Comments
  • Persistent query show/hide states

    Persistent query show/hide states

    As a user in dev mode, it's frustrating that query results collapse every time the page reloads.

    Desired behaviour:

    • The query results are kept as shown / hidden based on what the state was before reload

    Rough approach

    • Store the state of each query in local storage, and read from there on reload

    Questions

    • This seems like a weird implementation of storage - I feel like you typically put svelte store variables in store.js?
    • But since (I think) you want to control each query independently, you'd perhaps need to independently make variables for each query - and you don't know how many there are before the queries are compiled. So I put them in the component instead.

    Next

    • It would seem logical to also include the show/hide state of the query text itself, once we've worked out the best approach for this
    opened by archiewood 19
  • Decide on Chart Architecture

    Decide on Chart Architecture

    📢 Feedback Needed - help us build an amazing chart library!

    We've heard from several community members that they want to see more capabilities built into our chart library. We've also received requests to include support for third-party chart libraries in Evidence (like Vega, Highcharts, Plotly and Chart.js).

    Given the importance of charts to the overall experience of using Evidence, we want to do a thorough assessment of our options for building the best chart library you've ever used.

    We will post our criteria and thoughts in this issue, and hope to hear feedback about:

    • What you would like to see in our chart library (please let us know if your ideas are must-haves or nice-to-haves for you)
    • Any experience you've had with third-party chart libraries (positive or negative!)
    enhancement components 
    opened by hughess 19
  • issue-312 custom formats

    issue-312 custom formats

    You can enable this feature by simply adding a file named sites/example-project/.custom-settings.json with content:

    {
      "panelEnabled": true
    }
    

    Example of how this works right now on the UI (you should be able to apply it by aliasing the format name in query): See https://github.com/evidence-dev/evidence/pull/325#issuecomment-1160726392

    Also, I scoped creeped basic code formatting with prettier. Currently only meant to be run manually using pnpx prettier --write <file-to-be-formatted>. Errors may show up if you have the vscode prettier plugin installed, however this won't cause any build issues or preventing your from making merges.

    opened by ud3sh 16
  • 100 percent stacked bar and area charts

    100 percent stacked bar and area charts

    This PR implements 100% stacked bar and area charts using the type prop (type=stacked100).

    How It Works

    • When you use type=stacked100, a new function called getStackPercentages will calculate the percentage of the group total for each row in the table
    • New columns are appended to the data object: (1) total for each X - sum up y grouped by x, (2) percentage - for each row, the y value as a % of the “total for each X”
    • The chart treats the percentage column as the y-axis for the chart

    Other Changes

    • This PR adds props to add more control to charts:
      • yMax: set the maximum value of the y-axis. Helpful if you are passing your own 100% stacked data to a normal bar chart
      • yAxisLabels and xAxisLabels: allow you to turn on or off labels on the x and y axis

    Current Limitations

    • This only works in BarChart and AreaChart and does not work in composable charts
    • If you use a y-axis title, it gets cut off by bars when using 100% stack
    • If your y column has a format tag included in it (e.g., sales_usd), the column name resulting from the aggregation will append another format tag to the end for percentage (e.g., sales_usd_pct ---> Sales Usd (%))
    opened by hughess 14
  • Update chart size on render finished

    Update chart size on render finished

    I was playing around and found an issue related to chart layout when performing client side navigation

    https://user-images.githubusercontent.com/14029371/199779439-3b1c19a1-0d19-408e-980d-275415adebd1.mov

    I'm new using echarts so not sure if this would be the most idiomatic solution to this problem. Happy to hear your thoughts

    opened by afzalsayed96 11
  • Athena support

    Athena support

    Addresses #472

    Here's a first pass. It will need some documentation and cleanup, and also I haven't been able to get the test to run locally for some reason.

    opened by rabidaudio 10
  • Explicit unicode column names are converted to lowercase when using postgresql

    Explicit unicode column names are converted to lowercase when using postgresql

    Bug Description

    Serbian cyrilic: Опис is converted to опис.

    select description as "Опис"
    from forms f
    

    image

    Serbian latin:

    select description as "Šuma"
    from forms f
    

    image

    Severity

    • [x] Low - I'm able to keep using Evidence as normal, but flagging this for the team to fix

    Expected Behavior

    Workarounds There doesn't seem to be workaround

    Environment Information Operating System: Windows 10 Node version (node -v): 19.1.0 npm version (npm -v): 8.9.13 Package versions (npm list --depth=0): `@evidence-dev/[email protected]

    Database:

    • [x] Postgres
    bug 
    opened by majkinetor 9
  • [Feature]: Adds support for scrolling on datatable

    [Feature]: Adds support for scrolling on datatable

    This is a quick experiment to show how UX on QueryDataTable can be improved by listening to vertical wheel event on table and updating the rows rendered accordingly.

    https://user-images.githubusercontent.com/14029371/202231411-9e3c85ba-924c-438c-9a69-21370a21bdbe.mov

    opened by afzalsayed96 9
  • BigValue Component

    BigValue Component

    Adds a beloved BI feature: the Big Number Viz™ to Evidence.

    Includes:

    • Number
    • Sparkline
    • Comparison
    • Comparison is configurable for "down is good"
    • Multiple Big Numbers stack into a row
    • Smart enough defaults that only the data prop is required

    BigValue Example

    opened by mcrascal 9
  • add separate download data option to charts

    add separate download data option to charts

    Description

    Resolves #528

    Hopefully, this is what was requested.

    • DownloadData now has text as prop ("Download Data", "Save Image", etc). Might be good to rename (DownloadButton)
    • Data added as prop to ECharts component
    • Echarts now has download option for Chart PNG and CSV data

    Before:

    Screen Shot 2022-12-14 at 7 47 36 PM

    After:

    https://user-images.githubusercontent.com/14843458/207746387-3914617d-e623-474d-91bd-a65443d3fd4f.mp4

    opened by yukseltron 8
  • Postgres Connector Case Normalization Fix

    Postgres Connector Case Normalization Fix

    Context:

    • User reported difficult connecting to Cube.js via their Postgres SQL API
    • Cube.js compiles queries, and then passes the results back to the query engine.
    • Cube.js by default uses camel case for the names of the fields that are passed back

    Problem

    • The postgres connector lowercases all of the field names: myColumn -> mycolumn
    • But the evidenceType column name is not lowercased
    • This means that if when Evidence looks for the type of mycolumn it cannot find it.

    Result: image

    Proposed fix

    • Lowercase the field name for the column types object also
    opened by archiewood 8
  • The PDF generated will only contain the columns visible in the UI when the table contains multiple columns. Only the first few columns are displayed when a scroll bar is present in the resulted table.

    The PDF generated will only contain the columns visible in the UI when the table contains multiple columns. Only the first few columns are displayed when a scroll bar is present in the resulted table.

    The PDF generated will only contain the columns visible in the UI when the table contains multiple columns. Only the first few columns are displayed when a scroll bar is present in the resulted table. Displayissue1

    opened by arunqa 4
  • Patch fix for 100% stacked charts

    Patch fix for 100% stacked charts

    Description

    This PR adds a patch fix to 100% stacked charts to avoid them erroring out on changes to the data object or page.

    Here is the rough approach to building 100% stacked charts:

    • In Chart.svelte, we receive the y column(s) as inputted by the user (e.g., sales)
    • The total of the y column is calculated
    • Each y column value is divided by the y column total to get the percentage for use in the stack
    • Once the stacked percentages are calculated, the y variable name is overwritten as y + "_pct" (e.g., sales_pct)

    The problem appears when the chart re-runs reactively (either from a change to the data object or to the page):

    • Chart.svelte's script tag re-runs, but the y variable has not been reset to the original name supplied by the user (e.g., it still shows up as sales_pct)
    • Chart.svelte runs a check of the inputted columns and throws an error when it can't find sales_pct in the dataset

    Naive Fix

    The naive fix to this is to remove "_pct" from the y variable so Chart.svelte can find it in the dataset. We only want to do this if it's a 100% stacked chart and if it's not the first run of the chart.

    This PR adds a new variable originalRun which is true when the chart loads, and is set to false when the y variable is overwritten. A check has been added in Chart.svelte which will reset the y variable when it finds originalRun===false.

    This is a bandaid fix to get 100% stacked charts working. The fix doesn't negatively impact our other charts, but it doesn't solve the root cause of the issue, which is that the y variable is not reset to the original value supplied by the user when 100% stacked charts re-run reactively.

    Checklist

    opened by hughess 2
  • Exported dates should be in ISO format

    Exported dates should be in ISO format

    Bug Description When downloading data in Evidence, date columns are exported in full javascript date object format. This makes them difficult to use in other tools.

    We should export dates in ISO format (YYYY-MM-DD) so the dates are usable.

    Severity

    • [x] Low - I'm able to keep using Evidence as normal, but flagging this for the team to fix

    Error Messages and Screenshots CleanShot 2022-12-23 at 14 46 57@2x

    Details

    • This is caused by the date parsing logic used across the product - we need to convert dates into javascript date objects to standardize the treatment of timezones across databases and browsers
    • In the date parsing logic, we may be able to convert the dates back into strings - either within the parsing logic itself, or within the component where data will be exported
    bug 
    opened by hughess 0
  • Improved DataTable

    Improved DataTable

    Description

    This PR is the first step in implementing improved tables in Evidence (see #468). It covers the following improvements to our DataTable component:

    • Horizontal scroll: if you have more columns than the page width can display, the table with automatically allow for horizontal scroll (rather than truncate the column contents as it currently does)

    • Column selection: select specific columns to include in your table by using the new Column component like so:

      <DataTable data={my_query}>
         <Column id=date/>
         <Column id=country/>
         <Column id=sales/>
      </DataTable>
      
      • If you don't select any columns, the table will include all columns in the query result:
      <DataTable data={my_query}/>
      
    • Column-specific settings: using props within the Column component, you can make decisions on a column-by-column basis for the following options (more to follow in the future):

      • title: override the title of the column
      • align: left / center / right align the contents and header of the column
    • New pagination approach: the data scrubber in DataTable has been replaced by page numbers and buttons shown at the bottom of the table

    • Sortable Columns: you can sort columns by clicking on the column header (using sortable=true; it is on by default)

    • Search: using search=true, you can add a search bar to the top of your table. This is off by default to avoid cluttering your page

    • Updated data download: when you click the download button at the bottom right of your table, it will export only the columns you've supplied to the table through the Column components - we're hoping this will be helpful in cases where you have many columns in your query and only want to export what's visible to your readers

    • Updated table styling options:

      • rowNumbers: true | false - turns on or off row index numbers (off by default)
      • rowLines: true | false - turns on or off borders at the bottom of each row (on by default)
      • rowShading: true | false - shades every second row in light grey (off by default)

    Examples

    Sample Table

    datatable-all

    Interactive Demonstration

    datatable-interactive

    Development Demonstration

    datatable-development

    Checklist

    • [x] For UI or styling changes, I have added a screenshot or gif showing before & after
    • [x] I have added a changeset

    Issues with this PR

    • Table generation code: there is likely a way to implement the table generation logic more concisely.
      • The problem with this implementation is it requires a lot of duplication to cover these two scenarios:
        1. Columns are passed explicitly through Column components
        2. No columns are passed (fallback to full data object)
      • I tried preparing all data in the script tag before generating the table (removing the duplicated logic in the main Svelte code), but this caused the table to flash in an unfinished state when each page was loaded
    • Error handling: errors currently do not get handled in an ErrorChart as in the previous DataTable. This is because the error needs to be passed from the Column component to the DataTable, and this will take a bit more effort to get working
    • Sort icon styling:
      • Ideally, the column sort icons would be vertically centered
      • We may need an icon to indicate that the columns are sortable (e.g., show both up and down errors when not in a sorted state). I left these out to keep the table headers looking clean, but it might be required for usability
      • Columns shift quite a bit when the sort button is clicked, since column widths are determined by the content visible on the page
    • Pagination styling: alignment of elements needs a bit of work here. This is not an issue now, but will be in the future when we want to offer more font size/density options within tables
    • Data exporting
      • Date format: dates are exported in a format that is not usable in other tools. It is currently using javascript Date object formatting, but an ISO string would be better. This can either be fixed in the table or (preferably) in the date parsing logic implemented throughout Evidence
      • Overridden column names: overridden column names are not included in data export (only raw sql column names)
    • Search behaviour:
      • Works across all columns: search works across all columns of the underlying dataset. This may produce unexpected behaviour, as you can search within columns that are not visible in the table
      • Jumping content: When you navigate to a page of data with fewer records, content from lower down on the page jumps up to fill the gap left by the missing records
    • Mobile search experience: the automatic zoom-in when typing in the search bar on mobile makes the experience worse. Fix for this is increasing the font size until zoom is not required, but the required font size makes the search bar stand out from the contents of the page too much
    • Print formatting: print formatting can be a lot better (e.g., pagination buttons remain in print version)
      • A stopgap solution here could be to display a nicer looking “X of Y records” message rather than the pagination buttons
    • Search icon styling: the search icon in development mode appears in a light grey (as expected), but it appears black in production (see vercel dev-workspace preview for an example)
    opened by hughess 2
  • Null series in chart does not appear in legend

    Null series in chart does not appear in legend

    Bug Description In a multi-series chart, a null series will not appear in the legend. On a tooltip, the series will be given a sequenced name like "series1"

    Severity

    • [x] Low - I'm able to keep using Evidence as normal, but flagging this for the team to fix

    Expected Behavior Would expect to see a series called null in the legend, and for that series to appear as null in the tooltip.

    Error Messages and Screenshots CleanShot 2022-12-21 at 08 03 45@2x

    bug 
    opened by hughess 0
Releases(@evidence-dev/[email protected])
Amundsen is a metadata driven application for improving the productivity of data analysts, data scientists and engineers when interacting with data.

Amundsen is a metadata driven application for improving the productivity of data analysts, data scientists and engineers when interacting with data.

Amundsen 3.7k Jan 3, 2023
Utilize data analytics skills to solve real-world business problems using Humana’s big data

Humana-Mays-2021-HealthCare-Analytics-Case-Competition- The goal of the project is to utilize data analytics skills to solve real-world business probl

Yongxian (Caroline) Lun 1 Dec 27, 2021
Data cleaning tools for Business analysis

Datacleaning datacleaning tools for Business analysis This program is made for Vicky's work. You can use it, too. 数据清洗 该数据清洗工具是为了商业分析 这个程序是为了Vicky的工作而

Lin Jian 3 Nov 16, 2021
Pipetools enables function composition similar to using Unix pipes.

Pipetools Complete documentation pipetools enables function composition similar to using Unix pipes. It allows forward-composition and piping of arbit

null 186 Dec 29, 2022
A collection of learning outcomes data analysis using Python and SQL, from DQLab.

Data Analyst with PYTHON Data Analyst berperan dalam menghasilkan analisa data serta mempresentasikan insight untuk membantu proses pengambilan keputu

null 6 Oct 11, 2022
WAL enables programmable waveform analysis.

This repro introcudes the Waveform Analysis Language (WAL). The initial paper on WAL will appear at ASPDAC'22 and can be downloaded here: https://www.

Institute for Complex Systems (ICS), Johannes Kepler University Linz 40 Dec 13, 2022
Data Intelligence Applications - Online Product Advertising and Pricing with Context Generation

Data Intelligence Applications - Online Product Advertising and Pricing with Context Generation Overview Consider the scenario in which advertisement

Manuel Bressan 2 Nov 18, 2021
Hue Editor: Open source SQL Query Assistant for Databases/Warehouses

Hue Editor: Open source SQL Query Assistant for Databases/Warehouses

Cloudera 759 Jan 7, 2023
Bearsql allows you to query pandas dataframe with sql syntax.

Bearsql adds sql syntax on pandas dataframe. It uses duckdb to speedup the pandas processing and as the sql engine

null 14 Jun 22, 2022
A data structure that extends pyspark.sql.DataFrame with metadata information.

MetaFrame A data structure that extends pyspark.sql.DataFrame with metadata info

Invent Analytics 8 Feb 15, 2022
songplays datamart provide details about the musical taste of our customers and can help us to improve our recomendation system

Songplays User activity datamart The following document describes the model used to build the songplays datamart table and the respective ETL process.

Leandro Kellermann de Oliveira 1 Jul 13, 2021
A computer algebra system written in pure Python

SymPy See the AUTHORS file for the list of authors. And many more people helped on the SymPy mailing list, reported bugs, helped organize SymPy's part

SymPy 9.9k Dec 31, 2022
PyPSA: Python for Power System Analysis

1 Python for Power System Analysis Contents 1 Python for Power System Analysis 1.1 About 1.2 Documentation 1.3 Functionality 1.4 Example scripts as Ju

null 758 Dec 30, 2022
A forecasting system dedicated to smart city data

smart-city-predictions System prognostyczny dedykowany dla danych inteligentnych miast Praca inżynierska realizowana przez Michała Stawikowskiego and

Kevin Lai 1 Nov 8, 2021
Galvanalyser is a system for automatically storing data generated by battery cycling machines in a database

Galvanalyser is a system for automatically storing data generated by battery cycling machines in a database, using a set of "harvesters", whose job it

Battery Intelligence Lab 20 Sep 28, 2022
PySpark bindings for H3, a hierarchical hexagonal geospatial indexing system

h3-pyspark: Uber's H3 Hexagonal Hierarchical Geospatial Indexing System in PySpark PySpark bindings for the H3 core library. For available functions,

Kevin Schaich 12 Dec 24, 2022
A Pythonic introduction to methods for scaling your data science and machine learning work to larger datasets and larger models, using the tools and APIs you know and love from the PyData stack (such as numpy, pandas, and scikit-learn).

This tutorial's purpose is to introduce Pythonistas to methods for scaling their data science and machine learning work to larger datasets and larger models, using the tools and APIs they know and love from the PyData stack (such as numpy, pandas, and scikit-learn).

Coiled 102 Nov 10, 2022
🧪 Panel-Chemistry - exploratory data analysis and build powerful data and viz tools within the domain of Chemistry using Python and HoloViz Panel.

???? ??. The purpose of the panel-chemistry project is to make it really easy for you to do DATA ANALYSIS and build powerful DATA AND VIZ APPLICATIONS within the domain of Chemistry using using Python and HoloViz Panel.

Marc Skov Madsen 97 Dec 8, 2022
Created covid data pipeline using PySpark and MySQL that collected data stream from API and do some processing and store it into MYSQL database.

Created covid data pipeline using PySpark and MySQL that collected data stream from API and do some processing and store it into MYSQL database.

null 2 Nov 20, 2021