Model synchronization from dbt to Metabase.

Overview

dbt-metabase

GitHub Actions PyPI Downloads License: MIT Code style: black

Model synchronization from dbt to Metabase.

If dbt is your source of truth for database schemas and you use Metabase as your analytics tool, dbt-metabase can propagate table relationships, model and column descriptions and semantic types (e.g. currency, category, URL) to your Metabase data model.

Requirements

Requires Python 3.6 or above.

Main features

The main features provided by dbt-metabase are:

  • Parsing your dbt project (either through the manifest.json or directly through the YAML files)
  • Triggering a Metabase schema sync before propagating the metadata
  • Propagating table descriptions to Metabase
  • Propagating columns description to Metabase
  • Propagating columns semantic types and visibility types to Metabase through the use of dbt meta fields
  • Propagating table relationships represented as dbt relationships column tests
  • Extracting dbt model exposures from Metabase and generating YAML files to be included and revisioned with your dbt deployment

Usage

You can install dbt-metabase from PyPI:

pip install dbt-metabase

Basic Example

Let's start by defining a short sample schema.yml as below.

models:
  - name: stg_users
    description: User records.
    columns:
      - name: id
        description: Primary key.
        tests:
          - not_null
          - unique
      - name: email
        description: User's email address.
      - name: group_id
        description: Foreign key to user group.
        tests:
          - not_null
          - relationships:
              to: ref('groups')
              field: id

  - name: stg_groups
    description: User groups.
    columns:
      - name: id
        description: Primary key.
        tests:
          - not_null
          - unique
      - name: name
        description: Group name.

That's already enough to propagate the primary keys, foreign keys and descriptions to Metabase by executing the below command.

dbt-metabase models \
    --dbt_path . \
    --dbt_database business \
    --metabase_host metabase.example.com \
    --metabase_user [email protected] \
    --metabase_password Password123 \
    --metabase_database business \
    --schema public

Check your Metabase instance by going into Settings > Admin > Data Model, you will notice that ID in STG_USERS is now marked as "Entity Key" and GROUP_ID is marked as "Foreign Key" pointing to ID in STG_GROUPS.

Exposure Extraction

dbt-metabase also allows us to extract exposures from Metabase. The invocation is almost identical to our models function with the addition of output name and location args. dbt exposures let us understand how our dbt models are exposed in BI which closes the loop between ELT, modelling, and consumption.

dbt-metabase exposures \
    --dbt_manifest_path ./target/manifest.json \
    --dbt_database business \
    --metabase_host metabase.example.com \
    --metabase_user [email protected] \
    --metabase_password Password123 \
    --metabase_database business \
    --output_path ./models/ \
    --output_name metabase_exposures

Once execution completes, a look at the output metabase_exposures.yml will reveal all metabase exposures documented with the documentation, descriptions, creator emails & names, links to exposures, and even native SQL propagated over from Metabase.

exposures:
  - name: Number_of_orders_over_time
    description: '
      ### Visualization: Line

      A line chart depicting how order volume changes over time

      #### Metadata

      Metabase Id: __8__

      Created On: __2021-07-21T08:01:38.016244Z__'
    type: analysis
    url: http://your.metabase.com/card/8
    maturity: medium
    owner:
      name: Indiana Jones
      email: [email protected]
    depends_on:
      - ref('orders')

Questions which are native queries will have the SQL propagated to a code block in the documentation's description for full visibility. This YAML, like the rest of your dbt project can be committed to source control to understand how exposures change over time. In a production environment, one can trigger dbt docs generate after dbt-metabase exposures (or alternatively run the exposure extraction job on a cadence every X days) in order to keep a dbt docs site fully synchronized with BI. This makes dbt docs a useful utility for introspecting the data model from source -> consumption with zero extra/repeated human input.

Reading your dbt project

There are two approaches provided by this library to read your dbt project:

1. Artifacts

The recommended approach is to instruct dbt-metabase to read your manifest.json, a dbt artifact containing the full representation of your dbt project's resources. If your dbt project uses multiple schemas, multiple databases or model aliases, you must use this approach.

Note that you you have to run dbt compile --target prod or any of the other dbt commands listed in the dbt documentation above to get a fresh copy of your manifest.json. Remember to run it against your production target.

When using the dbt-metabase CLI, you must provide a --dbt_manifest_path argument pointing to your manifest.json file (usually in the target/ folder of your dbt project).

2. Direct parsing

The second alternative is to provide the path to your dbt project root folder using the argument --dbt_path. dbt-metabase will then look for all .yml files and parse your documentation and tests directly from there. It will not support dbt projects with custom schemas.

Semantic Types

Now that we have primary and foreign keys, let's tell Metabase that email column contains email addresses.

Change the email column as follows:

- name: email
  description: User's email address.
  meta:
    metabase.semantic_type: type/Email

Once you run dbt-metabase models again, you will notice that EMAIL is now marked as "Email".

Here is the list of semantic types (formerly known as special types) currently accepted by Metabase:

  • type/PK
  • type/FK
  • type/AvatarURL
  • type/Category
  • type/City
  • type/Country
  • type/Currency
  • type/Description
  • type/Email
  • type/Enum
  • type/ImageURL
  • type/SerializedJSON
  • type/Latitude
  • type/Longitude
  • type/Number
  • type/State
  • type/URL
  • type/ZipCode
  • type/Quantity
  • type/Income
  • type/Discount
  • type/CreationTimestamp
  • type/CreationTime
  • type/CreationDate
  • type/CancelationTimestamp
  • type/CancelationTime
  • type/CancelationDate
  • type/DeletionTimestamp
  • type/DeletionTime
  • type/DeletionDate
  • type/Product
  • type/User
  • type/Source
  • type/Price
  • type/JoinTimestamp
  • type/JoinTime
  • type/JoinDate
  • type/Share
  • type/Owner
  • type/Company
  • type/Subscription
  • type/Score
  • type/Title
  • type/Comment
  • type/Cost
  • type/GrossMargin
  • type/Birthdate

If you notice new ones, please submit a PR to update this readme.

Visibility Types

In addition to semantic types, you can optionally specify visibility for each field. This affects whether or not they are displayed in the Metabase UI.

Here is how you would hide that same email:

- name: email
  description: User's email address.
  meta:
    metabase.semantic_type: type/Email
    metabase.visibility_type: sensitive

Here are the visibility types supported by Metabase:

  • normal (default)
  • details-only
  • sensitive
  • hidden (supported but not reflected in the UI)
  • retired (supported but not reflected in the UI)

If you notice new ones, please submit a PR to update this readme.

Model Extra Fields

In addition to the model description, Metabase accepts two extra information fields. Those optional fields are called caveats and points_of_interest and can be defined under the meta tag of the model.

This is how you can specify them in the stg_users example:

- name: stg_users
  description: User records.
  meta:
    metabase.points_of_interest: Relevant records.
    metabase.caveats: Sensitive information about users.

Database Sync

By default, dbt-metabase will tell Metabase to synchronize database fields and wait for the data model to contain all the tables and columns in your dbt project.

You can control this behavior with two arguments:

  • --metabase_sync_skip - boolean to optionally disable pre-synchronization
  • --metabase_sync_timeout - number of seconds to wait and re-check data model before giving up

Configuration

dbt-metabase config

Using the above command, you can enter an interactive configuration session where you can cache default selections for arguments. This creates a config.yml in ~/.dbt-metabase. This is particularly useful for arguments which are repeated on every invocation like metabase_user, metabase_host, metabase_password, dbt_manifest_path, etc.

In addition, there are a few injected env vars that make deploying dbt-metabase in a CI/CD environment simpler without exposing secrets. Listed below are acceptable env vars which correspond to their CLI flags:

  • DBT_DATABASE
  • DBT_PATH
  • DBT_MANIFEST_PATH
  • MB_USER
  • MB_PASS
  • MB_HOST
  • MB_DATABASE

If any one of the above is present in the environment, the corresponding CLI flag is not needed unless overriding the environment value. In the absence of a CLI flag, dbt-metabase will first look to the environment for any env vars to inject, then we will look to the config.yml for cached defaults.

A config.yml can be created or updated manually as well if needed. The only requirement is that it must be located in ~/.dbt-metabase. The layout is as follows:

config:
    dbt_database: reporting
    dbt_manifest_path: /home/user/dbt/target/manifest.json
    metabase_database: Reporting
    metabase_host: reporting.metabase.io
    metabase_user: [email protected]
    metabase_password: ...
    metabase_use_http: false
    metabase_sync: true
    metabase_sync_timeout: null
    dbt_schema_excludes:
      - development
      - testing
    dbt_excludes:
      - test_monday_io_site_diff

Programmatic Invocation

As you have already seen, you can invoke dbt-metabase from the command line. But if you prefer to call it from your code, here's how to do it:

from dbtmetabase.models.interface import MetabaseInterface, DbtInterface

# Instantiate dbt interface
dbt = DbtInterface(
    path=dbt_path,
    manifest_path=dbt_manifest_path,
    database=dbt_database,
    schema=dbt_schema,
    schema_excludes=dbt_schema_excludes,
    includes=dbt_includes,
    excludes=dbt_excludes,
)

# Load models
dbt_models, aliases = dbt.read_models(
    include_tags=dbt_include_tags,
    docs_url=dbt_docs_url,
)

# Instantiate Metabase interface
metabase = MetabaseInterface(
    host=metabase_host,
    user=metabase_user,
    password=metabase_password,
    use_http=metabase_use_http,
    verify=metabase_verify,
    database=metabase_database,
    sync=metabase_sync,
    sync_timeout=metabase_sync_timeout,
)

# Load client
metabase.prepare_metabase_client(dbt_models)

# Propagate models to Metabase
metabase.client.export_models(
    database=metabase.database,
    models=dbt_models,
    aliases=aliases,
)

# Parse exposures from Metabase into dbt schema yml
metabase.client.extract_exposures(
    models=dbt_models,
    output_path=output_path,
    output_name=output_name,
    include_personal_collections=include_personal_collections,
    collection_excludes=collection_excludes,
)

Code of Conduct

All contributors are expected to follow the PyPA Code of Conduct.

Comments
  • Parsing of manifest.json as well as better yaml parsing support / alias support

    Parsing of manifest.json as well as better yaml parsing support / alias support

    Picking up the ball where fernandobrito left off. I have a major enhancement (one of two in the works). Lots of cleaned up code, bug fixes from the other fork, personally tested on my own production replicates and internally at my company, etc. There may be some more changes/commits I add (to readme mostly) but I just wanted to post it to get your eyes and awareness. The other PR will involve auto documenting all exposures revealed in metabase through questions / dashboards and constructing a YML that allows us to sync a DAG with metabase fully including click throughs from dbt docs exposures directly to dashboards.

    opened by z3z1ma 20
  • Exposure Parsing

    Exposure Parsing

    Description

    Reposting because some branch renaming fudged up the PR. This is still ready for review.

    Parsing exposures from Metabase automatically in relation to a dbt project. This function is huge for understanding exposures within your data model. Close the loop between model composition and tangible BI exposures of those models creating an extremely valuable company asset in the dbt docs. In the nature of this project, the aim is we can use almost the same invocation parameters users set up in CI / CD for dbt-metabase export as they can with dbt-metabase exposures. This lets you run the invocation in production to essentially keep you docs in full sync with BI.

    Type of change

    • [x] New feature (non-breaking change which adds functionality)
    • [x] This change requires a documentation update

    How Has This Been Tested?

    Open to ideas but ultimately I think making a mock api from metabase running on top of a database that has been seeded and ran through with the dbt jaffle shop project would be most robust. Just a lot of work.

    • [x] CI test for exposure documentation

    Test Configuration:

    • Python version: 3.8

    Checklist:

    • [x] My code follows the style guidelines of this project
    • [x] I have performed a self-review of my own code
    • [x] I have commented my code, particularly in hard-to-understand areas
    • [x] I have made corresponding changes to the documentation
    • [x] My changes generate no new warnings
    • [x] I have added tests that prove my fix is effective or that my feature works
    • [x] New and existing unit tests pass locally with my changes

    References #22

    opened by z3z1ma 13
  • feat: Allow explicit null semantic types

    feat: Allow explicit null semantic types

    Providing a null metabase.semantic_type explicitly writes it to the API, omitting it leaves the existing one.

    Changes behaviour in #118 and #119

    Resolves #131

    opened by gouline 9
  • feat: Refactor CLI - Simplify Config - Allow Env Vars - Add Config Command

    feat: Refactor CLI - Simplify Config - Allow Env Vars - Add Config Command

    Description

    Large refactor to use click as cli interface, improve and further abstract interface complexity, allow env vars implicitly, improve logging with global logger + rich, add config yaml with interactive setup via config command, load config via hook in CLI, split commands in sub commands, optimize help text, add implicit file / folder existence checking on file / folder args.

    Sub commands broken out:

    image

    Optimized help text

    image

    Interactive config file generation:

    image

    Checking config file contents with simple dbt-metabase config --inspect and running a CLI command with just dbt-metabase models with auto config loading:

    image

    An example config:

    config:
      dbt_database: reporting
      mb_user: alex@email
      ...
      mb_database: Reporting
    

    Type of change

    • [x] New feature (non-breaking change which adds functionality)
    • [x] This change requires a documentation update

    How Has This Been Tested?

    Existing unit tests as well as production use. In addition, this time around using nox I have tested the package on Python 3.6, 3.7 and 3.8

    image

    Test Configuration:

    • Python version: 3.6
    • Python version: 3.7
    • Python version: 3.8

    Checklist:

    • [x] My code follows the style guidelines of this project
    • [x] I have performed a self-review of my own code
    • [x] I have commented my code, particularly in hard-to-understand areas
    • [x] I have made corresponding changes to the documentation
    • [x] My changes generate no new warnings
    • [x] I have added tests that prove my fix is effective or that my feature works
    • [x] New and existing unit tests pass locally with my changes

    Closing Notes

    Env Vars taking precedence over config.yml defaults (via injection) CLI flags taking highest precedence as in a standard CLI tool (overrides)

    opened by z3z1ma 8
  • Command terminates without error

    Command terminates without error

    When I run the following command, the session is established but then the process ends without any message:

    dbt-metabase models \
        --dbt_manifest_path ./target/manifest.json \
        --dbt_schema dbt_prod \
        --metabase_host $url \
        --metabase_user $metabase_email \
        --metabase_password $metabase_pw \
        --metabase_database $dbname \
        --dbt_database dbt_prod
    

    2021-09-14 09:42:07,987 - INFO - Session established successfully

    Am I doing something wrong?

    opened by mmmatthew 8
  • Metrics Parsing

    Metrics Parsing

    Description

    Bring your metrics into revision control, bring them into your data model, sync it directly with Metabase.

    dbt-metabase metrics \
      --dbt_database test \
      --dbt_path tests/fixtures/metric/ \
       --schema public \
       --metabase_host localhost:3000 \
       --metabase_user alex@... \
       --metabase_password "..." \
       --metabase_database unit_testing \
       --metabase_use_http --verbose
    

    Expression syntax: https://www.metabase.com/docs/latest/users-guide/expressions.html

      - name: Number of Customers with Large Orders
        description: Customers who are big spenders should be tracked independently of total,
          any customer who orders over 20 AUD of jaffle is counted
        metric: countif([customer_lifetime_value] > 20)
    

    We will parse: countif([customer_lifetime_value] > 20) into['count-where', ['>', ['field', 41, None], 20]] The parser should be able to handle any type of expression allowing users to use the nice excel like syntax built by metabase team directly alongside your data models. They become centralized, self contained, and gain all the advantages of dbt/jinja.

    image

    Parsing Examples

    Purposefully convoluted examples showing robustness and possibilities (most metrics are simple in theory with preprocessing logic in the model)

    Input: Sum(case([site_dispenser_count] + 1 > 1 or [site_dispenser_count] - 1 > 1, [site_dispenser_count] + 1)) Output: ['sum', ['case', [[['or', ['>', ['+', ['field', 1, 'site_dispenser_count'], 1], 1], ['>', ['-', ['field', 1, 'site_dispenser_count'], 1], 1]], ['+', ['field', 1, 'site_dispenser_count'], 1]]]]]

    Input: Sum([table.order] + [qty] * 2 + 4 + 5 - 4 + 5) Output: ['sum', ['+', ['-', ['+', ['field', 1, 'table.order'], ['*', ['field', 1, 'qty'], 2], 4, 5], 4], 5]]

    Input: SumIf([site_dispenser_count], [site_city] > "Phoenix" or [site_state] = "Arizona") Output: ['sum-where', ['field', 1, 'site_dispenser_count'], ['or', ['>', ['field', 1, 'site_city'], '"Phoenix"'], ['=', ['field', 1, 'site_state'], '"Arizona"']]]

    Input: Distinct(case([site_city] = "Phoenix", [site_panel_count])) / distinct([site_dispenser_count]) Output: ['/', ['distinct', ['case', [[['=', ['field', 1, 'site_city'], '"Phoenix"'], ['field', 1, 'site_panel_count']]]]], ['distinct', ['field', 1, 'site_dispenser_count']]]

    Type of change

    • [x] New feature (non-breaking change which adds functionality)
    • [x] This change requires a documentation update

    How Has This Been Tested?

    • [ ] CI test for metric propagation and synchronization

    Test Configuration:

    • Python version: 3.8

    Checklist:

    • [x] My code follows the style guidelines of this project
    • [x] I have performed a self-review of my own code
    • [x] I have commented my code, particularly in hard-to-understand areas
    • [ ] I have made corresponding changes to the documentation
    • [x] My changes generate no new warnings
    • [ ] I have added tests that prove my fix is effective or that my feature works
    • [x] New and existing unit tests pass locally with my changes

    References #25

    opened by z3z1ma 8
  • Fix sync when models are listed under dbt sources

    Fix sync when models are listed under dbt sources

    #17 introduced a bug by collecting models listed under DBT sources.

    The problem is that sources' tables are typically not stored under the DBT schema but this script only collected Metabase models under the DB schema specified as an argument. In consequence, when a "source" model is loaded, the script won't be able to match it with a Metabase model and the script times out.

    The solution implemented makes the script "schema-aware" by collecting the schema name explicitly for sources. It also indexes all models from Metabase in order to be able to find models outside the DBT Schema.

    The behavior for DBT models outside sources is unchanged. They are assume to be stored under the "schema" passed explicitly as an argument.

    This change is a stepping stone towards addressing #3

    opened by remigabillet 8
  • What's the supported dbt version?

    What's the supported dbt version?

    Hey, thanks for sharing you work on this.

    I'm having some problems using the library to parse my project. Neither the the manifest parser nor the DbtFolderReader is able to parse it without failing. For my project I'm using dbt 0.20.0, is this version supported by this package? The manifest schema has slightly changed the last couple of versions and I'm wondering if you might be working with another version.

    opened by danielsepulvedab 7
  • Add Argument --mb_http

    Add Argument --mb_http

    I've started testing dbt-metabase and I ran into an issue with running dbt-metabase export from the command line with the --mb_https option.

    I am running Metabase locally in a docker container and was getting an SSL error. I found while troubleshooting that I can set mb_https to False when calling the export function from a Python script, but not from the command line.

    I did some research and came up with a solution that adds an argument. I've tested it locally and I'm able to run the export from the command line. I've never made an open-source PR before 🙈 , so please feel free to give me feedback/suggestions! I realize that adding an argument may not be optimal.

    opened by erika-e 7
  • Model meta metabase.display_name not parsed

    Model meta metabase.display_name not parsed

    The export_model function supports setting the display name for models from a display_name property on MetabaseModel.

    However, this isn't accessible, as the meta: metabase.display_name property is not parsed and populated in the construction of the MetabaseModel in _read_model.

    bug 
    opened by willbryant 6
  • Ensures that boolean option flags are overridden by config.yml when no CLI flags are provided

    Ensures that boolean option flags are overridden by config.yml when no CLI flags are provided

    Previously, any values passed in a config.yml file for boolean flags (metabase_use_http, metabase_sync) were ignored.

    This change will override any flags that are of type click.types.BoolParamType with values provided in a config.yml file, even if specified in a command line argument.

    For example, when metabase_use_http is set to True in config.yml, running dbt-metabase models --metabase_https will still default to using HTTP.

    Resolves #99

    opened by jack-cook-repo 6
  • Metabase exposures cause dbt to crash when referencing existing questions / variables because of jinja interpolation

    Metabase exposures cause dbt to crash when referencing existing questions / variables because of jinja interpolation

    This happens when referencing other metabase questions in metabase

    In the metabase exposures yml here is an extract of a problematic line

    exposures:
      - name: my_viz
        description: "### Visualization: Table\n\nSee https://example.com/xx\n\
          \n#### Query\n\n```\nwith \nappointments as (\n  select * FROM `xxxx.fct_appointments`\
          \ \n),\nusers as (\n  select * from {{#161}}\n  where last_active_at_cache >\
          \ timestamp_sub(current_timestamp, interval 365 DAY)\n),\napts as (\n  SELECT\
    

    I believe dbt tries to interpolate the metabase question references as jinja expression, thus causing problems

    jinja2.exceptions.TemplateSyntaxError: unexpected char '#' at 276
      line 13
        select * from {{#161}}
    

    This is likely to cause problems with every metabase custom variable/reference wrapped under {{}}

    Is there a way to either post-process the exposures.yml to escape those characters, or find a way to disable dbt jinja interpolation for metabase exposures ?

    bug 
    opened by Startouf 0
  • Sync metabase exposures as a list of files instead of a single file

    Sync metabase exposures as a list of files instead of a single file

    To make code exploration and review easier, would it be possible to replicate metabase file and folder hierarchy when generating exposures ? Like reusing the slug

    /metabase-exposures/question/42-my-awesome-question.yml /metabase-exposures/dashboard/84-my-super-dashboard

    Or maybe just the ID if to avoid too many file changes when just renaming something ?

    /metabase-exposures/cards/1.yml /metabase-exposures/cards/2.yml /metabase-exposures/dashboards/1.yml

    Currently all exposures are sent to a single file and it's messy

    enhancement help wanted 
    opened by Startouf 0
  • Permission System sync

    Permission System sync

    On Metabase, it's possible to define permissions restrictions to tables for certain groups (Such as in the picture) image

    In the Open Source plan for Metabase, we can define if this group has either "Unrestricted" or "No self-Service" permission to the table, I am not sure if this changes for the other plans.

    Is the automation for the permission system, something planned/wanted for this package?

    One suggestion that could make it possible is to add this in the meta section of the table definition, and define which groups have which level of permission: image

    enhancement help wanted 
    opened by patygold 1
  • Add dbt-metabase explore command

    Add dbt-metabase explore command

    Lets consider our usual flow of

    dbt compile -m my_model navigate to compiled model copy paste sql find somewhere to run it to test it OR dbt run -m my_model check in datagrip, dbeaver, dbvisualizer, vscode sql ext, etc. repeat x?... which the latter implies potentially multiple CTAS, tables being dropped and rewritten, etc. or the inability to test in production vs compile and pasting somewhere you can run it.

    I think we can easily offer an ability to invoke something like:

    dbt-metabase explore my_model

    And on our end conceptually, we use a subprocess to invoke dbt compile -m my_model Read the compiled sql from the manifest.json or file. Construct and pass it into a json object that looks something like {dataset_query: "SELECT * FROM ..."} with your compiled model query inputted Base64 encode the json dumped string And construct a URL like this as many Metabase aficionados may be familiar with (its how metabase caches your in progress queries in the editor).

    https://my.metabase/question#eyJkYXRhc2V0X3F1ZXJ5Ijp7InR5cGUiO...c2V0dGluZ3MiOnt9fQ==

    This will allow, with a simple invocation, using metabase as a front end graphical dbt model explorer without the above workflow being an impedance. I have a similar goal in my project dbt-osmosis which uses a dbt-osmosis workbench invocation with streamlit as a backend for rapidly developing/prototyping and diffing the data output of model changes. I think for Metabase power users, this is a simple and effective alternative too.

    We empower users to explore models, all without actually ever asking them for db creds. Interesting right. Opening this now to gauge feedback prior to implementation.

    enhancement 
    opened by z3z1ma 2
  • KeyError: 'fk_target_table' when when column has semantic type type/FK and has no tests

    KeyError: 'fk_target_table' when when column has semantic type type/FK and has no tests

    Hello,

    I've got a KeyError: 'fk_target_table' when propagating data from dbt table to Metabase. This error occurs in two specific cases:

    1. A column in the the schema.yml has a semantic type equal to type/FK but it has not a tests section within it.
    2. A column in the the schema.yml has a semantic type equal to type/FK but its tests section contains a dbt_utils.relationships_where test within it.
    enhancement help wanted 
    opened by heberaguilar 4
Releases(v0.9.14)
  • v0.9.14(Dec 9, 2022)

  • v0.9.13(Nov 8, 2022)

  • v0.9.12(Nov 7, 2022)

  • v0.9.11(Nov 7, 2022)

  • v0.9.10(Sep 15, 2022)

    Features

    • More Metabase column settings: has_field_values, coercion_strategy, number_style (#145)

    https://pypi.org/project/dbt-metabase/0.9.10/

    Source code(tar.gz)
    Source code(zip)
  • v0.9.9(Sep 1, 2022)

  • v0.9.8(Aug 25, 2022)

  • v0.9.7(Aug 22, 2022)

    Features

    • Support explicit foreign key relationships (#139)

    Fixes

    • Pin requirement versions and fix default sync period (#138)

    https://pypi.org/project/dbt-metabase/0.9.7/

    Source code(tar.gz)
    Source code(zip)
  • v0.9.6(Aug 18, 2022)

    Fixes

    • Consistent MB_PASSWORD environment variable (#134)
    • Update the known table visibility_types in docs (#135)

    https://pypi.org/project/dbt-metabase/0.9.6/

    Source code(tar.gz)
    Source code(zip)
  • v0.9.5(Aug 11, 2022)

    Fixes

    • Fix metabase.display_name population from manifests (#130)
    • Allow explicit null semantic types to override detected ones (#132)

    https://pypi.org/project/dbt-metabase/0.9.5/

    Source code(tar.gz)
    Source code(zip)
  • v0.9.4(Jul 18, 2022)

    Fixes

    • Added metabase_session_id parameter for exposures (#127)
    • Consistency update README.rst (#126)

    https://pypi.org/project/dbt-metabase/0.9.4/

    Source code(tar.gz)
    Source code(zip)
  • v0.9.3(Jul 9, 2022)

    Fixes

    • Allow self-referenced models (#121)
    • Make dbt_includes and dbt_excludes case insensitive (#123)

    https://pypi.org/project/dbt-metabase/0.9.3/

    Source code(tar.gz)
    Source code(zip)
  • v0.9.2(Jul 7, 2022)

  • v0.9.1(Jul 7, 2022)

    Features

    • Add Metabase display name for columns (#111, #112)

    Fixes

    • TypeError: models() got an unexpected keyword argument 'metabase_session_id' (#114)
    • Improve debug message when skipping due to wrong target database (#115)
    • Default visibility_type for models (#117)

    https://pypi.org/project/dbt-metabase/0.9.1/

    Source code(tar.gz)
    Source code(zip)
  • v0.9.0(Jul 6, 2022)

    Features

    • Add Metabase visibility type for models (#68, #107)
    • Add Metabase display name for models (#95, #109)
    • Allow excluding dbt sources when exporting to Metabase (#95, #109)
    • Support Metabase authentication with session ID (#83, #108)

    Fixes

    • Handle 404 error for deactivated users when exporting exposures (#88, #96)
    • Circular depends_on references in manifest reader (#106, #110)

    https://pypi.org/project/dbt-metabase/0.9.0/

    Source code(tar.gz)
    Source code(zip)
  • v0.8.6(Jul 5, 2022)

  • v0.8.5(Mar 5, 2022)

  • v0.8.4(Feb 18, 2022)

  • v0.8.3(Jan 20, 2022)

  • v0.8.2(Nov 28, 2021)

  • v0.8.0(Nov 17, 2021)

    Breaking changes:

    • New dbt manifest parser to eventually replace now deprecated folder parser (#19, #27)
    • Rename CLI and programmatic arguments (#29, #31 , #34, #58)

    Other changes:

    • Support for aliased models and source blocks (#17)
    • Fix for new Metabase API pagination envelope in X.40.0 (#32)
    • Fix Metabase API 400 on empty-string descriptions (#23)
    • Fix for users of BigQuery that has no schemas, defaulting to public (#40)
    • Exposure parsing (#22, #38)
    • Fix error installing pip package directly from git relating to dynamic SCM version (#42)
    • Restore Python 3.6 compatibility (#55)
    • Add explicit UTF-8 encoding for opening files on systems with other defaults (#56)
    • Improvements to config and logging (#65)
    • Resiliency for empty and incorrect inputs (#64)

    https://pypi.org/project/dbt-metabase/0.8.0/

    Source code(tar.gz)
    Source code(zip)
  • v0.8.0-rc5(Oct 5, 2021)

    • Improvements to config and logging (#65)
    • Resiliency for empty and incorrect inputs (#64)
    • Last release with API changes (excluding bug fixes)

    https://pypi.org/project/dbt-metabase/0.8.0rc5/

    Source code(tar.gz)
    Source code(zip)
  • v0.8.0-rc4(Sep 13, 2021)

    • BREAKING CHANGE: Fix rename to the export command introduced in RC3 (#58)
    • Restore Python 3.6 compatibility (#55)
    • Add explicit UTF-8 encoding for opening files on systems with other defaults (#56)

    https://pypi.org/project/dbt-metabase/0.8.0rc4/

    Source code(tar.gz)
    Source code(zip)
  • v0.8.0-rc3(Aug 1, 2021)

    • BREAKING CHANGE: Rename export command to models and more changes to CLI arguments and programmatic configs
    • Exposure parsing (#22, #38)
    • Fix error installing pip package directly from git relating to dynamic SCM version (#42)

    https://pypi.org/project/dbt-metabase/0.8.0rc3/

    Source code(tar.gz)
    Source code(zip)
  • v0.8.0-rc2(Jul 23, 2021)

  • v0.8.0-rc1(Jul 17, 2021)

    • BREAKING CHANGE: New dbt manifest parser to eventually replace now deprecated folder parser (#19, #27)
    • BREAKING CHANGE: Rename CLI and programmatic arguments (#29, #31 , #34)
    • Support for aliased models and source blocks (#17)
    • Fix for new Metabase API pagination envelope in X.40.0 (#32)
    • Fix Metabase API 400 on empty-string descriptions (#23)

    https://pypi.org/project/dbt-metabase/0.8.0rc1/

    Source code(tar.gz)
    Source code(zip)
  • v0.7.1(Jul 17, 2021)

  • v0.7.0(May 31, 2021)

  • v0.6.0(Apr 21, 2021)

    Metabase 0.39.0 renamed special_type to semantic_type (#15):

    • DEPRECATION: dbt meta field metabase.special_type is now deprecated, use metabase.semantic_type instead (will be removed in a future release)
    • Exporter checks API response for special_type/semantic_type and applies whichever one is present for backwards compatibility with earlier Metabase versions

    https://pypi.org/project/dbt-metabase/0.6.0/

    Source code(tar.gz)
    Source code(zip)
  • v0.5.2(Feb 26, 2021)

    • Include hidden tables in Metabase (#10)
    • Display all model compatibility errors at once (#11)

    https://pypi.org/project/dbt-metabase/0.5.2/

    Source code(tar.gz)
    Source code(zip)
dbt adapter for Firebolt

dbt-firebolt dbt adapter for Firebolt dbt-firebolt supports dbt 0.21 and newer Installation First, download the JDBC driver and place it wherever you'

null 23 Dec 14, 2022
An Airflow operator to call the main function from the dbt-core Python package

airflow-dbt-python An Airflow operator to call the main function from the dbt-core Python package Motivation Airflow running in a managed environment

Tomás Farías Santana 93 Jan 8, 2023
Make dbt docs and Apache Superset talk to one another

dbt-superset-lineage Make dbt docs and Apache Superset talk to one another Why do I need something like this? Odds are rather high that you use dbt to

Slido 81 Jan 6, 2023
dbt (data build tool) adapter for Oracle Autonomous Database

dbt-oracle version 1.0.0 dbt (data build tool) adapter for the Oracle database. dbt "adapters" are responsible for adapting dbt's functionality to a g

Oracle 22 Nov 15, 2022
A data engineering project with Kafka, Spark Streaming, dbt, Docker, Airflow, Terraform, GCP and much more!

Streamify A data pipeline with Kafka, Spark Streaming, dbt, Docker, Airflow, Terraform, GCP and much more! Description Objective The project will stre

Ankur Chavda 206 Dec 30, 2022
Model Quantization Benchmark

MQBench Update V0.0.2 Fix academic prepare setting. More deployable prepare process. Fix setup.py. Fix deploy on SNPE. Fix convert_deploy bug. Add Qua

null 500 Jan 6, 2023
A web app for presenting my research in BEM(building energy model) simulation

BEM(building energy model)-SIM-APP The is a web app presenting my research in BEM(building energy model) calibration. You can play around with some pa

null 8 Sep 3, 2021
Generate your personal 8-bit avatars using Cellular Automata, a mathematical model that simulates life, survival, and extinction

Try the interactive demo here ✨ ✨ Sprites-as-a-Service is an open-source web application that allows you to generate custom 8-bit sprites using Cellul

Lj Miranda 265 Dec 26, 2022
Suite of tools for retrieving USGS NWIS observations and evaluating National Water Model (NWM) data.

Documentation OWPHydroTools GitHub pages documentation Motivation We developed OWPHydroTools with data scientists in mind. We attempted to ensure the

null 36 Dec 11, 2022
DOP-Tuning(Domain-Oriented Prefix-tuning model)

DOP-Tuning DOP-Tuning(Domain-Oriented Prefix-tuning model)代码基于Prefix-Tuning改进. Files ├── seq2seq # Code for encoder-decoder arch

Andrew Zeng 5 Nov 2, 2022
Aerospace utilities: flight conditions package, standard atmosphere model, and more.

Aerospace Utilities About Module that contains commonly-used aerospace utilities for problem solving. Flight Condition: input altitude to compute comm

null 1 Jan 3, 2022
Python for downloading model data (HRRR, RAP, GFS, NBM, etc.) from NOMADS, NOAA's Big Data Program partners (Amazon, Google, Microsoft), and the University of Utah Pando Archive System.

Python for downloading model data (HRRR, RAP, GFS, NBM, etc.) from NOMADS, NOAA's Big Data Program partners (Amazon, Google, Microsoft), and the University of Utah Pando Archive System.

Brian Blaylock 194 Jan 2, 2023
A Python script to parse Fortinet products serial numbers, and detect the associated model and version.

ParseFortinetSerialNumber A Python script to parse Fortinet products serial numbers, and detect the associated model and version. Example $ ./ParseFor

Podalirius 10 Oct 28, 2022
Active Transport Analytics Model: A new strategic transport modelling and data visualization framework

{ATAM} Active Transport Analytics Model Active Transport Analytics Model (“ATAM”

ATAM Analytics 2 Dec 21, 2022
This is a working model for which I have used python.

Jarvis_voiceAssistance This is a working model for which I have used python. This model can: 1)Play a video or song on youtube. 2)Tell us time. 3)Tell

Hardik Jain 1 Jan 30, 2022
CBO uses its Capital Tax model (CBO-CapTax) to estimate the effects of federal taxes on capital income from new investment

CBO’s CapTax Model CBO uses its Capital Tax model (CBO-CapTax) to estimate the effects of federal taxes on capital income from new investment. Specifi

Congressional Budget Office 7 Dec 16, 2022
dbt-subdocs is a python CLI you can used to generate a dbt-docs for a subset of your dbt project

dbt-subdocs dbt-subdocs is a python CLI you can used to generate a dbt-docs for a subset of your dbt project ?? Description This project is useful if

Jambe 6 Jan 3, 2023
Cross-platform desktop synchronization client for the Nuxeo platform.

Nuxeo Drive Desktop Synchronization Client for Nuxeo This is an ongoing development project for desktop synchronization of local folders with remote N

Nuxeo 63 Dec 16, 2022
Sync2Gen Code for ICCV 2021 paper: Scene Synthesis via Uncertainty-Driven Attribute Synchronization

Sync2Gen Code for ICCV 2021 paper: Scene Synthesis via Uncertainty-Driven Attribute Synchronization 0. Environment Environment: python 3.6 and cuda 10

Haitao Yang 62 Dec 30, 2022
Sync Toolbox - Python package with reference implementations for efficient, robust, and accurate music synchronization based on dynamic time warping (DTW)

Sync Toolbox - Python package with reference implementations for efficient, robust, and accurate music synchronization based on dynamic time warping (DTW)

Meinard Mueller 66 Jan 2, 2023