dbt adapter for Firebolt

Overview

dbt-firebolt

dbt adapter for Firebolt

dbt-firebolt supports dbt 0.21 and newer

Installation

First, download the JDBC driver and place it wherever you'd prefer. If you've never installed a Java Runtime Environment you will need to download and install one from either OpenJDK or Oracle.

Now install dbt-firebolt. For the current version:

pip install dbt-firebolt

Setup

Engines

For dbt to function with Firebolt you must have an engine connected to your database and available. In addition, these needs must be met:

  1. The engine must be a general-purpose read-write engine.
  2. You must have permissions to access the engine.
  3. The engine must be running.
  4. If you're not using the default engine for the database, the name of the engine must be specified.

YAML configuration file

You'll need to add your project to the profiles.yml file. These fields are necessary:

  • type
  • user
  • password
  • database
  • schema
  • jar_path

These fields are optional:

  • engine_name (See note above.)
  • host (The host defaults to api.app.firebolt.io. If you want to use a dev account you must include the host field and set it to api.dev.firebolt.io.)

Note that, although the value of type is always firebolt, it must be included either in profiles.yml or in the dbt_project.yml file for your application.

Example file:

my_project:
  target: fb_app
  fb_app:
      type: firebolt
      user: 
   
    
      password: 
    
     
      database: 
     
      
      schema: 
      
       
      jar_path: 
       
         threads: 1 # The following two fields are optional. Please see the notes above. engine_name: 
        
          host: api.app.firebolt.io 
        
       
      
     
    
   

dbt feature support

feature supported
tables/views
ephemeral
tests
docs
incremental
snapshot
source_freshness

Model Configuration in Firebolt

Dimension and Fact Tables

Both fact and dimension tables are supported. When materialized='table', table type and primary index configurations can be set. table type can be either fact or dimension. primary_index is required for fact tables, and can be a string or a list of strings.

These configs can be set by either:

  1. a config block at the top of that model's SQL file (like below), or
{{
  config(
    materialized = 'table',
    table_type = 'dimension',
    primary_index = ['customer_id', 'first_name']
    )
}}
  1. in the dbt_project.yml or model schema YAML file.

Read more in dbt docs on configuring models

Join and Aggregating Indexes

In addition to primary indexes, Firebolt also supports:

dbt-firebolt follows the same convention for indexes as was introduced to dbt-postgres (more info on indexes usage in dbt-postgres).

Naming

In dbt-firebolt, indexes are named as follows, with the number being a unix timestamp at the time of execution

  • template: table-name__key-column__index-type_unix-timestamp
  • join index: my_orders__order_id__join_1633504263
  • aggregating index: my_orders__order_id__aggregating_1633504263

Usage

The index argument takes a list of dictionaries, where each dictionary is an index you'd like to define. there are two types of indexes that can be defined here: aggregating and join. The required fields for each index are as follows:

  • aggregating: key_column (string) and aggregation (string of list of strings)
  • join: join_column (string) and dimension_column (string of list of strings)

Fact table with aggregating index

-- orders.sql
{{
  config(
    materialized = 'table',
    table_type = 'fact',
    primary_index = 'id',
    indexes = [
      {
        'type': 'aggregating',
        'key_column': 'order_id',
        'aggregation': ['COUNT(DISTINCT status)', 'AVG(customer_id)']
      }
    ]
    )
}}

Dimension table with join index

-- orders.sql
{{
  config(
    materialized = 'table',
    table_type = 'dimension',
    indexes = [
      {
        'type': 'join',
        'join_column': 'order_id',
        'dimension_column': ['customer_id', 'status']
      }
    ]
    )
}}

Fact table with two aggregating indexes and one join index

-- orders.sql
{{
  config(
    materialized = 'table',
    table_type = 'dimension',
    indexes = [
      {
        'type': 'aggregating',
        'key_column': 'order_id',
        'aggregation': ['COUNT(DISTINCT status)', 'AVG(customer_id)']
      },
      {
        'type': 'aggregating',
        'key_column': 'customer_id',
        'aggregation': 'COUNT(DISTINCT status)'
      },
      {
        'type': 'join',
        'join_column': 'order_id',
        'dimension_column': ['customer_id', 'status']
      }
    ]
    )
}}

Recommended dbt project configurations

quote_columns

To prevent a warning, you should add a configuration as below to your dbt_project.yml. For more info, see the relevant dbt docs page

seeds:
  +quote_columns: false # or `true` if you have csv column headers with spaces

dbt projects with concurrent users

Currently, with dbt-firebolt, all models will be run in the same schema, so the schema provided isn't used, but is still required. If you are a team of analytics engineers using a the same database and engine, a recommended practice is to add the following macro to your project. It will prefix the model name/alias with the schema value to provide namespacing so that multiple developers are not interacting with the same set of models.

For example, consider two analytics engineers on the same project: Shahar and Eric.

If in their .dbt/profiles.yml, Sharar provides schema=sh, and Eric, schema=er, when they each run the customers model, the models will land in the database as sh_customers and er_customers, respectively.

-- macros/generate_alias_name.sql
{% macro generate_alias_name(custom_alias_name=none, node=none) -%}

    {%- if custom_alias_name is none -%}

        {{ node.schema }}_{{ node.name }}

    {%- else -%}

        {{ node.schema }}_{{ custom_alias_name | trim }}

    {%- endif -%}

{%- endmacro %}

External Tables

Documentation on dbt's use of external tables can be found in the dbt documentation.

Documentation on using external tables including properly configuring IAM can be found in the Firebolt documentation.

Installation

To install and use dbt-external-tables with firebolt, you must:

  1. Add the package to your packages.yml,
    packages:
    - package: dbt-labs/dbt_external_tables
        version: 
         
  2. add this to your dbt_project.yml, and
    dispatch:
      - macro_namespace: dbt_external_tables
        search_order: ['dbt', 'dbt_external_tables']
  3. call dbt deps

Usage

To use external tables, you must define a table as EXTERNAL in your project.yml file. Every external table must contain fields for url, type, and object pattern. Note that the Firebolt external table specification differs slightly from the dbt specification in the dbt documentation in that it does not require all the fields shown in dbt's documentation.

In addition to specifying the columns, an external table may specify partitions. Partitions are not columns and a partition name cannot have the same name as a column. An example yaml file follows. In order to avoid yml parsing errors it is likely necessary to quote at least the url, object_pattern, and regex values.

external: url: 's3:// /' object_pattern: ' ' type: ' ' credentials: internal_role_arn: arn:aws:iam::id: / external_role_id: object_pattern: ' ' compression: ' ' partitions: - name: data_type: regex: ' ' columns: - name: data_type: ">
sources:
  - name: firebolt_external
    schema: "{{ target.schema }}"
    loader: S3

    tables:
      - name: 
                
        external:
          url: 's3://
                
                 /
                 '
                
          object_pattern: '
                
                 '
                
          type: '
                
                 '
                
          credentials:
            internal_role_arn: arn:aws:iam::id:
                
                 /
                 
                
            external_role_id: 
                
          object_pattern: '
                
                 '
                
          compression: '
                
                 '
                
          partitions:
            - name: 
                
              data_type: 
                
              regex: '
                
                 '
                
          columns:
            - name: 
                
              data_type: 
                

Changelog

See our changelog or our release history for more info

Comments
  • Add pre-commit hooks for linting; linted files

    Add pre-commit hooks for linting; linted files

    @swanderz's understanding: This PR adds pre-commit hooks, which will prevent developers from making local commits without either passing all of these linting checks or commenting them out. This forces developers to adopt the flake8, isosort, and black style.

    Still to do in a future PR:

    • Add a GitHub action for linting
    • #24
    opened by ima-hima 6
  • fix: External table models with missing fields now produce an error

    fix: External table models with missing fields now produce an error

    Description

    Previously, external table models with missing or misspelled regex or data_type fields caused indecipherable errors. Those errors are now clear. In addition, drop/create on external tables only occurs if variable is set: dbt run-operation stage_external_sources --vars "ext_full_refresh: true". Otherwise, table is skipped.

    Fixes

    dbt seeds doesn't truncate tables before running

    Checklist

    • [X] I have run this code in development and it appears to resolve the stated issue.
    • [X] This PR includes tests, or tests are not required/relevant for this PR.
    • [X] I have updated CHANGELOG.md and added information about my change.
    • [X] If this PR requires a new PyPI release I have bumped the version number.
    • [X] I have pulled/merged from the main branch if there are merge conflicts.
    • [X] I have verified that this PR contains only code changes relevant to this PR.
    opened by ericf-firebolt 4
  • ci: Add mypy

    ci: Add mypy

    Resolves #

    Description

    Checklist

    • [x] I have run this code in development and it appears to resolve the stated issue.
    • [x] This PR includes tests, or tests are not required/relevant for this PR.
    • [x] I have updated CHANGELOG.md and added information about my change.
    • [x] I have removed any print or log calls that were added for debugging.
    • [x] If this PR requires a new PyPI release I have bumped the version number.
    • [x] I have verified that this PR contains only code changes relevant to this PR.
    • [x] If further integration tests are now passing I've edited firebolt.dbtspec to account for this.
    • [x] I have pulled/merged from the main branch if there are merge conflicts.
    • [x] After pulling/merging main I have run pytest on the included or updated firebolt.dbtspec.
    opened by ericf-firebolt 3
  • faster integration tests via pre-hook

    faster integration tests via pre-hook

    this change forces each test to use the set firebolt_dont_wait_for_upload_to_s3=1 pre-hook. this will make tests run much faster (provided that the driver/SDK allows pre-hook statements)

    opened by dataders 3
  • temp workaround for false approximate match

    temp workaround for false approximate match

    fixes: #11

    #8 might be closer to the permanent solution, but I'm still bottoming out on root cause and will likely need to meet w/ @jtcohen6 sometime next week.

    this isn't a permanent fix, but it will work in the short-term, while we dive deep on the source of the adapter.get_relation issue

    an overly-simplified normal operation for the table materialization is:

    1. get the database, schema and identifier of the relation we are trying to make
    2. go look and see if a relation with the same three-part name already exists and assign result to old_relation
      1. if yes, return an Relation object with the three-part name, and a type property (so we know if it exists as a view or as a table)
      2. if no, return None
    3. if old_relation is not None, drop the pre-existing relation.

    however, because we currently do not support views, if we assume that there are no views currently in the database, then we don't need to step 2 above. instead we can always just call DROP TABLE <TABLE_NAME> IF EXISTS, because:

    1. the relation should always exist, and
    2. if the relation doesn't already exist, the statement won't fail because of the ... IF EXISTS clause
    opened by dataders 3
  • refactor: Remove redundant mat test

    refactor: Remove redundant mat test

    Ref #71

    Description

    Removing redundant materialisation which is already implemented on the dbt-core side.

    Checklist

    • [x] I have run this code in development and it appears to resolve the stated issue.
    • [x] This PR includes tests, or tests are not required/relevant for this PR.
    • [ ] I have updated CHANGELOG.md and added information about my change.
    • [x] I have removed any print or log calls that were added for debugging.
    • [ ] If this PR requires a new PyPI release I have bumped the version number.
    • [x] I have verified that this PR contains only code changes relevant to this PR.
    • [x] If further integration tests are now passing I've edited tests/functional/adapter/test_basic.py to account for this.
    • [x] I have pulled/merged from the main branch if there are merge conflicts.
    • [x] After pulling/merging main I have run pytest on the included or updated tests/functional/adapter/test_basic.py.
    opened by ptiurin 2
  • fix: quoting database name

    fix: quoting database name

    Description

    This fixes the issue we were having when database name had mixed-case letters. Quoting ensures the casing is preserved.

    TESTs

    Run both jaffle shop and pytest tests. Confirmed users can override the value in dbt-project.yml

    Checklist

    • [x] I have run this code in development and it appears to resolve the stated issue.
    • [x] This PR includes tests, or tests are not required/relevant for this PR.
    • [x] I have updated CHANGELOG.md and added information about my change.
    • [x] I have removed any print or log calls that were added for debugging.
    • [x] If this PR requires a new PyPI release I have bumped the version number.
    • [x] I have verified that this PR contains only code changes relevant to this PR.
    • [x] If further integration tests are now passing I've edited firebolt.dbtspec to account for this.
    • [x] I have pulled/merged from the main branch if there are merge conflicts.
    • [x] After pulling/merging main I have run pytest on the included or updated firebolt.dbtspec.
    opened by ptiurin 2
  • fix: Aggregating indices now allow multiple field names in the key column

    fix: Aggregating indices now allow multiple field names in the key column

    Resolves https://packboard.atlassian.net/browse/FIR-8715

    Checklist

    • [x] I have run this code in development and it appears to resolve the stated issue.
    • [x] This PR includes tests, or tests are not required/relevant for this PR.
    • [x] I have updated CHANGELOG.md and added information about my change.
    • [x] If this PR requires a new PyPI release I have bumped the version number.
    • [x] I have pulled/merged from the main branch if there are merge conflicts.
    • [x] I have verified that this PR contains only code changes relevant to this PR.
    opened by ericf-firebolt 2
  • ci: Add pytests to pr actions

    ci: Add pytests to pr actions

    Resolves #

    Description

    Checklist

    • [x] I have run this code in development and it appears to resolve the stated issue.
    • [x] This PR includes tests, or tests are not required/relevant for this PR.
    • [x] I have updated CHANGELOG.md and added information about my change.
    • [x] If this PR requires a new PyPI release I have bumped the version number.
    • [x] I have pulled/merged from the main branch if there are merge conflicts.
    • [x] I have verified that this PR contains only code changes relevant to this PR.
    opened by ericf-firebolt 2
  • ci: Jaffle shop test action

    ci: Jaffle shop test action

    Further fixes for jaffle shop test action

    Checklist

    • [x] I have run this code in development and it appears to resolve the stated issue.
    • [x] This PR includes tests, or tests are not required/relevant for this PR.
    • [ ] I have updated CHANGELOG.md and added information about my change.
    • [ ] If this PR requires a new PyPI release I have bumped the version number.
    • [x] I have pulled/merged from the main branch if there are merge conflicts.
    • [x] I have verified that this PR contains only code changes relevant to this PR.
    opened by stepansergeevitch 2
  • Use only setup.cfg in lieu of setup.py

    Use only setup.cfg in lieu of setup.py

    #20 adds a setup.cfg to our project even though we already have a setup.py. This will only cause problems for us by not having a single source of truth for package configuration. This should be addressed at somepoint

    Might be wrong here, but I'm fairly certain we don't need this file. We're currently using setup.py instead of a .cfg. relevant SO question?

    Originally posted by @swanderz in https://github.com/firebolt-db/dbt-firebolt/pull/20#r754762061

    opened by dataders 2
  • upgrade to support dbt-core v1.3.0

    upgrade to support dbt-core v1.3.0

    Background

    The latest release cut for 1.3.0, dbt-core==1.3.0rc2 was published on October 3, 2022 (PyPI | Github). We are targeting releasing the official cut of 1.3.0 in time for the week of October 16 (in time for Coalesce conference).

    We're trying to establish a following precedent w.r.t. minor versions: Partner adapter maintainers release their adapter's minor version within four weeks of the initial RC being released. Given the delay on our side in notifying you, we'd like to set a target date of November 7 (four weeks from today) for maintainers to release their minor version

    | Timeframe | Date (intended) | Date (Actual) | Event | | ----------- | --------------- | ------------- | ------------------------------------------------------ | | D - 3 weeks | Sep 21 | Oct 10 | dbt Labs informs maintainers of upcoming minor release | | D - 2 weeks | Sep 28 | Sep 28 | core 1.3 RC is released | | Day D | October 12 | Oct 12 | core 1.3 official is published | | D + 2 weeks | October 26 | Nov 7 | dbt-adapter 1.3 is published |

    How to upgrade

    https://github.com/dbt-labs/dbt-core/discussions/6011 is an open discussion with more detailed information, and https://github.com/dbt-labs/dbt-core/issues/6040 is for keeping track of the community's progress on releasing 1.2.0

    Below is a checklist of work that would enable a successful 1.2.0 release of your adapter.

    • [ ] Python Models (if applicable)
    • [ ] Incremental Materialization: cleanup and standardization
    • [ ] More functional adapter tests to inherit
    opened by dataders 0
  • upgrade to support dbt-core v1.3.0

    upgrade to support dbt-core v1.3.0

    Background

    The latest release cut for 1.3.0, dbt-core==1.3.0rc2 was published on October 3, 2022 (PyPI | Github). We are targeting releasing the official cut of 1.3.0 in time for the week of October 16 (in time for Coalesce conference).

    We're trying to establish a following precedent w.r.t. minor versions: Partner adapter maintainers release their adapter's minor version within four weeks of the initial RC being released. Given the delay on our side in notifying you, we'd like to set a target date of November 7 (four weeks from today) for maintainers to release their minor version

    | Timeframe | Date (intended) | Date (Actual) | Event | | ----------- | --------------- | ------------- | ------------------------------------------------------ | | D - 3 weeks | Sep 21 | Oct 10 | dbt Labs informs maintainers of upcoming minor release | | D - 2 weeks | Sep 28 | Sep 28 | core 1.3 RC is released | | Day D | October 12 | Oct 12 | core 1.3 official is published | | D + 2 weeks | October 26 | Nov 7 | dbt-adapter 1.3 is published |

    How to upgrade

    https://github.com/dbt-labs/dbt-core/discussions/6011 is an open discussion with more detailed information, and https://github.com/dbt-labs/dbt-core/issues/6040 is for keeping track of the community's progress on releasing 1.2.0

    Below is a checklist of work that would enable a successful 1.2.0 release of your adapter.

    • [ ] Python Models (if applicable)
    • [ ] Incremental Materialization: cleanup and standardization
    • [ ] More functional adapter tests to inherit
    opened by dataders 0
Releases(1.2.0)
  • 1.2.0(Dec 20, 2022)

    What's Changed

    • test: Migrate to new test framework by @ptiurin in https://github.com/firebolt-db/dbt-firebolt/pull/79
    • test: Add test_utils for 1.2 compatibility by @ptiurin in https://github.com/firebolt-db/dbt-firebolt/pull/81
    • test: Fixing catalog error by @ptiurin in https://github.com/firebolt-db/dbt-firebolt/pull/80
    • refactor: Remove redundant mat test by @ptiurin in https://github.com/firebolt-db/dbt-firebolt/pull/83
    • test: Basic doc tests by @ptiurin in https://github.com/firebolt-db/dbt-firebolt/pull/82
    • feat: Retry connection on error by @ptiurin in https://github.com/firebolt-db/dbt-firebolt/pull/84
    • docs: Clarify why test is skipped by @ptiurin in https://github.com/firebolt-db/dbt-firebolt/pull/86
    • test: Adding grant tests by @ptiurin in https://github.com/firebolt-db/dbt-firebolt/pull/85
    • build: Changelog by @ptiurin in https://github.com/firebolt-db/dbt-firebolt/pull/87
    • build: dbt-core 1.2 by @ptiurin in https://github.com/firebolt-db/dbt-firebolt/pull/88
    • test: Fix grant test env var by @ptiurin in https://github.com/firebolt-db/dbt-firebolt/pull/89

    Full Changelog: https://github.com/firebolt-db/dbt-firebolt/compare/1.1.3...1.2.0

    Source code(tar.gz)
    Source code(zip)
  • 1.1.3(Oct 20, 2022)

    What's Changed

    • fix: quoting database name by @ptiurin in https://github.com/firebolt-db/dbt-firebolt/pull/78

    Full Changelog: https://github.com/firebolt-db/dbt-firebolt/compare/1.1.2...1.1.3

    Source code(tar.gz)
    Source code(zip)
  • 1.1.2(Oct 20, 2022)

    What's Changed

    • feat: pass better error messages when API times out by @ericf-firebolt in https://github.com/firebolt-db/dbt-firebolt/pull/75

    Full Changelog: https://github.com/firebolt-db/dbt-firebolt/compare/v1.1.1...v1.1.2

    Source code(tar.gz)
    Source code(zip)
  • 1.1.1(Oct 20, 2022)

    What's Changed

    • chore: Upgrade sdk to 0.9.2 by @ptiurin in https://github.com/firebolt-db/dbt-firebolt/pull/74

    Full Changelog: https://github.com/firebolt-db/dbt-firebolt/compare/1.1.0...v1.1.1

    Source code(tar.gz)
    Source code(zip)
  • 1.1.0(Jun 10, 2022)

    What's Changed

    • ci: Add mypy by @ericf-firebolt in https://github.com/firebolt-db/dbt-firebolt/pull/66
    • feat: Insert overwrite incremental model by @ericf-firebolt in https://github.com/firebolt-db/dbt-firebolt/pull/67

    Full Changelog: https://github.com/firebolt-db/dbt-firebolt/compare/1.0.6...1.1.0

    Source code(tar.gz)
    Source code(zip)
  • 1.0.6(May 19, 2022)

    What's Changed

    • build: require dbt1.1 by @ericf-firebolt in https://github.com/firebolt-db/dbt-firebolt/pull/69

    Full Changelog: https://github.com/firebolt-db/dbt-firebolt/compare/1.0.5...1.0.6

    Source code(tar.gz)
    Source code(zip)
  • 1.0.5(May 12, 2022)

    What's Changed

    • ci: Moved dev reqs for pre-commit from setup.cfg into pre-commit config file. by @ericf-firebolt in https://github.com/firebolt-db/dbt-firebolt/pull/63
    • fix: return 'text' for firebolt string columns by @miguel-firebolt in https://github.com/firebolt-db/dbt-firebolt/pull/65

    New Contributors

    • @miguel-firebolt made their first contribution in https://github.com/firebolt-db/dbt-firebolt/pull/65

    Full Changelog: https://github.com/firebolt-db/dbt-firebolt/compare/1.0.4...1.0.5

    Source code(tar.gz)
    Source code(zip)
  • 1.0.4(May 5, 2022)

    What's Changed

    • style: Better log and debug output by @ericf-firebolt in https://github.com/firebolt-db/dbt-firebolt/pull/55
    • feat: Added append-only incremental strategy by @ericf-firebolt in https://github.com/firebolt-db/dbt-firebolt/pull/54
    • fix: Removed SHOW VIEWS and SHOW TABLES from all code. by @ericf-firebolt in https://github.com/firebolt-db/dbt-firebolt/pull/56
    • docs(FIR-12448): Create CONTRIBUTING.MD by @ptiurin in https://github.com/firebolt-db/dbt-firebolt/pull/57
    • ci: Enable Fossa PR decoration by @ptiurin in https://github.com/firebolt-db/dbt-firebolt/pull/59
    • docs: Added docstrings to adapters.sql and cleaned up a variable name or two. by @ericf-firebolt in https://github.com/firebolt-db/dbt-firebolt/pull/58
    • feat: Better responses from cursor by @ericf-firebolt in https://github.com/firebolt-db/dbt-firebolt/pull/60
    • ci: Upgrade black version by @ptiurin in https://github.com/firebolt-db/dbt-firebolt/pull/61
    • fix: Correctly format index names by @ericf-firebolt in https://github.com/firebolt-db/dbt-firebolt/pull/62
    • feat: Incremental append-only by @ericf-firebolt in https://github.com/firebolt-db/dbt-firebolt/pull/64

    Full Changelog: https://github.com/firebolt-db/dbt-firebolt/compare/1.0.3...1.0.4

    Source code(tar.gz)
    Source code(zip)
  • 1.0.3(Mar 28, 2022)

    What's Changed

    • fix: Moved all dev-requirements into setup.cfg. by @ima-hima in https://github.com/firebolt-db/dbt-firebolt/pull/52
    • fix: Updated version number and changelog. by @ericf-firebolt in https://github.com/firebolt-db/dbt-firebolt/pull/53

    Full Changelog: https://github.com/firebolt-db/dbt-firebolt/compare/1.0.2...1.0.3

    Source code(tar.gz)
    Source code(zip)
  • 1.0.2(Mar 11, 2022)

    What's Changed

    • fix: External table models with missing fields now produce an error by @ericf-firebolt in https://github.com/firebolt-db/dbt-firebolt/pull/47
    • fix: Aggregating indices now allow multiple field names in the key column by @ericf-firebolt in https://github.com/firebolt-db/dbt-firebolt/pull/51

    Full Changelog: https://github.com/firebolt-db/dbt-firebolt/compare/1.0.1...1.0.2

    Source code(tar.gz)
    Source code(zip)
  • 1.0.1(Mar 1, 2022)

    What's Changed

    • ci: Jaffle shop test action by @stepansergeevitch in https://github.com/firebolt-db/dbt-firebolt/pull/45
    • fix: Seeds now drop and create instead of truncating by @ericf-firebolt in https://github.com/firebolt-db/dbt-firebolt/pull/46

    Full Changelog: https://github.com/firebolt-db/dbt-firebolt/compare/1.0.0...1.0.1

    Source code(tar.gz)
    Source code(zip)
  • 1.0.0(Feb 10, 2022)

    What's Changed

    • fix: 0.21.10 had incorrect version number. by @ericf-firebolt in https://github.com/firebolt-db/dbt-firebolt/pull/40
    • ci: Create release action by @stepansergeevitch in https://github.com/firebolt-db/dbt-firebolt/pull/43
    • ci: Setup testing by @stepansergeevitch in https://github.com/firebolt-db/dbt-firebolt/pull/42
    • docs: Update README.md by @octavianzarzu30 in https://github.com/firebolt-db/dbt-firebolt/pull/41
    • ci: Adding security scan by @ptiurin in https://github.com/firebolt-db/dbt-firebolt/pull/36
    • test: create jaffle_shop test action by @stepansergeevitch in https://github.com/firebolt-db/dbt-firebolt/pull/44
    • feat: Bump version number and update adapter to work with dbt v1.0.0 by @ericf-firebolt in https://github.com/firebolt-db/dbt-firebolt/pull/38

    New Contributors

    • @ptiurin made their first contribution in https://github.com/firebolt-db/dbt-firebolt/pull/36

    Full Changelog: https://github.com/firebolt-db/dbt-firebolt/compare/0.21.10...1.0.0

    Source code(tar.gz)
    Source code(zip)
  • 0.21.10(Feb 2, 2022)

    What's Changed

    • feat: Move to our db api by @ima-hima in https://github.com/firebolt-db/dbt-firebolt/pull/10

    Full Changelog: https://github.com/firebolt-db/dbt-firebolt/compare/0.21.9...0.21.10

    Source code(tar.gz)
    Source code(zip)
  • 0.21.9(Feb 1, 2022)

    What's Changed

    • fix: We now use firebolt-sdk
    • fix: Setuptools config by @stepansergeevitch in https://github.com/firebolt-db/dbt-firebolt/pull/37
    • fix: Fixed missing adapter firebolt error by @ericf-firebolt in https://github.com/firebolt-db/dbt-firebolt/pull/39

    New Contributors

    • @stepansergeevitch made their first contribution in https://github.com/firebolt-db/dbt-firebolt/pull/37

    Full Changelog: https://github.com/firebolt-db/dbt-firebolt/compare/0.21.7...0.21.9

    Source code(tar.gz)
    Source code(zip)
  • 0.21.7(Jan 21, 2022)

  • 0.21.6(Dec 31, 2021)

  • 0.21.4(Dec 7, 2021)

    What's Changed

    • add link to repo from PyPI by @swanderz in https://github.com/firebolt-db/dbt-firebolt/pull/19
    • Polishing README; change names of engine and account profile params by @kevinmarr in https://github.com/firebolt-db/dbt-firebolt/pull/18
    • Fix broken link in README by @kevinmarr in https://github.com/firebolt-db/dbt-firebolt/pull/23
    • drop the prefixed, dbt-generated metadata comment from SQL statements submitted to firebolt by @swanderz in https://github.com/firebolt-db/dbt-firebolt/pull/17
    • enable views for non-leaf models by using default drop_relation by @swanderz in https://github.com/firebolt-db/dbt-firebolt/pull/22
    • Add pre-commit hooks for linting; linted files by @ima-hima in https://github.com/firebolt-db/dbt-firebolt/pull/20
    • add a PR template by @swanderz in https://github.com/firebolt-db/dbt-firebolt/pull/27
    • view workaround for get_relation bug by @swanderz in https://github.com/firebolt-db/dbt-firebolt/pull/25

    Full Changelog: https://github.com/firebolt-db/dbt-firebolt/compare/0.21.3...0.21.4

    Source code(tar.gz)
    Source code(zip)
  • 0.21.3(Nov 18, 2021)

    What's Changed

    • Update README.md by @marknoack in https://github.com/firebolt-db/dbt-firebolt/pull/6
    • Update README.md by @octavianzarzu30 in https://github.com/firebolt-db/dbt-firebolt/pull/9
    • temp workaround for false approximate match by @swanderz in https://github.com/firebolt-db/dbt-firebolt/pull/12

    New Contributors

    • @marknoack made their first contribution in https://github.com/firebolt-db/dbt-firebolt/pull/6
    • @octavianzarzu30 made their first contribution in https://github.com/firebolt-db/dbt-firebolt/pull/9

    Full Changelog: https://github.com/firebolt-db/dbt-firebolt/compare/0.21.2...0.21.3

    Source code(tar.gz)
    Source code(zip)
    dbt-firebolt-0.21.3.tar.gz(20.12 KB)
    dbt_firebolt-0.21.3-py3-none-any.whl(20.32 KB)
  • 0.21.2(Nov 1, 2021)

    Breaking Changes

    • engine_name has been renamed to engine please update your profiles.yml accordingly #4

    Pull Requests

    • HOTFIX: allow patch differences w/ dbt-core by @swanderz in https://github.com/firebolt-db/dbt-firebolt/pull/3
    • Added ability to include account value in profiles.yml to specify whi… by @ima-hima in https://github.com/firebolt-db/dbt-firebolt/pull/4
    • Update url path generation on Windows by @swanderz in https://github.com/firebolt-db/dbt-firebolt/pull/5

    For more info see CHANGELOG.md

    New Contributors

    • @swanderz made their first contribution in https://github.com/firebolt-db/dbt-firebolt/pull/3

    Full Changelog: https://github.com/firebolt-db/dbt-firebolt/compare/0.21.1...0.21.2

    Source code(tar.gz)
    Source code(zip)
    dbt-firebolt-0.21.2.tar.gz(19.51 KB)
    dbt_firebolt-0.21.2-py3-none-any.whl(20.17 KB)
  • 0.21.1(Oct 28, 2021)

    What's Changed

    • Update README with pip install dbt-firebolt by @kevinmarr in https://github.com/firebolt-db/dbt-firebolt/pull/1
    • Update readme cleanup dropif by @ima-hima in https://github.com/firebolt-db/dbt-firebolt/pull/2

    New Contributors

    • @kevinmarr made their first contribution in https://github.com/firebolt-db/dbt-firebolt/pull/1
    • @ima-hima made their first contribution in https://github.com/firebolt-db/dbt-firebolt/pull/2

    Full Changelog: https://github.com/firebolt-db/dbt-firebolt/compare/0.21.0...0.21.1

    Source code(tar.gz)
    Source code(zip)
  • 0.21.0(Oct 25, 2021)

Owner
null
:fishing_pole_and_fish: List of `pre-commit` hooks to ensure the quality of your `dbt` projects.

pre-commit-dbt List of pre-commit hooks to ensure the quality of your dbt projects. BETA NOTICE: This tool is still BETA and may have some bugs, so pl

Offbi 262 Nov 25, 2022
An Airflow operator to call the main function from the dbt-core Python package

airflow-dbt-python An Airflow operator to call the main function from the dbt-core Python package Motivation Airflow running in a managed environment

Tomás Farías Santana 93 Jan 8, 2023
Make dbt docs and Apache Superset talk to one another

dbt-superset-lineage Make dbt docs and Apache Superset talk to one another Why do I need something like this? Odds are rather high that you use dbt to

Slido 81 Jan 6, 2023
Model synchronization from dbt to Metabase.

dbt-metabase Model synchronization from dbt to Metabase. If dbt is your source of truth for database schemas and you use Metabase as your analytics to

Mike Gouline 270 Jan 8, 2023
A data engineering project with Kafka, Spark Streaming, dbt, Docker, Airflow, Terraform, GCP and much more!

Streamify A data pipeline with Kafka, Spark Streaming, dbt, Docker, Airflow, Terraform, GCP and much more! Description Objective The project will stre

Ankur Chavda 206 Dec 30, 2022
dbt-subdocs is a python CLI you can used to generate a dbt-docs for a subset of your dbt project

dbt-subdocs dbt-subdocs is a python CLI you can used to generate a dbt-docs for a subset of your dbt project ?? Description This project is useful if

Jambe 6 Jan 3, 2023
dbt (data build tool) adapter for Oracle Autonomous Database

dbt-oracle version 1.0.0 dbt (data build tool) adapter for the Oracle database. dbt "adapters" are responsible for adapting dbt's functionality to a g

Oracle 22 Nov 15, 2022
:fishing_pole_and_fish: List of `pre-commit` hooks to ensure the quality of your `dbt` projects.

pre-commit-dbt List of pre-commit hooks to ensure the quality of your dbt projects. BETA NOTICE: This tool is still BETA and may have some bugs, so pl

Offbi 262 Nov 25, 2022
Generate lookml for views from dbt models

dbt2looker Use dbt2looker to generate Looker view files automatically from dbt models. Features Column descriptions synced to looker Dimension for eac

lightdash 126 Dec 28, 2022
Singer Tap for dbt Artifacts built with the Meltano SDK

tap-dbt-artifacts tap-dbt-artifacts is a Singer tap for dbtArtifacts. Built with the Meltano SDK for Singer Taps.

Prratek Ramchandani 9 Nov 25, 2022
First and foremost, we want dbt documentation to retain a DRY principle. Every time we repeat ourselves, we waste our time. Second, we want to understand column level lineage and automate impact analysis.

dbt-osmosis First and foremost, we want dbt documentation to retain a DRY principle. Every time we repeat ourselves, we waste our time. Second, we wan

Alexander Butler 150 Jan 6, 2023
An Airflow operator to call the main function from the dbt-core Python package

airflow-dbt-python An Airflow operator to call the main function from the dbt-core Python package Motivation Airflow running in a managed environment

Tomás Farías Santana 93 Jan 8, 2023
Make dbt docs and Apache Superset talk to one another

dbt-superset-lineage Make dbt docs and Apache Superset talk to one another Why do I need something like this? Odds are rather high that you use dbt to

Slido 81 Jan 6, 2023
Model synchronization from dbt to Metabase.

dbt-metabase Model synchronization from dbt to Metabase. If dbt is your source of truth for database schemas and you use Metabase as your analytics to

Mike Gouline 270 Jan 8, 2023
A data engineering project with Kafka, Spark Streaming, dbt, Docker, Airflow, Terraform, GCP and much more!

Streamify A data pipeline with Kafka, Spark Streaming, dbt, Docker, Airflow, Terraform, GCP and much more! Description Objective The project will stre

Ankur Chavda 206 Dec 30, 2022
PostgreSQL database adapter for the Python programming language

psycopg2 - Python-PostgreSQL Database Adapter Psycopg is the most popular PostgreSQL database adapter for the Python programming language. Its main fe

The Psycopg Team 2.8k Jan 5, 2023
《K-Adapter: Infusing Knowledge into Pre-Trained Models with Adapters》(2020)

K-Adapter: Infusing Knowledge into Pre-Trained Models with Adapters This repository is the implementation of the paper "K-Adapter: Infusing Knowledge

Microsoft 118 Dec 13, 2022
Code for the ACL2021 paper "Lexicon Enhanced Chinese Sequence Labelling Using BERT Adapter"

Lexicon Enhanced Chinese Sequence Labeling Using BERT Adapter Code and checkpoints for the ACL2021 paper "Lexicon Enhanced Chinese Sequence Labelling

null 274 Dec 6, 2022
The Adapter-Bot: All-In-One Controllable Conversational Model

The Adapter-Bot: All-In-One Controllable Conversational Model This is the implementation of the paper: The Adapter-Bot: All-In-One Controllable Conver

CAiRE 37 Nov 4, 2022
Chainlink Python Serverless External Adapter Template

This template shows a basic usecase of an external adapter written in Python for the CryptoCompare API. It can be ran locally, in Docker, AWS Lambda, or GCP Functions.

Chainlink Hackathon - Insureblox 1 Nov 17, 2021