Directory of SQL tools and Utils

SQL toolkit and utils

This is for data engineers / analytics engineers who want to level-up their data management by

• ensuring team consistency in SQL queries -> linting problem.

• extracting metadata from your SQL queries -> parsing problem.

• mapping views/tables from source to its destination -> lineage problem.

• modeling joint relationships from your tables -> data modeling problem.

Free Utils / Open source

1. SQL Lint, formatter, fix anti-patterns:

   • https://github.com/sqlfluff/sqlfluff

2. SQL parser, transpiler, optimizer

   • https://github.com/tobymao/sqlglot

     • convert dialects

3. Parsing Metadata

   • https://github.com/macbre/sql-metadata

   • https://pypi.org/project/simple-ddl-parser/

4. Data lineage

   • https://github.com/reata/sqllineage

   • https://datascale.cloud/demo/lineage

Tools

1. Database Diagrams:

   • SQL to DAG or ERD: https://getdatascale.com/sqldiagram

   • ERD as Code: https://dbdiagram.io/

   • Enterprise data modeling: https://sqldbm.com/

2. Data quality:

   • Data diff: https://www.datafold.com/data-diff

Comparisons

1. DBT vs SQLMesh

2. Datascale's SQL notes vs traditional saved queries or DBeaver's script management

// Last updated May 20, 2024