modern
data integration
driven by
Azure Data Studio
We're Changing the Data Integration Landscape
The paradigm for building ELT/ETL logic has been established based on legacy applications, which box you into a separate platform, instead of relying on your existing toolsets. The question to ask is:
“Does this still make sense for modern Data Integration solutions?”
Generally, more than 80% of work that goes into an ELT/ETL solution is centric to SQL coding
Extract from a source system with an SQL statement, and load to a destination database
Transform data, perhaps into a dimensional model, using SQL statements
EL
T
adds the missing 20% needed for ELT/ETL orchestration
Your ELT/ETL developers can now work in the comfort of Azure Data Studio, or any other SQL editor, to rapidly create Data Integration projects, with natively supported features.
Git Source
Control
SQL Editor Intellisense
Rich Documentation
with Notebooks
Simple Built-In
Extensibility
SQL Editor Based
Parameterization
If you know SQL, you'll know notebookSnap
Virtually all ELT/ETL tools on the market try to box you into a graphical Point and Click, Drag and Drop development paradigm. While this may lower the threshold for getting started, it also separates the developer from native SQL, which is ultimately how you interact, and have the greatest control and flexibility when working with a modern Database system. A legacy ETL tool will generally:
Preclude full SQL Database Platform capabilities
Hinder code reuse (a fundamental to all coding efforts)
Interfere with source control and versioning
Yield slower and more error-prone maintenance cycles
Lead to vendor-lock
Extended Notebook Features
Multi-Kernel
Provides Multi-Kernel Capabilities
EL & T
Dataflows & Transformations Enabled Using ADS Notebook Cells
Runtime
Includes a Python Runtime
Execute
Projects can Shell to any Command Line Interface
Parameters
Supports Environment Parameterization
Enabling
Enables Data Integration Using only ADS Notebooks
Why Notebooks?
Notebooks combine parameterized SQL code, rich documentation, along with sample query results. These components are methodically chained together, providing both a solution, and step-by-step development instructions. In fact, the notebook doubles as valuable documentation, which includes readable story of the solution for both the business, and the ELT/ETL development team.
Why Azure Data Studio?
Azure Data Studio (ADS) is representative of the next generation of software from Microsoft. It is Open Source, Cross-Platform, Multi-Database, Highly Extensible and best of all, FREE! In addition, despite having ‘Azure’ in its name, ADS works just as well with your on-premises database systems.
Azure Data Studio also offers Notebooks as a first-class citizen, providing perfect synergy between Jupyter Notebooks and the SQL language. notebookSnap simply adds a layer of metadata to your ADS files providing convertibility of your notebooks, as well any .SQL files from your favorite editor, into fully orchestrated ELT solutions. Simple and elegant repurposing of your existing (and future) SQL assets!
Azure Data Studio Integration
Our Azure Data Studio (ADS) extension works directly in a Notebook, allowing you to annotate Code cells with metadata sufficient to convert the SQL into Dataflows and Transformations. This is accomplished by ingesting your notebooks into our Free (SQL Express) database, which then enables our Python based runtime to execute via a simple command line interface.
Multiple Runtime Framework Options:
Runtime Logging with Row Counts
Informational Emails with Alerts
Environment Parameter Configurations
Intelligent Restart-Ability
Power BI Dashboard Integration (included)
Query Parameterization - A Focus On Process
A common problem with SQL development for Data Integration projects is coding to support multiple environments (i.e., Dev, Test, Prod) when going through a deployment. NotebookSnap provides snippet driven parameters, allowing for full developer unit testing, without also requiring subsequent code changes to the deployment process. This key requirement is made possible by fusing parameter snippets in ADS (as well as other SQL editors) with automated ‘downstream’ parameter management.
This approach to parameterization not only saves time, but also lets your developers focus on “business requirements”, while easily facilitating your Continuous Integration and Deployment (CI/CD) processes.
New to Notebooks? We've got you covered
BI Tracks is dedicated to helping Azure Data Studio (ADS) users through tailored videos and live webinars. We believe ADS represents the future of data engineering and integration, and our growing video catalog speaks to our enthusiasm. Join us on this journey by subscribing to our channel, and learning more about what Azure Data Studio has to offer.