Building Modern Data Pipelines Using DBT

Back in May 2024, I had the opportunity to write a guest column for Database Trends and Applications Magazine, a publication I once where I once wrote a monthly column on Microsoft SQL Server and Azure SQL for many years. I thought I'd take a moment to bring it to your attention. I'd love to hear your feedback! Cheers, -Kevin Earlier this year, I had the privilege of serving on the organizing committee for the DataTune conference in my hometown of Nashville, Tenn. Unlike many database-specific or platform-specific conferences, which tend to focus on analytics and data science, … [Read more...]

Powerful ETL Technologies in the Microsoft Data Platform

POWERFUL ETL TECHNOLOGIES IN THE MICROSOFT DATA PLATFORM Microsoft first truly disrupted the ETL marketplace with the introduction of SQL Server Integration Services (SSIS) back with the release of SQL Server 2005. Microsoft has upped the ante yet again by bringing to market powerful ETL features to the cloud via the Azure Data Factory, which enables IT shops to integrate a multitude of data sources, both on-premises and in the cloud, via a workflow (called a "pipeline) that utilizes Hive, Pig, and customized C# programs. READ MORE... Originally Posted March 03, 2016 … [Read more...]

Simplifying CSV Data Loads

Data files containing comma separated values, or CSV, are some of the most common data formats used for data representation and storage outside the database.  When it comes to loading CSV data into the database, many options exist, however, few make it as simple as CSVexpress, powered by expressor software.  I recently visited www.csvexpress.com to check out just how simple it could get.   In short, CSVexpress offers a repeatable and quick way to load any CSV file into SQL Server (or any other database).   For those whose data quality is not as pristine as it should be, CSVexpress also offers … [Read more...]

ETL Demo with Data From Data.Gov

A little over a month ago, I wrote an article (Is There Such a Thing as Easy ETL) about expressor software and their desktop ETL application, expressor Studio.  I wrote about how it seemed much easier than the native ETL tools in SQL Server when I was reading up on the tool, but that the "proof would be in the pudding" so to speak when I actually tried it out loading some free (and incredibly useful) data from the US federal data clearinghouse, Data.Gov. If you'd rather not read my entire previous article - quick recap, expressor Studio uses “semantic types” to manage and abstract mappings … [Read more...]

Is There Such a Thing as Easy ETL?

E.T.L. That's Extract - Transform - Load.  That doesn't sound like a lot of work when all you need to get loaded is a simple Access database or an Excel spreadsheet.  In a situation like that, the process is so simple, all you really need to focus on is the L in ETL.  There's not a whole lot of E.T. to process, despite how wonderful that movie is. [pun intended]  But as soon as your data loading process involves some difficult or sophisticated cleansing or transformations, it gets really, really hard. The other cross-thread that had really caught my interest lately is the USA federal … [Read more...]