Powerful ETL Technologies in the Microsoft Data Platform

POWERFUL ETL TECHNOLOGIES IN THE MICROSOFT DATA PLATFORM Microsoft first truly disrupted the ETL marketplace with the introduction of SQL Server Integration Services (SSIS) back with the release of SQL Server 2005. Microsoft has upped the ante yet again by bringing to market powerful ETL features to the cloud via the Azure Data Factory, which enables IT shops to integrate a multitude of data sources, both on-premises and in the cloud, via a workflow (called a "pipeline) that utilizes Hive, Pig, and customized C# programs. READ MORE... Originally Posted March 03, 2016 … [Read more...]

Simplifying CSV Data Loads

Data files containing comma separated values, or CSV, are some of the most common data formats used for data representation and storage outside the database.  When it comes to loading CSV data into the database, many options exist, however, few make it as simple as CSVexpress, powered by expressor software.  I recently visited www.csvexpress.com to check out just how simple it could get.   In short, CSVexpress offers a repeatable and quick way to load any CSV file into SQL Server (or any other database).   For those whose data quality is not as pristine as it should be, CSVexpress also offers … [Read more...]

ETL Demo with Data From Data.Gov

A little over a month ago, I wrote an article (Is There Such a Thing as Easy ETL) about expressor software and their desktop ETL application, expressor Studio.  I wrote about how it seemed much easier than the native ETL tools in SQL Server when I was reading up on the tool, but that the "proof would be in the pudding" so to speak when I actually tried it out loading some free (and incredibly useful) data from the US federal data clearinghouse, Data.Gov. If you'd rather not read my entire previous article - quick recap, expressor Studio uses “semantic types” to manage and abstract mappings … [Read more...]

Is There Such a Thing as Easy ETL?

E.T.L. That's Extract - Transform - Load.  That doesn't sound like a lot of work when all you need to get loaded is a simple Access database or an Excel spreadsheet.  In a situation like that, the process is so simple, all you really need to focus on is the L in ETL.  There's not a whole lot of E.T. to process, despite how wonderful that movie is. [pun intended]  But as soon as your data loading process involves some difficult or sophisticated cleansing or transformations, it gets really, really hard. The other cross-thread that had really caught my interest lately is the USA federal … [Read more...]