Automate CSV Integration Tasks from PowerShell

The CData ADO.NET Provider for CSV is a standard ADO.NET Provider that make it easy to accomplish data cleansing, normalization, backup, and other integration tasks by enabling real-time access to CSV.

ADO.NET Provider

The ADO.NET Provider provides a SQL interface for CSV; this tutorial shows how to use the Provider to retrieve CSV data.

Once you have acquired the necessary connection properties, accessing CSV data in PowerShell can be enabled in three steps.

Connecting to Local or Cloud-Stored (Box, Google Drive, Amazon S3, SharePoint) CSV Files

CData Drivers let you work with CSV files stored locally and stored in cloud storage services like Box, Amazon S3, Google Drive, or SharePoint, right where they are.

Setting connection properties for local files

Set the URI property to local folder path.

Setting connection properties for files stored in Amazon S3

To connect to CSV file(s) within Amazon S3, set the URI property to the URI of the Bucket and Folder where the intended CSV files exist. In addition, at least set these properties:

• AWSAccessKey: AWS Access Key (username)
• AWSSecretKey: AWS Secret Key

Setting connection properties for files stored in Box

To connect to CSV file(s) within Box, set the URI property to the URI of the folder that includes the intended CSV file(s). Use the OAuth authentication method to connect to Box.

Dropbox

To connect to CSV file(s) within Dropbox, set the URI proprerty to the URI of the folder that includes the intended CSV file(s). Use the OAuth authentication method to connect to Dropbox. Either User Account or Service Account can be used to authenticate.

SharePoint Online (SOAP)

To connect to CSV file(s) within SharePoint with SOAP Schema, set the URI proprerty to the URI of the document library that includes the intended CSV file. Set User, Password, and StorageBaseURL.

SharePoint Online REST

To connect to CSV file(s) within SharePoint with REST Schema, set the URI proprerty to the URI of the document library that includes the intended CSV file. StorageBaseURL is optional. If not set, the driver will use the root drive. OAuth is used to authenticate.

Google Drive

To connect to CSV file(s) within Google Drive, set the URI property to the URI of the folder that includes the intended CSV file(s). Use the OAuth authentication method to connect and set InitiateOAuth to GETANDREFRESH.

1. Load the provider's assembly:

   
   [Reflection.Assembly]::LoadFile("C:\Program Files\CData\CData ADO.NET Provider for CSV\lib\System.Data.CData.CSV.dll")
       

2. Connect to CSV:

    
   $conn= New-Object System.Data.CData.CSV.CSVConnection("URI=/PATH/TO/MyCSVFilesFolder;")
   $conn.Open()
   

3. Instantiate the CSVDataAdapter, execute an SQL query, and output the results:

   
   $sql="SELECT City, TotalDue from Customer"
   
   $da= New-Object System.Data.CData.CSV.CSVDataAdapter($sql, $conn)
   $dt= New-Object System.Data.DataTable
   $da.Fill($dt)
   
   $dt.Rows | foreach {
   	Write-Host $_.city $_.totaldue
   }