Automate Avro Integration Tasks from PowerShell

The CData ADO.NET Provider for Avro is a standard ADO.NET Provider that make it easy to accomplish data cleansing, normalization, backup, and other integration tasks by enabling real-time access to Avro.

ADO.NET Provider

The ADO.NET Provider provides a SQL interface for Avro; this tutorial shows how to use the Provider to retrieve Avro data.

Once you have acquired the necessary connection properties, accessing Avro data in PowerShell can be enabled in three steps.

Connect to your local Avro file(s) by setting the URI connection property to the location of the Avro file.

1. Load the provider's assembly:

   
   [Reflection.Assembly]::LoadFile("C:\Program Files\CData\CData ADO.NET Provider for Avro\lib\System.Data.CData.Avro.dll")
       

2. Connect to Avro:

    
   $conn= New-Object System.Data.CData.Avro.AvroConnection("URI=C:/folder/table.avro;InitiateOAuth=GETANDREFRESH;")
   $conn.Open()
   

3. Instantiate the AvroDataAdapter, execute an SQL query, and output the results:

   
   $sql="SELECT Id, Column1 from SampleTable_1"
   
   $da= New-Object System.Data.CData.Avro.AvroDataAdapter($sql, $conn)
   $dt= New-Object System.Data.DataTable
   $da.Fill($dt)
   
   $dt.Rows | foreach {
   	Write-Host $_.id $_.column1
   }