Automate BigQuery Integration Tasks from PowerShell

The CData ADO.NET Provider for BigQuery is a standard ADO.NET Provider that make it easy to accomplish data cleansing, normalization, backup, and other integration tasks by enabling real-time and bidirectional access to BigQuery.

Getting Started

---

ADO.NET Provider

The ADO.NET Provider provides a SQL interface for BigQuery; this tutorial shows how to use the Provider to create, retrieve, update, and delete BigQuery data.

Once you have acquired the necessary connection properties, accessing BigQuery data in PowerShell can be enabled in three steps.

Google uses the OAuth authentication standard. To access Google APIs on behalf of individual users, you can use the embedded credentials or you can register your own OAuth app.

OAuth also enables you to use a service account to connect on behalf of users in a Google Apps domain. To authenticate with a service account, register an application to obtain the OAuth JWT values.

In addition to the OAuth values, specify the DatasetId and ProjectId. See the "Getting Started" chapter of the help documentation for a guide to using OAuth.

1. Load the provider's assembly:

   
   [Reflection.Assembly]::LoadFile("C:\Program Files\CData\CData ADO.NET Provider for BigQuery\lib\System.Data.CData.GoogleBigQuery.dll")
       

2. Connect to BigQuery:

    
   $conn= New-Object System.Data.CData.GoogleBigQuery.GoogleBigQueryConnection("DataSetId=MyDataSetId;ProjectId=MyProjectId;InitiateOAuth=GETANDREFRESH;")
   $conn.Open()
   

3. Instantiate the GoogleBigQueryDataAdapter, execute an SQL query, and output the results:

   
   $sql="SELECT OrderName, Freight from Orders"
   
   $da= New-Object System.Data.CData.GoogleBigQuery.GoogleBigQueryDataAdapter($sql, $conn)
   $dt= New-Object System.Data.DataTable
   $da.Fill($dt)
   
   $dt.Rows | foreach {
   	Write-Host $_.ordername $_.freight
   }
     

Update BigQuery Data

$cmd =  New-Object System.Data.CData.GoogleBigQuery.GoogleBigQueryCommand("UPDATE Orders SET ShipCity='New York' WHERE Id = @myId", $conn)
$cmd.Parameters.Add((New-Object System.Data.CData.GoogleBigQuery.GoogleBigQueryParameter("@myId","10456255-0015501366")))
$cmd.ExecuteNonQuery()

Insert BigQuery Data

$cmd =  New-Object System.Data.CData.GoogleBigQuery.GoogleBigQueryCommand("INSERT INTO Orders (ShipCity) VALUES (@myShipCity)", $conn)
$cmd.Parameters.Add((New-Object System.Data.CData.GoogleBigQuery.GoogleBigQueryParameter("@myShipCity","New York")))
$cmd.ExecuteNonQuery()

Delete BigQuery Data

$cmd =  New-Object System.Data.CData.GoogleBigQuery.GoogleBigQueryCommand("DELETE FROM Orders WHERE Id=@myId", $conn)
$cmd.Parameters.Add((New-Object System.Data.CData.GoogleBigQuery.GoogleBigQueryParameter("@myId","001d000000YBRseAAH")))
$cmd.ExecuteNonQuery()