Skip to content

Latest commit

 

History

History

sql

Folders and files

NameName
Last commit message
Last commit date

parent directory

..
 
 
 
 
 
 

Example Pipelines for Beam SQL and Schema Transforms

The examples included in this module serve to demonstrate the basic functionality of Apache Beam SQL, and act as starting points for the development of more complex pipelines.

SQL transform

An example that leverage the powerful SQL syntax in Beam SqlTransform directly in your Beam pipelines.

SqlTransformExample is a simple pipeline that calculates multiple metrics per key : Min, Max and Sum.

Schema transform

Beam Schemas offer a flexible way of writing in code the same operations that are so easy to express in SQL.

SchemaTransformExample is a simple pipeline that calculates multiple metrics per key : Min, Max and Sum.

Running Examples

See Apache Beam WordCount Example for information on running these examples.

Gradle cmd line can be similar to:

./gradlew clean :examples:java:sql:execute --args="--runner=DirectRunner" -Pdirect-runner -PmainClass=org.apache.beam.examples.sql.SqlTransformExample

Beyond SQL and Schemas

Both SQL and Schema Transforms leverage Row type. The same results can be achieved directly using Beam transforms on a KV input PCollection. See Composed Cobiners example.