Ana Suzuki·Sep 17, 2019Apache Flink on YARN with Kerberos AuthenticationSetting up Flink on YARN is written pretty much straight forward on the documentation. But what if what you need to do is much more…A response icon1A response icon1
Ana Suzuki·Jun 11, 2019Apache Spark Custom LoggingThe example below is only applicable if your spark job runs on yarn deployed on client.
Ana Suzuki·May 27, 2019Scala: List and Iterator SimulationA simple simulation on the difference between list and iterator. We all know that in Scala collection’s lineage, Iterable is very much the…
Ana Suzuki·Jan 16, 2019AVRO vs Parquet — what to use?I won’t say one is better and the other one is not as it totally depends where are they going to be used.A response icon1A response icon1
Ana Suzuki·Jun 13, 2018Who’s this Spark Listener?Jacek Laskowski made a good documentation regarding spark listeners. I made this page since we keep on encountering this ERROR:
Ana Suzuki·Oct 24, 2017Relocating Classes using Apache Maven Shade PluginI just had a weird encounter upon running Spark Job.
Ana Suzuki·Oct 4, 2017Import Mysql data to HDFS using SqoopThe trending topic in big data is now about AIs. It’s quite weird that I am posting something that’s not new in the big data world. This…
Ana Suzuki·Jul 5, 2017Streaming Kafka Receiver-Less ApproachKindly read https://spark.apache.org/docs/latest/streaming-kafka-0-8-integration.html for more information about Receiver-Less Approach.