报错信息
xception in thread "main" java.lang.IllegalArgumentException: Hash collision on user-specified ID "asdfasdfasfdfLiveCartInfoMergeProcessUid". Most likely cause is a non-unique ID. Please check that all IDs specified via `uid(String)` are unique.
at org.apache.flink.streaming.api.graph.StreamGraphHasherV2.generateNodeHash(StreamGraphHasherV2.java:185)
at org.apache.flink.streaming.api.graph.StreamGraphHasherV2.traverseStreamGraphAndGenerateHashes(StreamGraphHasherV2.java:110)
at org.apache.flink.streaming.api.graph.StreamingJobGraphGenerator.createJobGraph(StreamingJobGraphGenerator.java:171)
at org.apache.flink.streaming.api.graph.StreamingJobGraphGenerator.createJobGraph(StreamingJobGraphGenerator.java:117)
at org.apache.flink.streaming.api.graph.StreamGraph.getJobGraph(StreamGraph.java:934)
at org.apache.flink.client.StreamGraphTranslator.translateToJobGraph(StreamGraphTranslator.java:50)
at org.apache.flink.client.FlinkPipelineTranslationUtil.getJobGraph(FlinkPipelineTranslationUtil.java:39)
at org.apache.flink.client.deployment.executors.PipelineExecutorUtils.getJobGraph(PipelineExecutorUtils.java:56)
at org.apache.flink.client.deployment.executors.LocalExecutor.getJobGraph(LocalExecutor.java:104)
at org.apache.flink.client.deployment.executors.LocalExecutor.execute(LocalExecutor.java:82)
at org.apache.flink.streaming.api.environment.StreamExecutionEnvironment.executeAsync(StreamExecutionEnvironment.java:1957)
at org.apache.flink.streaming.api.environment.StreamExecutionEnvironment.execute(StreamExecutionEnvironment.java:1848)
at org.apache.flink.streaming.api.environment.LocalStreamEnvironment.execute(LocalStreamEnvironment.java:69)
at org.apache.flink.streaming.api.environment.StreamExecutionEnvironment.execute(StreamExecutionEnvironment.java:1834)
at com.ywwl.ywdc.live.main.LiveFansCart.main(LiveFansCart.java:90)
Disconnected from the target VM, address: 'javadebug', transport: 'shared memory'
原因分析
上游消费不同的kafka数据,下游同时调用一个方法;在拓扑关系中,就是两个流程;但是在方法中,使用了状态编程,用到了uid;在拓扑关系中,就是在两个流中存在同样的uid,导致uid不唯一,报出上述错误
解决
①上游不同的流,调用不同的方法,排除uid相同的问题;
②被上游不同流调用的方法中,不使用状态编程;
在Flink作业执行时遇到IllegalArgumentException,报错信息指出存在Hash碰撞,原因是用户指定的IDasdfasdfasfdfLiveCartInfoMergeProcessUid不唯一。解决方法包括确保上游不同流调用不同的方法以避免uid冲突,或者在被多个流调用的方法中避免使用状态编程,从而消除uid在同一拓扑中出现多次的情况。
1267

被折叠的 条评论
为什么被折叠?



