Description
This is a follow-up of #296
In that PR, support was introduced for async initFileFn
which allows to read from a stream and/or perform other async operation before Flow.js proceed with a given file.
One very important aspect of the FlowFile bootstrap is size determination (by default, from the regular file.size
) in order to compute the number of chunks needed. A initFileFn
could affect this computation.
FlowFile initialization is followed by the fileAdded
event which serves two purposes:
- As a hook, which allow consumer to
reject this file
- As a notification, which inform consumer that bootstrap is fully done and further processing [eg: flow.upload()] can be called (
Flow.upload()
would fails if theFlowFile.chunks
array was not computed)
One issue is that an async initFileFn
could finish after fileAdded
fires.
So one question is: Should the initFileFn
function really run if a file is going to be rejected?
If yes (eg, because rejection depends upon the number of chunks/file.size), then a PR could be made so that the event is fired after initFileFn
(even if async). It would move fileAdded
inside FlowFile's bootstrap function unless a nicer and higher-level solution exists (like of more pipeline-like vision of the process). I tend to think there should be less assumptions about file.size
at the beginning of the process (eg: FlowChunks could be built on-demand later) but this hook in its current shape forbids this.
Side note about Events: I think that having events to expect a return value makes them more hooks than events. It's a bit counter intuitive and keeps the code from moving to native Events. Not sure they are worth it.
The fileAdded
native event could just pass the corresponding (read-only) reference to Flow and the FlowFile and the consumer could delete
it as well (accessing to the global flow object). It'd be more flexible.