Skip to content

Bulk helper onFlush method #2553

Open
Open
@tomsquest

Description

@tomsquest

🚀 Feature Proposal

Hello,

Using the helper is convenient, but for a long running bulk, the operation will run without any feedback.

I am suggesting to have a method, onFlush that will be called on each flush operation.

The method would be similar to onSuccess but will receive the ongoing stats (number of success, errors so far).

Motivation

Ideally, I would like to have a feedback in terms of number of docs processed, instead of processed bytes.
This would make more sense for a human.

Example

await this.elastic.helpers.bulk<Document>({
  index: this.indexName,
  datasource: iterator,
  onDocument(doc) {
    return [
      { update: { _id: doc._id } },
      { doc_as_upsert: true, doc: omit(doc, "_id") },
    ];
  },
  onFlush({ result }) {
    console.log(`onFlush: '${result.total}'`);
  },
});

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions