Observability

Several client features help you observe and measure Elasticsearch client usage. As of version 8.15.0, the JavaScript client provides native support for OpenTelemetry. You can send client usage data to OpenTelemetry endpoints without making changes to your JavaScript codebase.

Rather than providing a default logger, the client offers an event emitter interface to hook into internal events like request and response. This allows you to log significant events or otherwise react to client usage. Because correlating events can be complex, the client provides a correlation ID system and other features.

The client supports OpenTelemetry’s zero-code instrumentation to enable tracking each client request as an OpenTelemetry span. These spans follow all of the semantic OpenTelemetry conventions for Elasticsearch except for db.query.text.

To start sending Elasticsearch trace data to your OpenTelemetry endpoint, instrument the client using the Elastic Distribution of OpenTelemetry (EDOT) JavaScript, or follow OpenTelemetry’s zero-code instrumentation guide.

As of @elastic/transport version 9.1.0—or 8.10.0 when using @elastic/elasticsearch 8.x—you can turn off OpenTelemetry tracing in several ways.

To entirely turn off OpenTelemetry collection, you can provide a custom Transport at client instantiation time that sets openTelemetry.enabled to false:

import { Transport } from '@elastic/transport'

class MyTransport extends Transport {
  async request(params, options = {}): Promise<any> {
    options.openTelemetry = { enabled: false }
    return super.request(params, options)
  }
}

const client = new Client({
  node: '...',
  auth: { ... },
  Transport: MyTransport
})
		

Alternatively, you can export the environment variable OTEL_ELASTICSEARCH_ENABLED=false.

To suppress tracing without turning off all OpenTelemetry collection, use the option openTelemetry.suppressInternalInstrumentation = true instead.

If you would like to keep either option enabled by default, but want to turn them off for a single API call, pass Transport options as a second argument to any API function call:

const response = await client.search({ ... }, {
  openTelemetry: { enabled: false }
})
		

The client is an event emitter. This means that you can listen for its events to add additional logic to your code, without needing to change the client’s internals or how you use the client. You can find the events' names by accessing the events key of the client:

const { events } = require('@elastic/elasticsearch')
console.log(events)
		

The event emitter functionality can be useful if you want to log every request, response or error that is created by the client:

const logger = require('my-logger')()
const { Client } = require('@elastic/elasticsearch')
const client = new Client({
  cloud: { id: '<cloud-id>' },
  auth: { apiKey: 'base64EncodedKey' }
})

client.diagnostic.on('response', (err, result) => {
  if (err) {
    logger.error(err)
  } else {
    logger.info(result)
  }
})
		

The client emits the following events:

Emitted before starting serialization and compression. If you want to measure this phase duration, you should measure the time elapsed between this event and request.

client.diagnostic.on("serialization", (err, result) => {
  console.log(err, result)
})
		

Emitted before sending the actual request to Elasticsearch (emitted multiple times in case of retries).

client.diagnostic.on("request", (err, result) => {
  console.log(err, result)
})
		

Emitted before starting deserialization and decompression. If you want to measure this phase duration, you should measure the time elapsed between this event and response.

This event might not be emitted in certain situations:

• When asStream is set to true, the response is returned in its raw stream form before deserialization occurs
• When a response is terminated early due to content length being too large
• When a response is terminated early by an AbortController

client.diagnostic.on("deserialization", (err, result) => {
  console.log(err, result)
})
		

Emitted once Elasticsearch response has been received and parsed.

client.diagnostic.on("response", (err, result) => {
  console.log(err, result)
})
		

Emitted when the client ends a sniffing request.

client.diagnostic.on("sniff", (err, result) => {
  console.log(err, result)
})
		

Emitted if the client is able to resurrect a dead node.

client.diagnostic.on("resurrect", (err, result) => {
  console.log(err, result)
})
		

The values of result in serialization, request, deserialization, response and sniff are:

body: any;
statusCode: number | null;
headers: anyObject | null;
warnings: string[] | null;
meta: {
  context: any;
  name: string;
  request: {
    params: TransportRequestParams;
    options: TransportRequestOptions;
    id: any;
  };
  connection: Connection;
  attempts: number;
  aborted: boolean;
  sniff?: {
    hosts: any[];
    reason: string;
  };
};
		

While the result value in resurrect is:

strategy: string;
isAlive: boolean;
connection: Connection;
name: string;
request: {
  id: any;
};
		

The event order is described in the following graph, in some edge cases, the order is not guaranteed. You can find in test/acceptance/events-order.test.js how the order changes based on the situation.

serialization
  │
  │ (serialization and compression happens between those two events)
  │
  └─▶ request
        │
        │ (actual time spent over the wire)
        │
        └─▶ deserialization
              │
              │ (deserialization and decompression happens between those two events)
              │
              └─▶ response
		

Correlating events can be hard, especially if there are many events at the same time. The client offers you an automatic (and configurable) system to help you handle this problem.

const { Client } = require('@elastic/elasticsearch')
const client = new Client({
  cloud: { id: '<cloud-id>' },
  auth: { apiKey: 'base64EncodedKey' }
})

client.diagnostic.on('request', (err, result) => {
  const { id } = result.meta.request
  if (err) {
    console.log({ error: err, reqId: id })
  }
})

client.diagnostic.on('response', (err, result) => {
  const { id } = result.meta.request
  if (err) {
    console.log({ error: err, reqId: id })
  }
})

client.search({
  index: 'my-index',
  query: { match_all: {} }
}).then(console.log, console.log)
		

By default the ID is an incremental integer, but you can configure it with the generateRequestId option:

const { Client } = require('@elastic/elasticsearch')
const client = new Client({
  cloud: { id: '<cloud-id>' },
  auth: { apiKey: 'base64EncodedKey' },
  // it takes two parameters, the request parameters and options
  generateRequestId: function (params, options) {
    // your id generation logic
    // must be synchronous
    return 'id'
  }
})
		

You can also specify a custom ID per request:

client.search({
  index: 'my-index',
  query: { match_all: {} }
}, {
  id: 'custom-id'
}).then(console.log, console.log)
		

Sometimes, you might need to make some custom data available in your events, you can do that via the context option of a request:

const { Client } = require('@elastic/elasticsearch')
const client = new Client({
  cloud: { id: '<cloud-id>' },
  auth: { apiKey: 'base64EncodedKey' }
})

client.diagnostic.on('request', (err, result) => {
  const { id } = result.meta.request
  const { context } = result.meta
  if (err) {
    console.log({ error: err, reqId: id, context })
  }
})

client.diagnostic.on('response', (err, result) => {
  const { id } = result.meta.request
  const { winter } = result.meta.context
  if (err) {
    console.log({ error: err, reqId: id, winter })
  }
})

client.search({
  index: 'my-index',
  query: { match_all: {} }
}, {
  context: { winter: 'is coming' }
}).then(console.log, console.log)
		

The context object can also be configured as a global option in the client configuration. If you provide both, the two context objects will be shallow merged, and the API level object will take precedence.

const { Client } = require('@elastic/elasticsearch')
const client = new Client({
  cloud: { id: '<cloud-id>' },
  auth: { apiKey: 'base64EncodedKey' },
  context: { winter: 'is coming' }
})

client.diagnostic.on('request', (err, result) => {
  const { id } = result.meta.request
  const { context } = result.meta
  if (err) {
    console.log({ error: err, reqId: id, context })
  }
})

client.diagnostic.on('response', (err, result) => {
  const { id } = result.meta.request
  const { winter } = result.meta.context
  if (err) {
    console.log({ error: err, reqId: id, winter })
  }
})

client.search({
  index: 'my-index',
  query: { match_all: {} }
}, {
  context: { winter: 'has come' }
}).then(console.log, console.log)
		

If you are using multiple instances of the client or if you are using multiple child clients (which is the recommended way to have multiple instances of the client), you might need to recognize which client you are using. The name options help you in this regard.

const { Client } = require('@elastic/elasticsearch')
const client = new Client({
  cloud: { id: '<cloud-id>' },
  auth: { apiKey: 'base64EncodedKey' },
  name: 'parent-client'
})

const child = client.child({
  name: 'child-client'
})

console.log(client.name, child.name)

client.diagnostic.on('request', (err, result) => {
  const { id } = result.meta.request
  const { name } = result.meta
  if (err) {
    console.log({ error: err, reqId: id, name })
  }
})

client.diagnostic.on('response', (err, result) => {
  const { id } = result.meta.request
  const { name } = result.meta
  if (err) {
    console.log({ error: err, reqId: id, name })
  }
})

client.search({
  index: 'my-index',
  query: { match_all: {} }
}).then(console.log, console.log)

child.search({
  index: 'my-index',
  query: { match_all: {} }
}).then(console.log, console.log)
		

1. default to 'elasticsearch-js'

To improve observability, the client offers an easy way to configure the X-Opaque-Id header. If you set the X-Opaque-Id in a specific request, this allows you to discover this identifier in the deprecation logs, helps you with identifying search slow log origin as well as identifying running tasks.

The X-Opaque-Id should be configured in each request, for doing that you can use the opaqueId option, as you can see in the following example. The resulting header will be { 'X-Opaque-Id': 'my-search' }.

const { Client } = require('@elastic/elasticsearch')
const client = new Client({
  cloud: { id: '<cloud-id>' },
  auth: { apiKey: 'base64EncodedKey' }
})

client.search({
  index: 'my-index',
  body: { foo: 'bar' }
}, {
  opaqueId: 'my-search'
}).then(console.log, console.log)
		

Sometimes it may be useful to prefix all the X-Opaque-Id headers with a specific string, in case you need to identify a specific client or server. For doing this, the client offers a top-level configuration option: opaqueIdPrefix. In the following example, the resulting header will be { 'X-Opaque-Id': 'proxy-client::my-search' }.

const { Client } = require('@elastic/elasticsearch')
const client = new Client({
  cloud: { id: '<cloud-id>' },
  auth: { apiKey: 'base64EncodedKey' },
  opaqueIdPrefix: 'proxy-client::'
})

client.search({
  index: 'my-index',
  body: { foo: 'bar' }
}, {
  opaqueId: 'my-search'
}).then(console.log, console.log)