Loading

Setting up machine learning

Serverless Stack

To use the Elastic Stack machine learning features, you must have:

  • the appropriate subscription level or the free trial period activated

  • xpack.ml.enabled set to its default value of true on every node in the cluster (refer to Machine learning settings in Elasticsearch)

  • ml value defined in the list of node.roles on the machine learning nodes

  • machine learning features visible in the Kibana space

  • security privileges assigned to the user that:

    • grant use of machine learning features, and
    • grant access to source and destination indices.
Tip

The fastest way to get started with machine learning features is to start a free 14-day trial of Elastic Cloud.

Assigning security privileges affects how users access machine learning features. Consider the two main categories:

  • Elasticsearch API user: uses an Elasticsearch client, cURL, or Kibana Dev Tools to access machine learning features via Elasticsearch APIs. It requires Elasticsearch security privileges.
  • Kibana user: uses the machine learning features in Kibana and does not use Dev Tools. It requires either Kibana feature privileges or Elasticsearch security privileges and is granted the most permissive combination of both. Kibana feature privileges are recommended if you control job level visibility via Spaces. Machine learning features must be visible in the relevant space. Refer to Feature visibility in Spaces for configuration information.

You can configure these privileges

  • under the Roles and Spaces management pages. Find these pages in the main menu or use the global search field.
  • via the respective Elasticsearch security APIs.

If you use machine learning APIs, you must have the following cluster and index privileges:

For full access:

  • machine_learning_admin built-in role or the equivalent cluster privileges
  • read and view_index_metadata on source indices
  • read, manage, and index on destination indices (for data frame analytics analytics jobs only)

For read-only access:

  • machine_learning_user built-in role or the equivalent cluster privileges
  • read index privileges on source indices
  • read index privileges on destination indices (for data frame analytics analytics jobs only)
Important

The machine_learning_admin and machine_learning_user built-in roles give access to the results of all anomaly detection jobs, irrespective of whether the user has access to the source indices. You must carefully consider who is given these roles, as anomaly detection job results may propagate field values that contain sensitive information from the source indices to the results.

Important

Granting All or Read Kibana feature privilege for Machine Learning will also grant the role the equivalent feature privileges to certain types of Kibana saved objects, namely index patterns, dashboards, saved searches, and visualizations as well as machine learning job, trained model and module saved objects.

In Kibana, the machine learning features must be visible in your space. To manage which features are visible in your space, go to the Spaces management page using the navigation menu or the global search field.

Manage spaces in Kibana

In addition to index privileges, source data views must also exist in the same space as your machine learning jobs. You can configure these under Data Views. To open Data Views, find Stack Management > Kibana in the main menu, or use the global search field.

Each machine learning job and trained model can be assigned to all, one, or multiple spaces. This can be configured in Machine Learning. To open Machine Learning, find the page in the main menu, or use the global search field. You can edit the spaces that a job or model is assigned to by clicking the icons in the Spaces column.

Assign machine learning jobs to spaces

Within a Kibana space, for full access to the machine learning features, you must have:

  • Machine Learning: All Kibana privileges
  • Data Views Management: All Kibana feature privileges
  • read, and view_index_metadata index privileges on your source indices
  • data views for your source indices
  • data views, read, manage, and index index privileges on destination indices (for data frame analytics analytics jobs only)

Within a Kibana space, for read-only access to the machine learning features, you must have:

  • Machine Learning: Read Kibana privileges
  • data views for your source indices
  • read index privilege on your source indices
  • data views and read index privileges on destination indices (for data frame analytics analytics jobs only)
Important

A user who has full or read-only access to machine learning features within a given Kibana space can view the results of all anomaly detection jobs that are visible in that space, even if they do not have access to the source indices of those jobs. You must carefully consider who is given access to machine learning features, as anomaly detection job results may propagate field values that contain sensitive information from the source indices to the results.

Note

Data views can be automatically created when creating a data frame analytics analytics job.

For access to use machine learning APIs via Dev Tools in Kibana, set the Elasticsearch security privileges and grant access to machine_learning_admin or machine_learning_user built-in roles.

Within a Kibana space, to upload and import files in the Data Visualizer, you must have:

  • Machine Learning: Read or Discover: All Kibana feature privileges
  • Data Views Management: All Kibana feature privileges
  • ingest_admin built-in role, or manage_ingest_pipelines cluster privilege
  • create, create_index, manage and read index privileges for destination indices

For more information, see Security privileges and Kibana privileges.