Skip to content

[pull] master from elastic:master #47

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 42 commits into
base: master
Choose a base branch
from
Open

Conversation

pull[bot]
Copy link

@pull pull bot commented Jul 21, 2022

See Commits and Changes for more details.


Created by pull[bot]

Can you help keep this open source service alive? 💖 Please sponsor : )

mark-vieira and others added 16 commits July 21, 2022 12:24
Use custom task instead of generic AckedClusterStateUpdateTask that is not
intended to be used with batching.
This adds support for the `cardinality` aggregation within a random_sampler.

This usecase is helpful in determining the ratio of unique values compared to the count of total documents within the sampled set.
Originally released in 7.14, bucket_correlation is now generally available.
)

When we cannot scale up (autoscaling is disabled) we should fail
requests to start a trained model deployment whose allocations
cannot be provided.
This was using health request assuming yellow index is one with all primaries
initialized. This is not true as newly created index remain yellow when
primaries allocation is throttled. This was discovered while working on a new
shards allocator.
Corrects the syntax of an example for a scripted metric aggregation.
…88626)

This commit adds the ability to register a PeerFinderListener with Coordinator. The listener has an
onFoundPeersUpdated that is called whenever the collection of peers changes (whether added to
or removed from).
We would like to start running all test sets on the feature branch to make sure
it is ready to be merged back to the main branch.
API docs for the new PUT /_security/api_key/<id> API.
… scenarios (#88623)

With 8.4, trained model assignments now take CPU into consideration. This changes our calculation for scaling down until we fully support autoscaling according to CPU requirements.

We shouldn't allow scaling down if there is ANY model assignment that isn't fully allocated (meaning, not enough CPUs)
We don't allow scaling down unless model assignments require less than half of the current scale's CPU count.
Point 2 is a place holder. Fix 1 will be a requirement even in the future with vCPU autoscaling.
@pull pull bot added the ⤵️ pull label Jul 21, 2022
Hendrik Muhs and others added 13 commits July 22, 2022 07:38
)

ensure transform has done something before stop gets called with wait_for_checkpoint=true

fixes #79621
There is no need to use stopRandomNode/namePredicate combination to stop a node by its name.
…ster state (#88652)

Revert of #46147, we want to keep this functionality around for a little longer.
This PR adds audit logging for API key metadata, when an API key is
created, granted, or updated.
Our docs currently recommend PBKDF2 as a cache hasher in FIPS mode.
However, the performance overhead of PBKDF2 is prohibitive; ssha256
is a more appropriate choice for in-memory credential hashing. This PR
updates the docs to reflect this. See #86740 for more context.
Making this a little smaller so it inlines more often and more importantly
avoid getting the current field name redundantly before entering the loop which
is not free when the parser is a `DotExpandingXContentParser`.
This change gives a significant boost to `BeatsMapperBenchmark.benchmarkParseKeywordFields`.

Results below are pretty stable, running with a large number of iterations.

before:

```
Result "org.elasticsearch.benchmark.index.mapper.BeatsMapperBenchmark.benchmarkParseKeywordFields":
  9039.270 ±(99.9%) 28.001 ns/op [Average]
  (min, avg, max) = (9008.778, 9039.270, 9062.246), stdev = 18.521
  CI (99.9%): [9011.269, 9067.271] (assumes normal distribution)
```

after:

```
Result "org.elasticsearch.benchmark.index.mapper.BeatsMapperBenchmark.benchmarkParseKeywordFields":
  8645.649 ±(99.9%) 53.677 ns/op [Average]
  (min, avg, max) = (8568.319, 8645.649, 8688.210), stdev = 35.504
  CI (99.9%): [8591.972, 8699.327] (assumes normal distribution)
```
weizijun and others added 13 commits July 22, 2022 15:26
Improve RollupShardIndexer by reducing the calls to BytesRef.deepCopyOf(tsid)
Today each time there's shards to snapshot we enqueue a lambda which
captures the current `SnapshotsInProgress$Entry`. This is a pretty
heavyweight object, possibly several MB in size, most of which is not
necessary to capture, and with concurrent snapshots across thousands of
shards we may enqueue many hundreds of slightly different such objects.
With this commit we compute a more efficient representation of the work
to be done by each task in the queue instead.

Relates #77466
This PR adds a new `knn` option to the `_search` API to support ANN search.
It's powered by the same Lucene ANN capabilities as the old `_knn_search`
endpoint. The `knn` option can be combined with other search features like
queries and aggregations.

Addresses #87625
ClosePointInTimeRequest#validate should return a validation error for an 
invalid pit_id, but it throws an IAE instead. This mistake prevents
close_point_in_time tasks of requests with empty pit_id from being removed.
If there is no shard failure we would like to use a generic
Internal Server Error other than a Service Unavailable.
Moreover, if the cause is known we use a status code that
reflects the cause.
* Convert disk watermarks to RelativeByteSizeValues

Similar to the existing watermark setting for the frozen tier.

Pre-requisite for PR 88639 that plans to introduce max headroom
settings for the disk watermarks, similar to the frozen tier max
headroom setting.

* Add changelog

* Revert 20gb to 20GB

* Make formatNoTrailingZerosPercent non static

* ByteSizeValue.MINUS_ONE

* Remove getMinimumTotalSizeForBelowWatermark

* Remove comment

* Fix minor stuff

* Make parsing of RelativeByteSizeValue faster

Mimicks older definitelyNotPercentage function

* Remove Locale from Strings.format

* More MINUS_ONE
The KnnSearchBuilder constructor validates that k is always less than num_cands.
But the mutateInstance method could increase k past num_cands. Now we construct
the instance with a larger num_cands to make sure this never happens.

Closes #88734.
@mark-vieira mark-vieira deleted the branch soitun:master July 22, 2022 23:15
@mark-vieira mark-vieira deleted the master branch July 22, 2022 23:15
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.