refactor(semantic_text): fail early in pre-8.11 indices #133080

mromaios · 2025-08-18T16:28:17Z

Reference Issue: #132551

In this PR we:

raise a mapper parsing exception when trying to create a semantic_text field in older indices. (Indices created prior to 8.11.0)

This happens for consistency as for

dense_vector fields, incompatible defaults are set including index=false
sparse_vector fields, were not supported in 8.0-8.10 and not indexed in 7.x.

elasticsearchmachine · 2025-08-18T16:30:43Z

Hi @mromaios, I've created a changelog YAML for you.

Mikep86

Looks to be on the right path to me! Testing will reveal whether the deprecation logger is emitting a warning header like we want to in this case.

...nference/src/main/java/org/elasticsearch/xpack/inference/mapper/SemanticTextFieldMapper.java

…aios/elasticsearch into 132551_enable_dense_vector_indexing

...nference/src/main/java/org/elasticsearch/xpack/inference/mapper/SemanticTextFieldMapper.java

elasticsearchmachine · 2025-08-19T14:38:48Z

Pinging @elastic/search-eng (Team:SearchOrg)

elasticsearchmachine · 2025-08-19T14:38:48Z

Pinging @elastic/search-relevance (Team:Search - Relevance)

…aios/elasticsearch into 132551_enable_dense_vector_indexing

* fix(semantic_text): index underlying dense_vector field in older indices * Update docs/changelog/133080.yaml * [CI] Auto commit changes from spotless * update msg, change versions, add ut * indent * remove todo * [CI] Auto commit changes from spotless * update msg * [CI] Auto commit changes from spotless * add densevectormapper indexed ut * [CI] Auto commit changes from spotless * fix ut * undo accidental ut removal * [CI] Auto commit changes from spotless * Update docs/changelog/133080.yaml Co-authored-by: Kathleen DeRusso <[email protected]> * remove sparse_vector exception change * [CI] Auto commit changes from spotless * reverting most of the stuff * removing unused import, rename test * [CI] Auto commit changes from spotless * syntax * revisit changelog * revisit changelog * Update docs/changelog/133080.yaml Co-authored-by: Kathleen DeRusso <[email protected]> * Update x-pack/plugin/inference/src/main/java/org/elasticsearch/xpack/inference/mapper/SemanticTextFieldMapper.java Co-authored-by: Kathleen DeRusso <[email protected]> * changelog format * fix UT --------- Co-authored-by: elasticsearchmachine <[email protected]> Co-authored-by: Kathleen DeRusso <[email protected]>

elasticsearchmachine · 2025-08-26T18:20:09Z

💔 Backport failed

Status	Branch	Result
✅	9.1
❌	9.0	Commit could not be cherrypicked due to conflicts
❌	8.18	Commit could not be cherrypicked due to conflicts
❌	8.19	Commit could not be cherrypicked due to conflicts

You can use sqren/backport to manually backport by running backport --upstream elastic/elasticsearch --pr 133080

* fix(semantic_text): index underlying dense_vector field in older indices * Update docs/changelog/133080.yaml * [CI] Auto commit changes from spotless * update msg, change versions, add ut * indent * remove todo * [CI] Auto commit changes from spotless * update msg * [CI] Auto commit changes from spotless * add densevectormapper indexed ut * [CI] Auto commit changes from spotless * fix ut * undo accidental ut removal * [CI] Auto commit changes from spotless * Update docs/changelog/133080.yaml Co-authored-by: Kathleen DeRusso <[email protected]> * remove sparse_vector exception change * [CI] Auto commit changes from spotless * reverting most of the stuff * removing unused import, rename test * [CI] Auto commit changes from spotless * syntax * revisit changelog * revisit changelog * Update docs/changelog/133080.yaml Co-authored-by: Kathleen DeRusso <[email protected]> * Update x-pack/plugin/inference/src/main/java/org/elasticsearch/xpack/inference/mapper/SemanticTextFieldMapper.java Co-authored-by: Kathleen DeRusso <[email protected]> * changelog format * fix UT --------- Co-authored-by: elasticsearchmachine <[email protected]> Co-authored-by: Kathleen DeRusso <[email protected]> (cherry picked from commit 8f41a4b) # Conflicts: # server/src/test/java/org/elasticsearch/index/mapper/vectors/DenseVectorFieldMapperTests.java # x-pack/plugin/inference/src/main/java/org/elasticsearch/xpack/inference/mapper/SemanticTextFieldMapper.java # x-pack/plugin/inference/src/test/java/org/elasticsearch/xpack/inference/mapper/SemanticTextFieldMapperTests.java

* fix(semantic_text): index underlying dense_vector field in older indices * Update docs/changelog/133080.yaml * [CI] Auto commit changes from spotless * update msg, change versions, add ut * indent * remove todo * [CI] Auto commit changes from spotless * update msg * [CI] Auto commit changes from spotless * add densevectormapper indexed ut * [CI] Auto commit changes from spotless * fix ut * undo accidental ut removal * [CI] Auto commit changes from spotless * Update docs/changelog/133080.yaml Co-authored-by: Kathleen DeRusso <[email protected]> * remove sparse_vector exception change * [CI] Auto commit changes from spotless * reverting most of the stuff * removing unused import, rename test * [CI] Auto commit changes from spotless * syntax * revisit changelog * revisit changelog * Update docs/changelog/133080.yaml Co-authored-by: Kathleen DeRusso <[email protected]> * Update x-pack/plugin/inference/src/main/java/org/elasticsearch/xpack/inference/mapper/SemanticTextFieldMapper.java Co-authored-by: Kathleen DeRusso <[email protected]> * changelog format * fix UT --------- Co-authored-by: elasticsearchmachine <[email protected]> Co-authored-by: Kathleen DeRusso <[email protected]> (cherry picked from commit 8f41a4b) # Conflicts: # server/src/test/java/org/elasticsearch/index/mapper/vectors/DenseVectorFieldMapperTests.java # x-pack/plugin/inference/src/main/java/org/elasticsearch/xpack/inference/mapper/SemanticTextFieldMapper.java

kderusso · 2025-08-26T19:13:39Z

💚 All backports created successfully

Status	Branch	Result
✅	9.0
✅	8.19
✅	8.18

Questions ?

Please refer to the Backport tool documentation

* fix(semantic_text): index underlying dense_vector field in older indices * Update docs/changelog/133080.yaml * [CI] Auto commit changes from spotless * update msg, change versions, add ut * indent * remove todo * [CI] Auto commit changes from spotless * update msg * [CI] Auto commit changes from spotless * add densevectormapper indexed ut * [CI] Auto commit changes from spotless * fix ut * undo accidental ut removal * [CI] Auto commit changes from spotless * Update docs/changelog/133080.yaml Co-authored-by: Kathleen DeRusso <[email protected]> * remove sparse_vector exception change * [CI] Auto commit changes from spotless * reverting most of the stuff * removing unused import, rename test * [CI] Auto commit changes from spotless * syntax * revisit changelog * revisit changelog * Update docs/changelog/133080.yaml Co-authored-by: Kathleen DeRusso <[email protected]> * Update x-pack/plugin/inference/src/main/java/org/elasticsearch/xpack/inference/mapper/SemanticTextFieldMapper.java Co-authored-by: Kathleen DeRusso <[email protected]> * changelog format * fix UT --------- Co-authored-by: elasticsearchmachine <[email protected]> Co-authored-by: Kathleen DeRusso <[email protected]> (cherry picked from commit 8f41a4b) # Conflicts: # server/src/test/java/org/elasticsearch/index/mapper/vectors/DenseVectorFieldMapperTests.java # x-pack/plugin/inference/src/main/java/org/elasticsearch/xpack/inference/mapper/SemanticTextFieldMapper.java # x-pack/plugin/inference/src/test/java/org/elasticsearch/xpack/inference/mapper/SemanticTextFieldMapperTests.java

…33577) * fix(semantic_text): index underlying dense_vector field in older indices * Update docs/changelog/133080.yaml * [CI] Auto commit changes from spotless * update msg, change versions, add ut * indent * remove todo * [CI] Auto commit changes from spotless * update msg * [CI] Auto commit changes from spotless * add densevectormapper indexed ut * [CI] Auto commit changes from spotless * fix ut * undo accidental ut removal * [CI] Auto commit changes from spotless * Update docs/changelog/133080.yaml * remove sparse_vector exception change * [CI] Auto commit changes from spotless * reverting most of the stuff * removing unused import, rename test * [CI] Auto commit changes from spotless * syntax * revisit changelog * revisit changelog * Update docs/changelog/133080.yaml * Update x-pack/plugin/inference/src/main/java/org/elasticsearch/xpack/inference/mapper/SemanticTextFieldMapper.java * changelog format * fix UT --------- Co-authored-by: elasticsearchmachine <[email protected]> Co-authored-by: Kathleen DeRusso <[email protected]>

… (#133578) * refactor(semantic_text): fail early in pre-8.11 indices (#133080) * fix(semantic_text): index underlying dense_vector field in older indices * Update docs/changelog/133080.yaml * [CI] Auto commit changes from spotless * update msg, change versions, add ut * indent * remove todo * [CI] Auto commit changes from spotless * update msg * [CI] Auto commit changes from spotless * add densevectormapper indexed ut * [CI] Auto commit changes from spotless * fix ut * undo accidental ut removal * [CI] Auto commit changes from spotless * Update docs/changelog/133080.yaml Co-authored-by: Kathleen DeRusso <[email protected]> * remove sparse_vector exception change * [CI] Auto commit changes from spotless * reverting most of the stuff * removing unused import, rename test * [CI] Auto commit changes from spotless * syntax * revisit changelog * revisit changelog * Update docs/changelog/133080.yaml Co-authored-by: Kathleen DeRusso <[email protected]> * Update x-pack/plugin/inference/src/main/java/org/elasticsearch/xpack/inference/mapper/SemanticTextFieldMapper.java Co-authored-by: Kathleen DeRusso <[email protected]> * changelog format * fix UT --------- Co-authored-by: elasticsearchmachine <[email protected]> Co-authored-by: Kathleen DeRusso <[email protected]> (cherry picked from commit 8f41a4b) # Conflicts: # server/src/test/java/org/elasticsearch/index/mapper/vectors/DenseVectorFieldMapperTests.java # x-pack/plugin/inference/src/main/java/org/elasticsearch/xpack/inference/mapper/SemanticTextFieldMapper.java # x-pack/plugin/inference/src/test/java/org/elasticsearch/xpack/inference/mapper/SemanticTextFieldMapperTests.java * [CI] Auto commit changes from spotless --------- Co-authored-by: Michail Romaios <[email protected]> Co-authored-by: elasticsearchmachine <[email protected]>

…) (#133580) * refactor(semantic_text): fail early in pre-8.11 indices (#133080) * fix(semantic_text): index underlying dense_vector field in older indices * Update docs/changelog/133080.yaml * [CI] Auto commit changes from spotless * update msg, change versions, add ut * indent * remove todo * [CI] Auto commit changes from spotless * update msg * [CI] Auto commit changes from spotless * add densevectormapper indexed ut * [CI] Auto commit changes from spotless * fix ut * undo accidental ut removal * [CI] Auto commit changes from spotless * Update docs/changelog/133080.yaml Co-authored-by: Kathleen DeRusso <[email protected]> * remove sparse_vector exception change * [CI] Auto commit changes from spotless * reverting most of the stuff * removing unused import, rename test * [CI] Auto commit changes from spotless * syntax * revisit changelog * revisit changelog * Update docs/changelog/133080.yaml Co-authored-by: Kathleen DeRusso <[email protected]> * Update x-pack/plugin/inference/src/main/java/org/elasticsearch/xpack/inference/mapper/SemanticTextFieldMapper.java Co-authored-by: Kathleen DeRusso <[email protected]> * changelog format * fix UT --------- Co-authored-by: elasticsearchmachine <[email protected]> Co-authored-by: Kathleen DeRusso <[email protected]> (cherry picked from commit 8f41a4b) # Conflicts: # server/src/test/java/org/elasticsearch/index/mapper/vectors/DenseVectorFieldMapperTests.java # x-pack/plugin/inference/src/main/java/org/elasticsearch/xpack/inference/mapper/SemanticTextFieldMapper.java * Fix test --------- Co-authored-by: Michail Romaios <[email protected]>

…) (#133582) * refactor(semantic_text): fail early in pre-8.11 indices (#133080) * fix(semantic_text): index underlying dense_vector field in older indices * Update docs/changelog/133080.yaml * [CI] Auto commit changes from spotless * update msg, change versions, add ut * indent * remove todo * [CI] Auto commit changes from spotless * update msg * [CI] Auto commit changes from spotless * add densevectormapper indexed ut * [CI] Auto commit changes from spotless * fix ut * undo accidental ut removal * [CI] Auto commit changes from spotless * Update docs/changelog/133080.yaml Co-authored-by: Kathleen DeRusso <[email protected]> * remove sparse_vector exception change * [CI] Auto commit changes from spotless * reverting most of the stuff * removing unused import, rename test * [CI] Auto commit changes from spotless * syntax * revisit changelog * revisit changelog * Update docs/changelog/133080.yaml Co-authored-by: Kathleen DeRusso <[email protected]> * Update x-pack/plugin/inference/src/main/java/org/elasticsearch/xpack/inference/mapper/SemanticTextFieldMapper.java Co-authored-by: Kathleen DeRusso <[email protected]> * changelog format * fix UT --------- Co-authored-by: elasticsearchmachine <[email protected]> Co-authored-by: Kathleen DeRusso <[email protected]> (cherry picked from commit 8f41a4b) # Conflicts: # server/src/test/java/org/elasticsearch/index/mapper/vectors/DenseVectorFieldMapperTests.java # x-pack/plugin/inference/src/main/java/org/elasticsearch/xpack/inference/mapper/SemanticTextFieldMapper.java # x-pack/plugin/inference/src/test/java/org/elasticsearch/xpack/inference/mapper/SemanticTextFieldMapperTests.java * [CI] Auto commit changes from spotless * Fix test --------- Co-authored-by: Michail Romaios <[email protected]> Co-authored-by: elasticsearchmachine <[email protected]>

…ic#133080) (elastic#133580) * refactor(semantic_text): fail early in pre-8.11 indices (elastic#133080) * fix(semantic_text): index underlying dense_vector field in older indices * Update docs/changelog/133080.yaml * [CI] Auto commit changes from spotless * update msg, change versions, add ut * indent * remove todo * [CI] Auto commit changes from spotless * update msg * [CI] Auto commit changes from spotless * add densevectormapper indexed ut * [CI] Auto commit changes from spotless * fix ut * undo accidental ut removal * [CI] Auto commit changes from spotless * Update docs/changelog/133080.yaml Co-authored-by: Kathleen DeRusso <[email protected]> * remove sparse_vector exception change * [CI] Auto commit changes from spotless * reverting most of the stuff * removing unused import, rename test * [CI] Auto commit changes from spotless * syntax * revisit changelog * revisit changelog * Update docs/changelog/133080.yaml Co-authored-by: Kathleen DeRusso <[email protected]> * Update x-pack/plugin/inference/src/main/java/org/elasticsearch/xpack/inference/mapper/SemanticTextFieldMapper.java Co-authored-by: Kathleen DeRusso <[email protected]> * changelog format * fix UT --------- Co-authored-by: elasticsearchmachine <[email protected]> Co-authored-by: Kathleen DeRusso <[email protected]> (cherry picked from commit 8f41a4b) # Conflicts: # server/src/test/java/org/elasticsearch/index/mapper/vectors/DenseVectorFieldMapperTests.java # x-pack/plugin/inference/src/main/java/org/elasticsearch/xpack/inference/mapper/SemanticTextFieldMapper.java * Fix test --------- Co-authored-by: Michail Romaios <[email protected]>

…tic#133080)" This reverts commit 8f41a4b.

fix(semantic_text): index underlying dense_vector field in older indices

4e4fae2

elasticsearchmachine added the v9.2.0 label Aug 18, 2025

mromaios self-assigned this Aug 18, 2025

mromaios added :SearchOrg/Relevance Label for the Search (solution/org) Relevance team >bug labels Aug 18, 2025

mromaios and others added 2 commits August 18, 2025 18:30

Update docs/changelog/133080.yaml

e63a078

[CI] Auto commit changes from spotless

a8fd795

Mikep86 reviewed Aug 18, 2025

View reviewed changes

...nference/src/main/java/org/elasticsearch/xpack/inference/mapper/SemanticTextFieldMapper.java Outdated Show resolved Hide resolved

mromaios and others added 15 commits August 19, 2025 15:23

update msg, change versions, add ut

a6eb387

Merge branch '132551_enable_dense_vector_indexing' of github.com:mrom…

fe8e551

…aios/elasticsearch into 132551_enable_dense_vector_indexing

indent

587726c

remove todo

46d0f47

[CI] Auto commit changes from spotless

db05915

Merge branch '132551_enable_dense_vector_indexing' of github.com:mrom…

13c713c

…aios/elasticsearch into 132551_enable_dense_vector_indexing

update msg

d8b18ac

[CI] Auto commit changes from spotless

42fa220

Merge branch '132551_enable_dense_vector_indexing' of github.com:mrom…

c2cef25

…aios/elasticsearch into 132551_enable_dense_vector_indexing

add densevectormapper indexed ut

d572fd7

[CI] Auto commit changes from spotless

9c48f54

Merge branch '132551_enable_dense_vector_indexing' of github.com:mrom…

ffa865c

…aios/elasticsearch into 132551_enable_dense_vector_indexing

fix ut

7fa6386

undo accidental ut removal

f0ab1e1

[CI] Auto commit changes from spotless

d5219ed

mromaios commented Aug 19, 2025

View reviewed changes

...nference/src/main/java/org/elasticsearch/xpack/inference/mapper/SemanticTextFieldMapper.java Outdated Show resolved Hide resolved

...nference/src/main/java/org/elasticsearch/xpack/inference/mapper/SemanticTextFieldMapper.java Outdated Show resolved Hide resolved

mromaios marked this pull request as ready for review August 19, 2025 14:38

elasticsearchmachine added Team:SearchOrg Meta label for the Search Org (Enterprise Search) Team:Search - Relevance The Search organization Search Relevance team labels Aug 19, 2025

mromaios added 2 commits August 22, 2025 09:29

Merge branch '132551_enable_dense_vector_indexing' of github.com:mrom…

2e262b3

…aios/elasticsearch into 132551_enable_dense_vector_indexing

fix UT

166964f

This was referenced Aug 22, 2025

docs(semantic_text): add index version limitation #133373

Open

[8.19] docs(semantic_text): add index version limitation #133374

Open

kderusso added v9.1.4 v9.0.7 auto-backport Automatically create backport pull requests when merged v8.19.4 v8.18.7 labels Aug 26, 2025

kderusso merged commit 8f41a4b into elastic:main Aug 26, 2025
34 checks passed

mromaios mentioned this pull request Aug 26, 2025

[9.1] refactor(semantic_text): fail early in pre-8.11 indices (#133080) #133577

Merged

elasticsearchmachine added the backport pending label Aug 26, 2025

This was referenced Aug 26, 2025

[9.0] refactor(semantic_text): fail early in pre-8.11 indices (#133080) #133578

Merged

[8.19] refactor(semantic_text): fail early in pre-8.11 indices (#133080) #133580

Merged

kderusso mentioned this pull request Aug 26, 2025

[8.18] refactor(semantic_text): fail early in pre-8.11 indices (#133080) #133582

Merged

Mikep86 removed the backport pending label Oct 2, 2025

Mikep86 added a commit to Mikep86/elasticsearch that referenced this pull request Oct 2, 2025

Revert "refactor(semantic_text): fail early in pre-8.11 indices (elas…

d19ed5d

…tic#133080)" This reverts commit 8f41a4b.

Mikep86 mentioned this pull request Oct 2, 2025

Defer Semantic Text Failures on Pre-8.11 Indices #135845

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

refactor(semantic_text): fail early in pre-8.11 indices #133080

refactor(semantic_text): fail early in pre-8.11 indices #133080

mromaios commented Aug 18, 2025 •

edited

Loading

Uh oh!

elasticsearchmachine commented Aug 18, 2025

Uh oh!

Mikep86 left a comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

elasticsearchmachine commented Aug 19, 2025

Uh oh!

elasticsearchmachine commented Aug 19, 2025

Uh oh!

Uh oh!

elasticsearchmachine commented Aug 26, 2025

Uh oh!

kderusso commented Aug 26, 2025

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

refactor(semantic_text): fail early in pre-8.11 indices #133080

refactor(semantic_text): fail early in pre-8.11 indices #133080

Conversation

mromaios commented Aug 18, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

elasticsearchmachine commented Aug 18, 2025

Uh oh!

Mikep86 left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

elasticsearchmachine commented Aug 19, 2025

Uh oh!

elasticsearchmachine commented Aug 19, 2025

Uh oh!

Uh oh!

elasticsearchmachine commented Aug 26, 2025

💔 Backport failed

Uh oh!

kderusso commented Aug 26, 2025

💚 All backports created successfully

Questions ?

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

mromaios commented Aug 18, 2025 •

edited

Loading