Skip to content

Commit 0469a3c

Browse files
tsallasexhaggi
authored andcommitted
DATAES-445 - Updated scroll API example.
Original pull request: spring-projects#218
1 parent 89d0633 commit 0469a3c

File tree

1 file changed

+42
-37
lines changed

1 file changed

+42
-37
lines changed

src/main/asciidoc/reference/elasticsearch-misc.adoc

Lines changed: 42 additions & 37 deletions
Original file line numberDiff line numberDiff line change
@@ -23,50 +23,55 @@ Page<SampleEntity> sampleEntities =
2323
----
2424
====
2525

26-
[[elasticsearch.scan.and.scroll]]
27-
== Using Scan And Scroll For Big Result Set
26+
[[elasticsearch.scroll]]
27+
== Using Scroll For Big Result Set
2828

29-
Elasticsearch has scan and scroll feature for getting big result set in chunks. `ElasticsearchTemplate` has scan and scroll methods that can be used as below.
29+
Elasticsearch has a scroll API for getting big result set in chunks. `ElasticsearchTemplate` has startScroll and continueScroll methods that can be used as below.
3030

31-
.Using Scan and Scroll
31+
.Using startScroll and continueScroll
3232
====
3333
[source,java]
3434
----
3535
SearchQuery searchQuery = new NativeSearchQueryBuilder()
3636
.withQuery(matchAllQuery())
37-
.withIndices("test-index")
38-
.withTypes("test-type")
39-
.withPageable(new PageRequest(0,1))
37+
.withIndices(INDEX_NAME)
38+
.withTypes(TYPE_NAME)
39+
.withFields("message")
40+
.withPageable(PageRequest.of(0, 10))
4041
.build();
41-
String scrollId = elasticsearchTemplate.scan(searchQuery,1000,false);
42-
List<SampleEntity> sampleEntities = new ArrayList<SampleEntity>();
43-
boolean hasRecords = true;
44-
while (hasRecords){
45-
Page<SampleEntity> page = elasticsearchTemplate.scroll(scrollId, 5000L , new ResultsMapper<SampleEntity>()
46-
{
47-
@Override
48-
public Page<SampleEntity> mapResults(SearchResponse response) {
49-
List<SampleEntity> chunk = new ArrayList<SampleEntity>();
50-
for(SearchHit searchHit : response.getHits()){
51-
if(response.getHits().getHits().length <= 0) {
52-
return null;
53-
}
54-
SampleEntity user = new SampleEntity();
55-
user.setId(searchHit.getId());
56-
user.setMessage((String)searchHit.getSource().get("message"));
57-
chunk.add(user);
58-
}
59-
return new PageImpl<SampleEntity>(chunk);
60-
}
61-
});
62-
if(page != null) {
63-
sampleEntities.addAll(page.getContent());
64-
hasRecords = page.hasNextPage();
65-
}
66-
else{
67-
hasRecords = false;
68-
}
69-
}
70-
}
42+
43+
Page<SampleEntity> scroll = elasticsearchTemplate.startScroll(1000, searchQuery, SampleEntity.class);
44+
45+
String scrollId = ((ScrolledPage) scroll).getScrollId();
46+
List<SampleEntity> sampleEntities = new ArrayList<>();
47+
while (scroll.hasContent()) {
48+
sampleEntities.addAll(scroll.getContent());
49+
scrollId = ((ScrolledPage) scroll).getScrollId();
50+
scroll = elasticsearchTemplate.continueScroll(scrollId, 1000, SampleEntity.class);
51+
}
52+
elasticsearchTemplate.clearScroll(scrollId);
7153
----
7254
====
55+
56+
`ElasticsearchTemplate` additionally has the stream method which wraps the scan and scroll operations into a CloseableIterator.
57+
58+
.Using stream
59+
====
60+
[source,java]
61+
----
62+
SearchQuery searchQuery = new NativeSearchQueryBuilder()
63+
.withQuery(matchAllQuery())
64+
.withIndices(INDEX_NAME)
65+
.withTypes(TYPE_NAME)
66+
.withFields("message")
67+
.withPageable(PageRequest.of(0, 10))
68+
.build();
69+
70+
CloseableIterator<SampleEntity> stream = elasticsearchTemplate.stream(searchQuery, SampleEntity.class);
71+
72+
List<SampleEntity> sampleEntities = new ArrayList<>();
73+
while (stream.hasNext()) {
74+
sampleEntities.add(stream.next());
75+
}
76+
----
77+
====

0 commit comments

Comments
 (0)