Skip to content

CookieStore slows down with increasing number of cookies #1580

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Closed
joshovi opened this issue Sep 28, 2018 · 3 comments · Fixed by #1708
Closed

CookieStore slows down with increasing number of cookies #1580

joshovi opened this issue Sep 28, 2018 · 3 comments · Fixed by #1708
Labels
Contributions Welcome!

Comments

@joshovi
Copy link

joshovi commented Sep 28, 2018

In

async-http-client/client/src/main/java/org/asynchttpclient/cookie/ThreadSafeCookieStore.java

Line 160 in 365c1e6

List<Cookie> result = cookieJar.entrySet().stream().filter(pair -> {
the look up over cookies traverses all entries in the map. The side effect to clean all expired cookies traverses the list again. The cost for algorithm scales with the number of cookies.

In our project we had to disable the cookie store, i.e.
new DefaultAsyncHttpClientConfig.Builder() .setCookieStore(null) ... .build()

Maybe you could rewrite the function using another data structure that facilitates the look up for candidate cookies and then make a lazy removal of the expired candidates, or remove expired cookies asynchronously and periodically.
@slandelle
Copy link
Contributor

Contributions welcome

@sivaalli
Copy link
Contributor

sivaalli commented Mar 8, 2020

@slandelle Can I work on a fix for this?

@slandelle
Copy link
Contributor

@sivaalli sure, contributions welcome

@sivaalli sivaalli mentioned this issue Mar 19, 2020
Merged
slandelle pushed a commit that referenced this issue Apr 3, 2020
@sivaalli
Cookiejar optimization, close #1580 (#1708)
d4f1e58 
Changes:

Segmented map based on domain names. So instead of traversing all the domains it traverses the domains that are of interest.
Use NettyTimer to clean up the expired cookies asynchronously. The timer task that provides this functionality is CookieEvictionTask.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
Contributions Welcome!
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

Cookiejar optimization sivaalli/async-http-client
3 participants
@slandelle @joshovi @sivaalli