GrizzlyResponse object reachable from SelectorRunner thread local

[elrodro83]

Hi,

We are using grizzly through AHC to perform some HTTP requests. After those requests have finished, in a thread dump i can still see the response objects being referenced, and not being garbage collected. Attached is a screenshot of there references from a heap dump:

In a case where the size of the transferred data is big, i see some heavy memory usage.

The problem seems to be with the reference chain TCPNIOConnection -> IndexedAttributeHolder -> $Snapshot -> HttpTransactionContext. Perhaps at least one of those references should be discarded after the request has completed, or provide a way to an app to clear that.

The following test can reproduce this issue.

    @Test
    public void referencedResponseHC() throws InterruptedException
    {
        AsyncHttpClientConfig config = new AsyncHttpClientConfig.Builder().build();
        AsyncHttpClient client = new AsyncHttpClient(new GrizzlyAsyncHttpProvider(config), config);

        final CountDownLatch responseLatch = new CountDownLatch(1);
        final AtomicReference<Response> responseRef = new AtomicReference<>();

        client.prepareGet("http://www.ning.com/").execute(new AsyncCompletionHandler<Response>()
        {

            @Override
            public Response onCompleted(Response response) throws Exception
            {
                responseLatch.countDown();
                responseRef.set(response);
                return response;
            }

            @Override
            public void onThrowable(Throwable t)
            {
                // Something wrong happened.
            }
        });

        responseLatch.await(5, TimeUnit.SECONDS);
        verifyNotLeaked(new PhantomReference<>(responseRef.getAndSet(null), new ReferenceQueue<>()));
    }

    private void verifyNotLeaked(PhantomReference possibleLeakPhantomRef) throws InterruptedException
    {
        for (int i = 0; i < 10; ++i)
        {
            System.gc();
            Thread.sleep(100);
            if (possibleLeakPhantomRef.isEnqueued())
            {
                break;
            }
        }
        assertTrue(possibleLeakPhantomRef.isEnqueued());
    }

Thanks in advance,

[elrodro83]

Just in case, this situation doesn't happen when using grizzly alone (without AHC)

[slandelle]

Just being curious, can you reproduce such leak with the Netty provider?

[elrodro83]

Hi,

Switching to the netty provider there is no reference to the response, the given test is ok in that case.

[slandelle]

Thanks for your feedback.
Assigning to Alexei, there isn't much I can do on the Grizzly side.

[oleksiys]

@elrodro83 thank you! I'll take a look asap

[oleksiys]

pls. check if the fix works for you.

[elrodro83]

Worked like a charm!

[slandelle]

This fix was shipped with 1.9.32 that's already out.