Skip to content

Support request compression #93

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
cstamas opened this issue Apr 25, 2012 · 10 comments
Closed

Support request compression #93

cstamas opened this issue Apr 25, 2012 · 10 comments

Comments

@cstamas
Copy link
Contributor

cstamas commented Apr 25, 2012

The following snippet fails to execute with AHC 1.7.3 and 1.7.4 (using default Netty provider):

  final AsyncHttpClientConfig.Builder configBuilder = new AsyncHttpClientConfig.Builder();
  configBuilder.setCompressionEnabled( true );
  configBuilder.setRequestCompressionLevel( 6 );
  final AsyncHttpClient client = new AsyncHttpClient( configBuilder.build() );

  try
  {
    final Response response = client.prepareGet( "http://www.google.com/" ).execute().get();
    System.out.println( response.getStatusCode() + " " + response.getStatusText() );
  }
  finally
  {
    client.close();
  }

If I comment out the line configBuilder.setRequestCompressionLevel( 6 ); all is fine and snippet above works.

Also, I'd like to have some explanation what this method is meant to do. By reading sources, it is used to set "compression level" on Netty's HttpContentEncoder (wanted to raise it from default 6 to 9). How are setRequestCompressionLevel() and setCompressionEnabled() corelated?

Exception thrown by snippet above is:

java.util.concurrent.ExecutionException:
java.lang.IllegalStateException: cannot send more responses than requests
       at com.ning.http.client.providers.netty.NettyResponseFuture.abort(NettyResponseFuture.java:297)
       at com.ning.http.client.providers.netty.NettyAsyncHttpProvider.abort(NettyAsyncHttpProvider.java:1320)
       at com.ning.http.client.providers.netty.NettyAsyncHttpProvider.exceptionCaught(NettyAsyncHttpProvider.java:1545)
       at org.jboss.netty.channel.SimpleChannelUpstreamHandler.handleUpstream(SimpleChannelUpstreamHandler.java:117)
       at org.jboss.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:558)
       at org.jboss.netty.channel.DefaultChannelPipeline$DefaultChannelHandlerContext.sendUpstream(DefaultChannelPipeline.java:777)
       at org.jboss.netty.handler.stream.ChunkedWriteHandler.handleUpstream(ChunkedWriteHandler.java:143)
       at org.jboss.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:558)
       at org.jboss.netty.channel.DefaultChannelPipeline$DefaultChannelHandlerContext.sendUpstream(DefaultChannelPipeline.java:777)
       at org.jboss.netty.channel.SimpleChannelUpstreamHandler.exceptionCaught(SimpleChannelUpstreamHandler.java:143)
       at org.jboss.netty.channel.SimpleChannelUpstreamHandler.handleUpstream(SimpleChannelUpstreamHandler.java:117)
       at org.jboss.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:558)
       at org.jboss.netty.channel.DefaultChannelPipeline$DefaultChannelHandlerContext.sendUpstream(DefaultChannelPipeline.java:777)
       at org.jboss.netty.channel.SimpleChannelHandler.exceptionCaught(SimpleChannelHandler.java:163)
       at org.jboss.netty.channel.SimpleChannelHandler.handleUpstream(SimpleChannelHandler.java:137)
       at org.jboss.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:558)
       at org.jboss.netty.channel.DefaultChannelPipeline$DefaultChannelHandlerContext.sendUpstream(DefaultChannelPipeline.java:777)
       at org.jboss.netty.handler.codec.replay.ReplayingDecoder.exceptionCaught(ReplayingDecoder.java:456)
       at org.jboss.netty.channel.SimpleChannelUpstreamHandler.handleUpstream(SimpleChannelUpstreamHandler.java:117)
       at org.jboss.netty.handler.codec.http.HttpClientCodec.handleUpstream(HttpClientCodec.java:72)
       at org.jboss.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:558)
       at org.jboss.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:553)
       at org.jboss.netty.channel.Channels.fireExceptionCaught(Channels.java:426)
       at org.jboss.netty.channel.AbstractChannelSink.exceptionCaught(AbstractChannelSink.java:47)
       at org.jboss.netty.channel.DefaultChannelPipeline.notifyHandlerException(DefaultChannelPipeline.java:645)
       at org.jboss.netty.channel.DefaultChannelPipeline.sendDownstream(DefaultChannelPipeline.java:593)
       at org.jboss.netty.channel.DefaultChannelPipeline$DefaultChannelHandlerContext.sendDownstream(DefaultChannelPipeline.java:770)
       at org.jboss.netty.handler.stream.ChunkedWriteHandler.flush(ChunkedWriteHandler.java:262)
       at org.jboss.netty.handler.stream.ChunkedWriteHandler.handleDownstream(ChunkedWriteHandler.java:119)
       at org.jboss.netty.channel.DefaultChannelPipeline.sendDownstream(DefaultChannelPipeline.java:585)
       at org.jboss.netty.channel.DefaultChannelPipeline.sendDownstream(DefaultChannelPipeline.java:576)
       at org.jboss.netty.channel.Channels.write(Channels.java:605)
       at org.jboss.netty.channel.Channels.write(Channels.java:572)
       at org.jboss.netty.channel.AbstractChannel.write(AbstractChannel.java:245)
       at com.ning.http.client.providers.netty.NettyAsyncHttpProvider.writeRequest(NettyAsyncHttpProvider.java:431)
       at com.ning.http.client.providers.netty.NettyConnectListener.operationComplete(NettyConnectListener.java:82)
       at org.jboss.netty.channel.DefaultChannelFuture.notifyListener(DefaultChannelFuture.java:397)
       at org.jboss.netty.channel.DefaultChannelFuture.notifyListeners(DefaultChannelFuture.java:388)
       at org.jboss.netty.channel.DefaultChannelFuture.setSuccess(DefaultChannelFuture.java:332)
       at org.jboss.netty.channel.socket.nio.NioWorker$RegisterTask.run(NioWorker.java:770)
       at org.jboss.netty.channel.socket.nio.NioWorker.processRegisterTaskQueue(NioWorker.java:250)
       at org.jboss.netty.channel.socket.nio.NioWorker.run(NioWorker.java:192)
       at org.jboss.netty.util.ThreadRenamingRunnable.run(ThreadRenamingRunnable.java:102)
       at org.jboss.netty.util.internal.DeadLockProofWorker$1.run(DeadLockProofWorker.java:42)
       at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
       at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
       at java.lang.Thread.run(Thread.java:680)
Caused by: java.lang.IllegalStateException: cannot send more responses than requests
       at org.jboss.netty.handler.codec.http.HttpContentEncoder.writeRequested(HttpContentEncoder.java:104)
       at org.jboss.netty.handler.stream.ChunkedWriteHandler.flush(ChunkedWriteHandler.java:262)
       at org.jboss.netty.handler.stream.ChunkedWriteHandler.handleDownstream(ChunkedWriteHandler.java:119)
       at org.jboss.netty.channel.Channels.write(Channels.java:605)
       at org.jboss.netty.channel.Channels.write(Channels.java:572)
       at org.jboss.netty.channel.AbstractChannel.write(AbstractChannel.java:245)
       at com.ning.http.client.providers.netty.NettyAsyncHttpProvider.writeRequest(NettyAsyncHttpProvider.java:431)
       at com.ning.http.client.providers.netty.NettyConnectListener.operationComplete(NettyConnectListener.java:82)
       at org.jboss.netty.channel.DefaultChannelFuture.notifyListener(DefaultChannelFuture.java:397)
       at org.jboss.netty.channel.DefaultChannelFuture.notifyListeners(DefaultChannelFuture.java:388)
       at org.jboss.netty.channel.DefaultChannelFuture.setSuccess(DefaultChannelFuture.java:332)
       at org.jboss.netty.channel.socket.nio.NioWorker$RegisterTask.run(NioWorker.java:770)
       at org.jboss.netty.channel.socket.nio.NioWorker.processRegisterTaskQueue(NioWorker.java:250)
       at org.jboss.netty.channel.socket.nio.NioWorker.run(NioWorker.java:192)
       ... 3 more

@slandelle
Copy link
Contributor

The error message is actually wrong, Netty 3 doesn't properly handle BAD_REQUEST status code: netty/netty#441

@cstamas I know this is an old issue, but would you have a simple test case (server side) so I can investigate, please?

@slandelle
Copy link
Contributor

I have a Gatling user who kindly provided a test case: https://docs.google.com/file/d/0B1zTVlhQrNiyZ01CZDFDWm5QYlk/edit

Installation instructions:

Send POST requests to localhost:3000/post/test, XML bodies, compressed on not, will be printed in the console.

RequestBuilder requestBuilder = new RequestBuilder("POST");
requestBuilder.setUrl("http://localhost:3001/post/test");
requestBuilder.setBody("<foo>bar</foo>".getBytes());
Request request = requestBuilder.build();

AsyncHttpClientConfig config = new AsyncHttpClientConfig.Builder().setRequestCompressionLevel(6).build();
AsyncHttpClient client = new AsyncHttpClient(config);
// AsyncHttpClient client = new AsyncHttpClient(); // works perfectly when no request compression

try {
    client.executeRequest(request).get();
} finally {
    client.close();
}

Note that IllegalStateException: cannot send more responses than requests is actually Netty not properly handling BAD_REQUEST status code.

@slandelle
Copy link
Contributor

NettyAsyncHttpProvider wrongly uses a HttpContentCompressor, which is a HttpContentEncoder, that, as doc states, is only used for encoding responses, not requests!

Actually, Netty doesn't support request compression yet. See netty/netty#2131 and netty/netty#2132

@slandelle
Copy link
Contributor

@jfarcand As this is a Netty feature request, I don't know when it will be implemented and I doubt it will be backported on Netty 3.

I say that we remove the feature for now, both in AHC1 and AHC2. This won't cause regression is AHC1 because no user could possibly use it: it just makes the request break.

@slandelle
Copy link
Contributor

Change title

@jfarcand
Copy link
Contributor

Agree. Closing as won't fix.

@slandelle
Copy link
Contributor

@jfarcand I disagree. I've been asked for this feature on Gatling and had to implement it on my own, and it's not great and I'd rather have it supported in Netty.

Let's just turn what first was an issue into a feature request (I did one on Netty).

@slandelle slandelle reopened this Jan 30, 2014
@slandelle slandelle changed the title Netty provider doesn't support request compression Support request compression Jul 17, 2014
@slandelle slandelle removed this from the 2.0.0.Alpha1 milestone Oct 28, 2015
@slandelle slandelle removed the Grizzly label Oct 28, 2015
@slandelle slandelle removed the Netty label Oct 12, 2016
@slandelle slandelle removed their assignment Oct 12, 2016
@slandelle
Copy link
Contributor

No activity there for a very long time. Closing for now.

@Diagoras
Copy link
Contributor

@slandelle I know you'd prefer to see this implemented in Netty, but what do you think about adding this to AHC until Netty finally supports request compression?

I think it'd be doable using just a RequestFilter, if that helps. And if you're leery about polluting the config class with more options we could just include a GzipRequestFilter in AHC and let people manually add it to the request filters in the config themselves.

If you think it's worth considering, I can open a PR. Let me know.

@slandelle
Copy link
Contributor

I don't think that would work with every kinds of response bodies.
I think the only way to implement it is to have a Netty handler, similar to the one use for compressing server side.

cs-workco pushed a commit to cs-workco/async-http-client that referenced this issue Apr 13, 2023
)

* nest timeout configuration type inside configuration

* move timeout to an extension
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

4 participants