Skip to content

Rework FeedableBodyGerator #949

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 1 commit into from
Aug 24, 2015
Merged

Conversation

bomgar
Copy link
Contributor

@bomgar bomgar commented Aug 16, 2015

the read method became really big.

@slandelle
Copy link
Contributor

I think there's a bug in BodyChunkedInput because of the way the ByteBuffer is computed.
If body.contentLength is 10 or less, we end up suspending or finishing because this code in readBodyPart:

int capacity = buffer.remaining() - 10; // be safe (we'll have to add size, ending, etc.)
int size = Math.min(nextPart.buffer.remaining(), capacity);

WDYT?

@bomgar
Copy link
Contributor Author

bomgar commented Aug 17, 2015

At the moment the whole thing relies on the outgoing buffer being big enough to hold the boundaries and at least one byte. The "closing" part has the same problem.

@bomgar bomgar force-pushed the master branch 3 times, most recently from e2bae09 to 4bf4fb5 Compare August 22, 2015 10:26
@bomgar
Copy link
Contributor Author

bomgar commented Aug 22, 2015

I rewrote parts of it. What do you think?

@bomgar bomgar changed the title Cleanup FeedableBodyGerator Rework FeedableBodyGerator Aug 23, 2015
queue.remove();
} else {
readBodyPart(buffer, nextPart);
reads++;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks wrong to me: read must return the number of bytes read, while this seems to be returning the number of parts read (and still assuming buffer is large enough).

@slandelle
Copy link
Contributor

I've added some comments.
Then, I'm still concerned with buffer overflow (I didn't implement this code) that would happen if a huge pile of parts arrive at the same time.

private final static class BodyPart {

private void move(ByteBuffer destination, ByteBuffer source) {
while(destination.hasRemaining() && source.hasRemaining()) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why not put a whole slice?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done in af0df14

@bomgar
Copy link
Contributor Author

bomgar commented Aug 24, 2015

It doesn't have to return the exact amount of bytes. A positive number is enough. The amount is represented in the ByteBuffer.
To solve the overflow problem one would have to implement backpressure. That would be great to have but this should not be part of this refactoring.

@bomgar
Copy link
Contributor Author

bomgar commented Aug 24, 2015

The code also does not assume the outgoing buffer is large enough. It stops writing as soon as it is full and continues with the next buffer at the exact same position. That's why i initialize the buffers in the BodyPart.

Also the loop is fine. The copy method from one buffer to another does exactly the same except it checks the destination buffer is large enough (not what i want).

- don't depend on outgoing buffer size
- fill the outgoing buffer with all available data

the read method became really big.
@slandelle
Copy link
Contributor

It doesn't have to return the exact amount of bytes. A positive number is enough.

Oh, right. Then, as AHC2 is still in alpha, I think we should change Body.read signature to return a State enum: CONTINUE, SUSPEND, FINISHED. WDYT?

@slandelle
Copy link
Contributor

To solve the overflow problem one would have to implement backpressure. That would be great to have but this should not be part of this refactoring.

Ultimately, yes (ping @jroper).

@bomgar
Copy link
Contributor Author

bomgar commented Aug 24, 2015

Oh, right. Then, as AHC2 is still in alpha, I think we should change Body.read signature to return a State enum: CONTINUE, SUSPEND, FINISHED. WDYT?

Sounds good to me.

slandelle added a commit that referenced this pull request Aug 24, 2015
Rework FeedableBodyGerator
@slandelle slandelle merged commit 65b742b into AsyncHttpClient:master Aug 24, 2015
@slandelle slandelle added this to the 2.0.0 milestone Aug 24, 2015
@slandelle
Copy link
Contributor

Merging, I'll take care of some clean up. Thanks!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants