Skip to content

Rewrite with compression support #163

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 56 commits into from
Feb 16, 2020
Merged
Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
56 commits
Select commit Hold shift + click to select a range
8604dee
Increase TestWASM timeout
nhooyr Oct 15, 2019
e55ac18
Document compression API
nhooyr Oct 14, 2019
e142e08
Improve compression docs
nhooyr Nov 12, 2019
53c1aea
Implement compression extension negotiation
nhooyr Nov 12, 2019
2cf6c28
Implement compression writer and reader pooling
nhooyr Nov 12, 2019
a01afea
Support x-webkit-deflate-frame extension for Safari
nhooyr Nov 12, 2019
531d4fa
Improve general compression API and write docs
nhooyr Nov 12, 2019
d0a8049
Rewrite core
nhooyr Nov 19, 2019
dd107dd
Update CI
nhooyr Nov 28, 2019
6c6b8e9
Cleanup wspb and wsjson
nhooyr Nov 28, 2019
6b782a3
Run make fmt
nhooyr Nov 28, 2019
989ba2f
Change websocket to WebSocket in docs/errors
nhooyr Nov 28, 2019
9f15963
Simplify dial.go
nhooyr Nov 28, 2019
120911b
Remove use of math/rand.Init
nhooyr Nov 28, 2019
7ad1514
Update README.md comparison
nhooyr Nov 29, 2019
746140b
Further improve README
nhooyr Nov 29, 2019
43cb01e
Refactor read.go/write.go
nhooyr Nov 29, 2019
e8dfe27
Make CI pass
nhooyr Nov 29, 2019
f6137f3
Add minor improvements
nhooyr Dec 6, 2019
6f6fa43
Refactor autobahn
nhooyr Dec 31, 2019
8c87970
Add slidingWindowReader
nhooyr Jan 4, 2020
aaf4b45
Up test coverage of accept.go to 100%
nhooyr Jan 26, 2020
6b76536
Up dial coverage to 100%
nhooyr Jan 30, 2020
0f115ed
Add Go 1.12 support
nhooyr Jan 31, 2020
b6b56b7
Both modes seem to work :)
nhooyr Feb 5, 2020
9e32354
Fix randString method in tests
nhooyr Feb 6, 2020
78da35e
Get test with multiple messages working
nhooyr Feb 7, 2020
d092686
Autobahn tests fully pass :)
nhooyr Feb 8, 2020
6975801
Fix race in tests
nhooyr Feb 9, 2020
bbaf469
Fix test step
nhooyr Feb 9, 2020
faadcc9
Simplify tests
nhooyr Feb 9, 2020
3f2589f
Remove quite a bit of slog
nhooyr Feb 9, 2020
b53f306
Get Wasm tests working
nhooyr Feb 9, 2020
69ff675
More tests and fixes
nhooyr Feb 9, 2020
085e671
Get coverage to 85%
nhooyr Feb 9, 2020
51769b3
Add wspb test
nhooyr Feb 9, 2020
670be05
Merge in handshake improvements from master
nhooyr Feb 9, 2020
988b8f2
Merge remote-tracking branch 'origin/master' into compress
nhooyr Feb 9, 2020
3a526d8
Fix bug in closeHandshake
nhooyr Feb 9, 2020
999b812
Fix race in msgReader
nhooyr Feb 9, 2020
4b84d25
Fix a race with c.closed
nhooyr Feb 9, 2020
85f249d
Up timeouts
nhooyr Feb 9, 2020
6b38ebb
Test fixes
nhooyr Feb 9, 2020
6770421
Fix goroutine leak from deadlock when closing
nhooyr Feb 12, 2020
c752365
Make flateThreshold work
nhooyr Feb 12, 2020
0ea9466
Cleanup writeMu and flateThreshold
nhooyr Feb 12, 2020
b33d48c
Minor cleanup
nhooyr Feb 12, 2020
9c5bfab
Simplifications of conn_test.go
nhooyr Feb 13, 2020
3673c2c
Use basic test assertions
nhooyr Feb 13, 2020
c5b0a00
Fix badPing test duration
nhooyr Feb 13, 2020
1c7c14e
Pool sliding windows
nhooyr Feb 13, 2020
503b469
Simplify sliding window API
nhooyr Feb 13, 2020
dff4af3
Add conn benchmark
nhooyr Feb 13, 2020
2377cca
Switch to klauspost/compress
nhooyr Feb 15, 2020
d57b253
Report how efficient compression is in BenchmarkConn
nhooyr Feb 16, 2020
1bc100d
Update docs and random little issues
nhooyr Feb 16, 2020
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
Add minor improvements
Closes #179
  • Loading branch information
nhooyr committed Dec 19, 2019
commit f6137f3f404630d19a84bc0fd59570ff6a967004
39 changes: 18 additions & 21 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
# websocket

[![version](https://img.shields.io/github/v/release/nhooyr/websocket?color=6b9ded&sort=semver)](https://github.com/nhooyr/websocket/releases)
[![docs](https://godoc.org/nhooyr.io/websocket?status.svg)](https://godoc.org/nhooyr.io/websocket)
[![release](https://img.shields.io/github/v/release/nhooyr/websocket?color=6b9ded&sort=semver)](https://github.com/nhooyr/websocket/releases)
[![godoc](https://godoc.org/nhooyr.io/websocket?status.svg)](https://godoc.org/nhooyr.io/websocket)
[![coverage](https://img.shields.io/coveralls/github/nhooyr/websocket?color=65d6a4)](https://coveralls.io/github/nhooyr/websocket)
[![ci](https://github.com/nhooyr/websocket/workflows/ci/badge.svg)](https://github.com/nhooyr/websocket/actions)

Expand All @@ -19,14 +19,14 @@ go get nhooyr.io/websocket
- First class [context.Context](https://blog.golang.org/context) support
- Thorough tests, fully passes the [autobahn-testsuite](https://github.com/crossbario/autobahn-testsuite)
- [Zero dependencies](https://godoc.org/nhooyr.io/websocket?imports)
- JSON and ProtoBuf helpers in the [wsjson](https://godoc.org/nhooyr.io/websocket/wsjson) and [wspb](https://godoc.org/nhooyr.io/websocket/wspb) subpackages
- JSON and protobuf helpers in the [wsjson](https://godoc.org/nhooyr.io/websocket/wsjson) and [wspb](https://godoc.org/nhooyr.io/websocket/wspb) subpackages
- Zero alloc reads and writes
- Concurrent writes
- [Close handshake](https://godoc.org/nhooyr.io/websocket#Conn.Close)
- [net.Conn](https://godoc.org/nhooyr.io/websocket#NetConn) wrapper
- [Ping pong](https://godoc.org/nhooyr.io/websocket#Conn.Ping)
- [Ping pong](https://godoc.org/nhooyr.io/websocket#Conn.Ping) API
- [RFC 7692](https://tools.ietf.org/html/rfc7692) permessage-deflate compression
- Compile to [Wasm](https://godoc.org/nhooyr.io/websocket#hdr-Wasm)
- Can target [Wasm](https://godoc.org/nhooyr.io/websocket#hdr-Wasm)

## Roadmap

Expand Down Expand Up @@ -85,7 +85,11 @@ c.Close(websocket.StatusNormalClosure, "")

### gorilla/websocket

[gorilla/websocket](https://github.com/gorilla/websocket) is a widely used and mature library.
Advantages of [gorilla/websocket](https://github.com/gorilla/websocket):

- Mature and widely used
- [Prepared writes](https://godoc.org/github.com/gorilla/websocket#PreparedMessage)
- Configurable [buffer sizes](https://godoc.org/github.com/gorilla/websocket#hdr-Buffers)

Advantages of nhooyr.io/websocket:

Expand All @@ -94,26 +98,26 @@ Advantages of nhooyr.io/websocket:
- [net.Conn](https://godoc.org/nhooyr.io/websocket#NetConn) wrapper
- Zero alloc reads and writes ([gorilla/websocket#535](https://github.com/gorilla/websocket/issues/535))
- Full [context.Context](https://blog.golang.org/context) support
- Uses [net/http.Client](https://golang.org/pkg/net/http/#Client) for dialing
- Dial uses [net/http.Client](https://golang.org/pkg/net/http/#Client)
- Will enable easy HTTP/2 support in the future
- Gorilla writes directly to a net.Conn and so duplicates features from net/http.Client.
- Gorilla writes directly to a net.Conn and so duplicates features of net/http.Client.
- Concurrent writes
- Close handshake ([gorilla/websocket#448](https://github.com/gorilla/websocket/issues/448))
- Idiomatic [ping](https://godoc.org/nhooyr.io/websocket#Conn.Ping) API
- gorilla/websocket requires registering a pong callback and then sending a Ping
- Wasm ([gorilla/websocket#432](https://github.com/gorilla/websocket/issues/432))
- Idiomatic [ping pong](https://godoc.org/nhooyr.io/websocket#Conn.Ping) API
- Gorilla requires registering a pong callback before sending a Ping
- Can target Wasm ([gorilla/websocket#432](https://github.com/gorilla/websocket/issues/432))
- Transparent message buffer reuse with [wsjson](https://godoc.org/nhooyr.io/websocket/wsjson) and [wspb](https://godoc.org/nhooyr.io/websocket/wspb) subpackages
- [1.75x](https://github.com/nhooyr/websocket/releases/tag/v1.7.4) faster WebSocket masking implementation in pure Go
- Gorilla's implementation depends on unsafe and is slower
- Gorilla's implementation is slower and uses [unsafe](https://golang.org/pkg/unsafe/).
- Full [permessage-deflate](https://tools.ietf.org/html/rfc7692) compression extension support
- Gorilla only supports no context takeover mode
- [CloseRead](https://godoc.org/nhooyr.io/websocket#Conn.CloseRead) helper
- [CloseRead](https://godoc.org/nhooyr.io/websocket#Conn.CloseRead) helper ([gorilla/websocket#492](https://github.com/gorilla/websocket/issues/492))
- Actively maintained ([gorilla/websocket#370](https://github.com/gorilla/websocket/issues/370))

#### golang.org/x/net/websocket

[golang.org/x/net/websocket](https://godoc.org/golang.org/x/net/websocket) is deprecated.
See ([golang/go/issues/18152](https://github.com/golang/go/issues/18152)).
See [golang/go/issues/18152](https://github.com/golang/go/issues/18152).

The [net.Conn](https://godoc.org/nhooyr.io/websocket#NetConn) wrapper will ease in transitioning
to nhooyr.io/websocket.
Expand All @@ -124,10 +128,3 @@ to nhooyr.io/websocket.
in an event driven style for performance. See the author's [blog post](https://medium.freecodecamp.org/million-websockets-and-go-cc58418460bb).

However when writing idiomatic Go, nhooyr.io/websocket will be faster and easier to use.

## Users

If your company or project is using this library, feel free to open an issue or PR to amend this list.

- [Coder](https://github.com/cdr)
- [Tatsu Works](https://github.com/tatsuworks) - Ingresses 20 TB in WebSocket data every month on their Discord bot.
20 changes: 12 additions & 8 deletions accept_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,7 @@ import (
"strings"
"testing"

"nhooyr.io/websocket/internal/assert"
"cdr.dev/slog/sloggers/slogtest/assert"
)

func TestAccept(t *testing.T) {
Expand All @@ -20,7 +20,7 @@ func TestAccept(t *testing.T) {
r := httptest.NewRequest("GET", "/", nil)

_, err := Accept(w, r, nil)
assert.ErrorContains(t, err, "protocol violation")
assert.ErrorContains(t, "Accept", err, "protocol violation")
})

t.Run("requireHttpHijacker", func(t *testing.T) {
Expand All @@ -34,7 +34,7 @@ func TestAccept(t *testing.T) {
r.Header.Set("Sec-WebSocket-Key", "meow123")

_, err := Accept(w, r, nil)
assert.ErrorContains(t, err, "http.ResponseWriter does not implement http.Hijacker")
assert.ErrorContains(t, "Accept", err, "http.ResponseWriter does not implement http.Hijacker")
})
}

Expand Down Expand Up @@ -127,9 +127,9 @@ func Test_verifyClientHandshake(t *testing.T) {

err := verifyClientRequest(r)
if tc.success {
assert.Success(t, err)
assert.Success(t, "verifyClientRequest", err)
} else {
assert.Error(t, err)
assert.Error(t, "verifyClientRequest", err)
}
})
}
Expand Down Expand Up @@ -179,7 +179,7 @@ func Test_selectSubprotocol(t *testing.T) {
r.Header.Set("Sec-WebSocket-Protocol", strings.Join(tc.clientProtocols, ","))

negotiated := selectSubprotocol(r, tc.serverProtocols)
assert.Equal(t, tc.negotiated, negotiated, "negotiated")
assert.Equal(t, "negotiated", tc.negotiated, negotiated)
})
}
}
Expand Down Expand Up @@ -234,10 +234,14 @@ func Test_authenticateOrigin(t *testing.T) {

err := authenticateOrigin(r)
if tc.success {
assert.Success(t, err)
assert.Success(t, "authenticateOrigin", err)
} else {
assert.Error(t, err)
assert.Error(t, "authenticateOrigin", err)
}
})
}
}

func Test_acceptCompression(t *testing.T) {

}
26 changes: 13 additions & 13 deletions assert_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -3,19 +3,19 @@ package websocket_test
import (
"context"
"crypto/rand"
"io"
"strings"
"testing"

"cdr.dev/slog/sloggers/slogtest/assert"

"nhooyr.io/websocket"
"nhooyr.io/websocket/internal/assert"
"nhooyr.io/websocket/wsjson"
)

func randBytes(t *testing.T, n int) []byte {
b := make([]byte, n)
_, err := io.ReadFull(rand.Reader, b)
assert.Success(t, err)
_, err := rand.Reader.Read(b)
assert.Success(t, "readRandBytes", err)
return b
}

Expand All @@ -25,7 +25,7 @@ func assertJSONEcho(t *testing.T, ctx context.Context, c *websocket.Conn, n int)

exp := randString(t, n)
err := wsjson.Write(ctx, c, exp)
assert.Success(t, err)
assert.Success(t, "wsjson.Write", err)

assertJSONRead(t, ctx, c, exp)

Expand All @@ -37,9 +37,9 @@ func assertJSONRead(t *testing.T, ctx context.Context, c *websocket.Conn, exp in

var act interface{}
err := wsjson.Read(ctx, c, &act)
assert.Success(t, err)
assert.Success(t, "wsjson.Read", err)

assert.Equal(t, exp, act, "JSON")
assert.Equal(t, "json", exp, act)
}

func randString(t *testing.T, n int) string {
Expand All @@ -60,19 +60,19 @@ func assertEcho(t *testing.T, ctx context.Context, c *websocket.Conn, typ websoc

p := randBytes(t, n)
err := c.Write(ctx, typ, p)
assert.Success(t, err)
assert.Success(t, "write", err)

typ2, p2, err := c.Read(ctx)
assert.Success(t, err)
assert.Success(t, "read", err)

assert.Equal(t, typ, typ2, "data type")
assert.Equal(t, p, p2, "payload")
assert.Equal(t, "dataType", typ, typ2)
assert.Equal(t, "payload", p, p2)
}

func assertSubprotocol(t *testing.T, c *websocket.Conn, exp string) {
t.Helper()

assert.Equal(t, exp, c.Subprotocol(), "subprotocol")
assert.Equal(t, "subprotocol", exp, c.Subprotocol())
}

func assertCloseStatus(t *testing.T, exp websocket.StatusCode, err error) {
Expand All @@ -82,5 +82,5 @@ func assertCloseStatus(t *testing.T, exp websocket.StatusCode, err error) {
t.Logf("error: %+v", err)
}
}()
assert.Equal(t, exp, websocket.CloseStatus(err), "StatusCode")
assert.Equal(t, "closeStatus", exp, websocket.CloseStatus(err))
}
2 changes: 0 additions & 2 deletions ci/image/Dockerfile
Original file line number Diff line number Diff line change
Expand Up @@ -2,8 +2,6 @@ FROM golang:1

RUN apt-get update
RUN apt-get install -y chromium
RUN apt-get install -y npm
RUN apt-get install -y jq

ENV GOFLAGS="-mod=readonly"
ENV PAGER=cat
Expand Down
8 changes: 1 addition & 7 deletions ci/test.mk
Original file line number Diff line number Diff line change
Expand Up @@ -9,13 +9,7 @@ ci/out/coverage.html: gotest
coveralls: gotest
# https://github.com/coverallsapp/github-action/blob/master/src/run.ts
echo "--- coveralls"
export GIT_BRANCH="$$GITHUB_REF"
export BUILD_NUMBER="$$GITHUB_SHA"
if [[ $$GITHUB_EVENT_NAME == pull_request ]]; then
export CI_PULL_REQUEST="$$(jq .number "$$GITHUB_EVENT_PATH")"
BUILD_NUMBER="$$BUILD_NUMBER-PR-$$CI_PULL_REQUEST"
fi
goveralls -coverprofile=ci/out/coverage.prof -service=github
goveralls -coverprofile=ci/out/coverage.prof

gotest:
go test -covermode=count -coverprofile=ci/out/coverage.prof -coverpkg=./... $${GOTESTFLAGS-} ./...
Expand Down
11 changes: 9 additions & 2 deletions close.go
Original file line number Diff line number Diff line change
Expand Up @@ -30,7 +30,7 @@ const (
StatusProtocolError StatusCode = 1002
StatusUnsupportedData StatusCode = 1003

// 1004 is reserved and so not exported.
// 1004 is reserved and so unexported.
statusReserved StatusCode = 1004

// StatusNoStatusRcvd cannot be sent in a close message.
Expand Down Expand Up @@ -103,7 +103,6 @@ func (c *Conn) Close(code StatusCode, reason string) error {

func (c *Conn) closeHandshake(code StatusCode, reason string) (err error) {
defer errd.Wrap(&err, "failed to close WebSocket")
defer c.close(nil)

err = c.writeClose(code, reason)
if err != nil {
Expand All @@ -124,6 +123,14 @@ func (c *Conn) writeError(code StatusCode, err error) {
}

func (c *Conn) writeClose(code StatusCode, reason string) error {
c.closeMu.Lock()
closing := c.wroteClose
c.wroteClose = true
c.closeMu.Unlock()
if closing {
return errors.New("already wrote close")
}

ce := CloseError{
Code: code,
Reason: reason,
Expand Down
16 changes: 8 additions & 8 deletions close_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,7 @@ import (
"strings"
"testing"

"nhooyr.io/websocket/internal/assert"
"cdr.dev/slog/sloggers/slogtest/assert"
)

func TestCloseError(t *testing.T) {
Expand Down Expand Up @@ -52,9 +52,9 @@ func TestCloseError(t *testing.T) {

_, err := tc.ce.bytesErr()
if tc.success {
assert.Success(t, err)
assert.Success(t, "CloseError.bytesErr", err)
} else {
assert.Error(t, err)
assert.Error(t, "CloseError.bytesErr", err)
}
})
}
Expand Down Expand Up @@ -104,10 +104,10 @@ func Test_parseClosePayload(t *testing.T) {

ce, err := parseClosePayload(tc.p)
if tc.success {
assert.Success(t, err)
assert.Equal(t, tc.ce, ce, "CloseError")
assert.Success(t, "parse err", err)
assert.Equal(t, "ce", tc.ce, ce)
} else {
assert.Error(t, err)
assert.Error(t, "parse err", err)
}
})
}
Expand Down Expand Up @@ -153,7 +153,7 @@ func Test_validWireCloseCode(t *testing.T) {
t.Run(tc.name, func(t *testing.T) {
t.Parallel()

assert.Equal(t, tc.valid, validWireCloseCode(tc.code), "validWireCloseCode")
assert.Equal(t, "valid", tc.valid, validWireCloseCode(tc.code))
})
}
}
Expand Down Expand Up @@ -190,7 +190,7 @@ func TestCloseStatus(t *testing.T) {
t.Run(tc.name, func(t *testing.T) {
t.Parallel()

assert.Equal(t, tc.exp, CloseStatus(tc.in), "CloseStatus")
assert.Equal(t, "closeStatus", tc.exp, CloseStatus(tc.in))
})
}
}
6 changes: 4 additions & 2 deletions conn.go
Original file line number Diff line number Diff line change
Expand Up @@ -63,7 +63,7 @@ type Conn struct {
closed chan struct{}
closeMu sync.Mutex
closeErr error
wroteClose int64
wroteClose bool

pingCounter int32
activePingsMu sync.Mutex
Expand Down Expand Up @@ -244,7 +244,9 @@ func (m *mu) Lock(ctx context.Context) error {
case <-m.c.closed:
return m.c.closeErr
case <-ctx.Done():
return ctx.Err()
err := fmt.Errorf("failed to acquire lock: %w", ctx.Err())
m.c.close(err)
return err
case m.ch <- struct{}{}:
return nil
}
Expand Down
7 changes: 4 additions & 3 deletions conn_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -13,8 +13,9 @@ import (
"testing"
"time"

"cdr.dev/slog/sloggers/slogtest/assert"

"nhooyr.io/websocket"
"nhooyr.io/websocket/internal/assert"
)

func TestConn(t *testing.T) {
Expand All @@ -27,7 +28,7 @@ func TestConn(t *testing.T) {
InsecureSkipVerify: true,
CompressionMode: websocket.CompressionNoContextTakeover,
})
assert.Success(t, err)
assert.Success(t, "accept", err)
defer c.Close(websocket.StatusInternalError, "")

err = echoLoop(r.Context(), c)
Expand All @@ -47,7 +48,7 @@ func TestConn(t *testing.T) {
opts.HTTPClient = s.Client()

c, _, err := websocket.Dial(ctx, wsURL, opts)
assert.Success(t, err)
assert.Success(t, "dial", err)
assertJSONEcho(t, ctx, c, 2)
})
}
Expand Down
Loading