* First commit
* Imported tests from the original PR by @apolcyn.
* Formatting fixes.
* More formating fixes
* more golint
* Make logs more informative.
* post-review update
* Added test to check flow control accounts after sending large messages.
* post-review update
* Empty commit to kickstart travis.
* Post-review update.
This change saves a lot of memory by reusing the underlying
gzip.{Writer,Reader}, which allocates up to 1.4mb at every instanciation
according to [1]. This was fixed by adding a Reset method by to the
object at [2].
The amount of memory (and GC time) saved is pretty high, as reported by
pprof:
flat flat% sum% cum cum%
28.33GB 85.70% 85.70% 32.74GB 99.05% compress/flate.NewWriter
flat flat% sum% cum cum%
19.39MB 16.74% 16.74% 22.07MB 19.05% compress/flate.NewWriter
And the benchmarks:
benchmark old ns/op new ns/op delta
BenchmarkGZIPCompressor1B-4 215170 22291 -89.64%
BenchmarkGZIPCompressor1KiB-4 225971 27213 -87.96%
BenchmarkGZIPCompressor8KiB-4 246696 54785 -77.79%
BenchmarkGZIPCompressor64KiB-4 444851 286924 -35.50%
BenchmarkGZIPCompressor512KiB-4 2279043 2115863 -7.16%
BenchmarkGZIPCompressor1MiB-4 4412989 4258635 -3.50%
benchmark old allocs new allocs delta
BenchmarkGZIPCompressor1B-4 17 0 -100.00%
BenchmarkGZIPCompressor1KiB-4 17 0 -100.00%
BenchmarkGZIPCompressor8KiB-4 17 0 -100.00%
BenchmarkGZIPCompressor64KiB-4 17 0 -100.00%
BenchmarkGZIPCompressor512KiB-4 17 0 -100.00%
BenchmarkGZIPCompressor1MiB-4 17 0 -100.00%
benchmark old bytes new bytes delta
BenchmarkGZIPCompressor1B-4 813872 8 -100.00%
BenchmarkGZIPCompressor1KiB-4 813872 16 -100.00%
BenchmarkGZIPCompressor8KiB-4 813875 27 -100.00%
BenchmarkGZIPCompressor64KiB-4 813918 190 -99.98%
BenchmarkGZIPCompressor512KiB-4 814928 1871 -99.77%
BenchmarkGZIPCompressor1MiB-4 820889 9735 -98.81%
[1] https://github.com/golang/go/issues/6138
[2] db12f9d4e4
Signed-off-by: Steeve Morin <steeve.morin@gmail.com>
With this change, the default dialer checks environment variables to see if proxy is needed. If so, it dials to the proxy and does an HTTP CONNECT handshake.
* use a global sharded pool of proto.Buffer caches in protoCodec
* fix goimports
* make global buffer pool index counter atomic
* hack to remove alloc in encode_len_struct
* remove extra slice alloc in proto codec marshal
* replce magic number for proto size field length with constant
* replace custom cache with sync.Pool
* remove 1 line functions in codec.go and add protoCodec microbenchmarks
* add concurrent usage test for protoCodec
* fix golint.gofmt,goimport checks
* fix issues in codec.go and codec_test.go
* use go parallel benchmark helpers
* replace proto.Codec with a guess of size needed
* update Fatalf -> Errorf in tests
* wrap proto.Buffer along with cached last size into larger struct for pool use
* make wrapped proto buffer only a literal
* fix style and imports
* move b.Run into inner function
* reverse micro benchmark op order to unmarshal-marshal and fix benchmark setup-in-test bug
* add test for large message
* remove use of defer in codec.marshal
* revert recent changes to codec bencmarks
* move sub-benchmarks into >= go-1.7 only file
* add commentfor marshaler and tweak benchmark subtests for easier usage
* move build tag for go1.7 on benchmarks to inside file
* move build tag to top of file
* comment Codec, embed proto.Buffer into cached struct and add an int32 cap
