updated dependencies and moved to new rendering
This commit is contained in:
parent
320dc096dd
commit
0a55030ae5
467 changed files with 64111 additions and 111050 deletions
6
vendor/github.com/glycerine/go-unsnap-stream/README.md
generated
vendored
6
vendor/github.com/glycerine/go-unsnap-stream/README.md
generated
vendored
|
|
@ -7,7 +7,9 @@ Note that the *streaming or framing format* for snappy is different from snappy
|
|||
|
||||
Strangely, though the streaming format was first proposed in Go[1][2], it was never upated, and I could not locate any other library for Go that would handle the streaming/framed snappy format. Hence this implementation of the spec. There is a command line tool[3] that has a C implementation, but this is the only Go implementation that I am aware of. The reference for the framing/streaming spec seems to be the python implementation[4].
|
||||
|
||||
For binary compatibility with the python implementation, one could use the C-snappy compressor/decompressor code directly; using github.com/dgryski/go-csnappy. In fact we did this for a while to verify byte-for-byte compatiblity, as the native Go implementation produces slightly different binary compression (still conformant with the standard of course), which made test-diffs harder, and some have complained about it being slower than the C.
|
||||
Update to the previous paragraph: Horray! Good news: Thanks to @nigeltao, we have since learned that the [github.com/golang/snappy](https://github.com/golang/snappy) package now provides the snappy streaming format too. Even though the type level descriptions are a little misleading because they don't mention that they are for the stream format, the [snappy package header documentation](https://godoc.org/github.com/golang/snappy) points out that the [snappy.Reader](https://godoc.org/github.com/golang/snappy#Reader) and [snappy.Writer](https://godoc.org/github.com/golang/snappy#Writer) types do indeed provide stream (vs block) handling. Although I have not benchmarked, you should probably prefer that package as it will likely be maintained more than I have time to devote, and also perhaps better integrated with the underlying snappy as they share the same repo.
|
||||
|
||||
For binary compatibility with the [python implementation](https://pypi.python.org/pypi/python-snappy) in [4], one could use the C-snappy compressor/decompressor code directly; using github.com/dgryski/go-csnappy. In fact we did this for a while to verify byte-for-byte compatiblity, as the native Go implementation produces slightly different binary compression (still conformant with the standard of course), which made test-diffs harder, and some have complained about it being slower than the C.
|
||||
|
||||
However, while the c-snappy was useful for checking compatibility, it introduced dependencies on external C libraries (both the c-snappy library and the C standard library). Our go binary executable that used the go-unsnap-stream library was no longer standalone, and deployment was painful if not impossible if the target had a different C standard library. So we've gone back to using the snappy-go implementation (entirely in Go) for ease of deployment. See the comments at the top of unsnap.go if you wish to use c-snappy instead.
|
||||
|
||||
|
|
@ -17,4 +19,4 @@ However, while the c-snappy was useful for checking compatibility, it introduced
|
|||
|
||||
[3] https://github.com/kubo/snzip
|
||||
|
||||
[4] https://pypi.python.org/pypi/python-snappy
|
||||
[4] https://pypi.python.org/pypi/python-snappy
|
||||
|
|
|
|||
8
vendor/github.com/glycerine/go-unsnap-stream/unsnap.go
generated
vendored
8
vendor/github.com/glycerine/go-unsnap-stream/unsnap.go
generated
vendored
|
|
@ -7,6 +7,7 @@ import (
|
|||
"io"
|
||||
"io/ioutil"
|
||||
"os"
|
||||
"strings"
|
||||
|
||||
"hash/crc32"
|
||||
|
||||
|
|
@ -189,7 +190,12 @@ func UnsnapOneFrame(r io.Reader, encBuf *FixedSizeRingBuf, outDecodedBuf *FixedS
|
|||
err = nil
|
||||
}
|
||||
} else {
|
||||
panic(err)
|
||||
// may be an odd already closed... don't panic on that
|
||||
if strings.Contains(err.Error(), "file already closed") {
|
||||
err = nil
|
||||
} else {
|
||||
panic(err)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue