updated dependencies
This commit is contained in:
parent
3f9b863171
commit
f0add04396
266 changed files with 18771 additions and 10200 deletions
7
vendor/github.com/RoaringBitmap/roaring/.travis.yml
generated
vendored
7
vendor/github.com/RoaringBitmap/roaring/.travis.yml
generated
vendored
|
|
@ -8,13 +8,8 @@ install:
|
|||
notifications:
|
||||
email: false
|
||||
go:
|
||||
- "1.7.x"
|
||||
- "1.8.x"
|
||||
- "1.9.x"
|
||||
- "1.10.x"
|
||||
- "1.11.x"
|
||||
- "1.12.x"
|
||||
- "1.13.x"
|
||||
- "1.14.x"
|
||||
- tip
|
||||
|
||||
# whitelist
|
||||
|
|
|
|||
4
vendor/github.com/RoaringBitmap/roaring/CONTRIBUTORS
generated
vendored
4
vendor/github.com/RoaringBitmap/roaring/CONTRIBUTORS
generated
vendored
|
|
@ -13,4 +13,6 @@ Forud Ghafouri (@fzerorubigd),
|
|||
Joe Nall (@joenall),
|
||||
(@fredim),
|
||||
Edd Robinson (@e-dard),
|
||||
Alexander Petrov (@alldroll)
|
||||
Alexander Petrov (@alldroll),
|
||||
Guy Molinari (@guymolinari),
|
||||
Ling Jin (@JinLingChristopher)
|
||||
|
|
|
|||
156
vendor/github.com/RoaringBitmap/roaring/README.md
generated
vendored
156
vendor/github.com/RoaringBitmap/roaring/README.md
generated
vendored
|
|
@ -1,5 +1,8 @@
|
|||
roaring [](https://travis-ci.org/RoaringBitmap/roaring) [](https://coveralls.io/github/RoaringBitmap/roaring?branch=master) [](https://godoc.org/github.com/RoaringBitmap/roaring) [](https://goreportcard.com/report/github.com/RoaringBitmap/roaring)
|
||||
roaring [](https://travis-ci.org/RoaringBitmap/roaring) [](https://godoc.org/github.com/RoaringBitmap/roaring) [](https://godoc.org/github.com/RoaringBitmap/roaring/roaring64) [](https://goreportcard.com/report/github.com/RoaringBitmap/roaring)
|
||||
[](https://cloud.drone.io/RoaringBitmap/roaring)
|
||||
|
||||
|
||||
|
||||
=============
|
||||
|
||||
This is a go version of the Roaring bitmap data structure.
|
||||
|
|
@ -7,7 +10,7 @@ This is a go version of the Roaring bitmap data structure.
|
|||
|
||||
|
||||
Roaring bitmaps are used by several major systems such as [Apache Lucene][lucene] and derivative systems such as [Solr][solr] and
|
||||
[Elasticsearch][elasticsearch], [Apache Druid (Incubating)][druid], [LinkedIn Pinot][pinot], [Netflix Atlas][atlas], [Apache Spark][spark], [OpenSearchServer][opensearchserver], [Cloud Torrent][cloudtorrent], [Whoosh][whoosh], [Pilosa][pilosa], [Microsoft Visual Studio Team Services (VSTS)][vsts], and eBay's [Apache Kylin][kylin].
|
||||
[Elasticsearch][elasticsearch], [Apache Druid (Incubating)][druid], [LinkedIn Pinot][pinot], [Netflix Atlas][atlas], [Apache Spark][spark], [OpenSearchServer][opensearchserver], [Cloud Torrent][cloudtorrent], [Whoosh][whoosh], [Pilosa][pilosa], [Microsoft Visual Studio Team Services (VSTS)][vsts], and eBay's [Apache Kylin][kylin]. The YouTube SQL Engine, [Google Procella](https://research.google/pubs/pub48388/), uses Roaring bitmaps for indexing.
|
||||
|
||||
[lucene]: https://lucene.apache.org/
|
||||
[solr]: https://lucene.apache.org/solr/
|
||||
|
|
@ -52,6 +55,93 @@ This code is licensed under Apache License, Version 2.0 (ASL2.0).
|
|||
|
||||
Copyright 2016-... by the authors.
|
||||
|
||||
When should you use a bitmap?
|
||||
===================================
|
||||
|
||||
|
||||
Sets are a fundamental abstraction in
|
||||
software. They can be implemented in various
|
||||
ways, as hash sets, as trees, and so forth.
|
||||
In databases and search engines, sets are often an integral
|
||||
part of indexes. For example, we may need to maintain a set
|
||||
of all documents or rows (represented by numerical identifier)
|
||||
that satisfy some property. Besides adding or removing
|
||||
elements from the set, we need fast functions
|
||||
to compute the intersection, the union, the difference between sets, and so on.
|
||||
|
||||
|
||||
To implement a set
|
||||
of integers, a particularly appealing strategy is the
|
||||
bitmap (also called bitset or bit vector). Using n bits,
|
||||
we can represent any set made of the integers from the range
|
||||
[0,n): the ith bit is set to one if integer i is present in the set.
|
||||
Commodity processors use words of W=32 or W=64 bits. By combining many such words, we can
|
||||
support large values of n. Intersections, unions and differences can then be implemented
|
||||
as bitwise AND, OR and ANDNOT operations.
|
||||
More complicated set functions can also be implemented as bitwise operations.
|
||||
|
||||
When the bitset approach is applicable, it can be orders of
|
||||
magnitude faster than other possible implementation of a set (e.g., as a hash set)
|
||||
while using several times less memory.
|
||||
|
||||
However, a bitset, even a compressed one is not always applicable. For example, if the
|
||||
you have 1000 random-looking integers, then a simple array might be the best representation.
|
||||
We refer to this case as the "sparse" scenario.
|
||||
|
||||
When should you use compressed bitmaps?
|
||||
===================================
|
||||
|
||||
An uncompressed BitSet can use a lot of memory. For example, if you take a BitSet
|
||||
and set the bit at position 1,000,000 to true and you have just over 100kB. That is over 100kB
|
||||
to store the position of one bit. This is wasteful even if you do not care about memory:
|
||||
suppose that you need to compute the intersection between this BitSet and another one
|
||||
that has a bit at position 1,000,001 to true, then you need to go through all these zeroes,
|
||||
whether you like it or not. That can become very wasteful.
|
||||
|
||||
This being said, there are definitively cases where attempting to use compressed bitmaps is wasteful.
|
||||
For example, if you have a small universe size. E.g., your bitmaps represent sets of integers
|
||||
from [0,n) where n is small (e.g., n=64 or n=128). If you are able to uncompressed BitSet and
|
||||
it does not blow up your memory usage, then compressed bitmaps are probably not useful
|
||||
to you. In fact, if you do not need compression, then a BitSet offers remarkable speed.
|
||||
|
||||
The sparse scenario is another use case where compressed bitmaps should not be used.
|
||||
Keep in mind that random-looking data is usually not compressible. E.g., if you have a small set of
|
||||
32-bit random integers, it is not mathematically possible to use far less than 32 bits per integer,
|
||||
and attempts at compression can be counterproductive.
|
||||
|
||||
How does Roaring compares with the alternatives?
|
||||
==================================================
|
||||
|
||||
|
||||
Most alternatives to Roaring are part of a larger family of compressed bitmaps that are run-length-encoded
|
||||
bitmaps. They identify long runs of 1s or 0s and they represent them with a marker word.
|
||||
If you have a local mix of 1s and 0, you use an uncompressed word.
|
||||
|
||||
There are many formats in this family:
|
||||
|
||||
* Oracle's BBC is an obsolete format at this point: though it may provide good compression,
|
||||
it is likely much slower than more recent alternatives due to excessive branching.
|
||||
* WAH is a patented variation on BBC that provides better performance.
|
||||
* Concise is a variation on the patented WAH. It some specific instances, it can compress
|
||||
much better than WAH (up to 2x better), but it is generally slower.
|
||||
* EWAH is both free of patent, and it is faster than all the above. On the downside, it
|
||||
does not compress quite as well. It is faster because it allows some form of "skipping"
|
||||
over uncompressed words. So though none of these formats are great at random access, EWAH
|
||||
is better than the alternatives.
|
||||
|
||||
|
||||
|
||||
There is a big problem with these formats however that can hurt you badly in some cases: there is no random access. If you want to check whether a given value is present in the set, you have to start from the beginning and "uncompress" the whole thing. This means that if you want to intersect a big set with a large set, you still have to uncompress the whole big set in the worst case...
|
||||
|
||||
Roaring solves this problem. It works in the following manner. It divides the data into chunks of 2<sup>16</sup> integers
|
||||
(e.g., [0, 2<sup>16</sup>), [2<sup>16</sup>, 2 x 2<sup>16</sup>), ...). Within a chunk, it can use an uncompressed bitmap, a simple list of integers,
|
||||
or a list of runs. Whatever format it uses, they all allow you to check for the present of any one value quickly
|
||||
(e.g., with a binary search). The net result is that Roaring can compute many operations much faster than run-length-encoded
|
||||
formats like WAH, EWAH, Concise... Maybe surprisingly, Roaring also generally offers better compression ratios.
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
### References
|
||||
|
||||
|
|
@ -172,10 +262,70 @@ That is, given a fixed overhead for the universe size (x), Roaring
|
|||
bitmaps never use more than 2 bytes per integer. You can call
|
||||
``BoundSerializedSizeInBytes`` for a more precise estimate.
|
||||
|
||||
### 64-bit Roaring
|
||||
|
||||
By default, roaring is used to stored unsigned 32-bit integers. However, we also offer
|
||||
an extension dedicated to 64-bit integers. It supports roughly the same functions:
|
||||
|
||||
```go
|
||||
package main
|
||||
|
||||
import (
|
||||
"fmt"
|
||||
"github.com/RoaringBitmap/roaring/roaring64"
|
||||
"bytes"
|
||||
)
|
||||
|
||||
|
||||
func main() {
|
||||
// example inspired by https://github.com/fzandona/goroar
|
||||
fmt.Println("==roaring64==")
|
||||
rb1 := roaring64.BitmapOf(1, 2, 3, 4, 5, 100, 1000)
|
||||
fmt.Println(rb1.String())
|
||||
|
||||
rb2 := roaring64.BitmapOf(3, 4, 1000)
|
||||
fmt.Println(rb2.String())
|
||||
|
||||
rb3 := roaring64.New()
|
||||
fmt.Println(rb3.String())
|
||||
|
||||
fmt.Println("Cardinality: ", rb1.GetCardinality())
|
||||
|
||||
fmt.Println("Contains 3? ", rb1.Contains(3))
|
||||
|
||||
rb1.And(rb2)
|
||||
|
||||
rb3.Add(1)
|
||||
rb3.Add(5)
|
||||
|
||||
rb3.Or(rb1)
|
||||
|
||||
|
||||
|
||||
// prints 1, 3, 4, 5, 1000
|
||||
i := rb3.Iterator()
|
||||
for i.HasNext() {
|
||||
fmt.Println(i.Next())
|
||||
}
|
||||
fmt.Println()
|
||||
|
||||
// next we include an example of serialization
|
||||
buf := new(bytes.Buffer)
|
||||
rb1.WriteTo(buf) // we omit error handling
|
||||
newrb:= roaring64.New()
|
||||
newrb.ReadFrom(buf)
|
||||
if rb1.Equals(newrb) {
|
||||
fmt.Println("I wrote the content to a byte stream and read it back.")
|
||||
}
|
||||
// you can iterate over bitmaps using ReverseIterator(), Iterator, ManyIterator()
|
||||
}
|
||||
```
|
||||
|
||||
Only the 32-bit roaring format is standard and cross-operable between Java, C++, C and Go. There is no guarantee that the 64-bit versions are compatible.
|
||||
|
||||
### Documentation
|
||||
|
||||
Current documentation is available at http://godoc.org/github.com/RoaringBitmap/roaring
|
||||
Current documentation is available at http://godoc.org/github.com/RoaringBitmap/roaring and http://godoc.org/github.com/RoaringBitmap/roaring64
|
||||
|
||||
### Goroutine safety
|
||||
|
||||
|
|
|
|||
73
vendor/github.com/RoaringBitmap/roaring/arraycontainer.go
generated
vendored
73
vendor/github.com/RoaringBitmap/roaring/arraycontainer.go
generated
vendored
|
|
@ -359,28 +359,17 @@ func (ac *arrayContainer) iorArray(value2 *arrayContainer) container {
|
|||
len1 := value1.getCardinality()
|
||||
len2 := value2.getCardinality()
|
||||
maxPossibleCardinality := len1 + len2
|
||||
if maxPossibleCardinality > arrayDefaultMaxSize { // it could be a bitmap!
|
||||
bc := newBitmapContainer()
|
||||
for k := 0; k < len(value2.content); k++ {
|
||||
v := value2.content[k]
|
||||
i := uint(v) >> 6
|
||||
mask := uint64(1) << (v % 64)
|
||||
bc.bitmap[i] |= mask
|
||||
}
|
||||
for k := 0; k < len(ac.content); k++ {
|
||||
v := ac.content[k]
|
||||
i := uint(v) >> 6
|
||||
mask := uint64(1) << (v % 64)
|
||||
bc.bitmap[i] |= mask
|
||||
}
|
||||
bc.cardinality = int(popcntSlice(bc.bitmap))
|
||||
if bc.cardinality <= arrayDefaultMaxSize {
|
||||
return bc.toArrayContainer()
|
||||
}
|
||||
return bc
|
||||
}
|
||||
if maxPossibleCardinality > cap(value1.content) {
|
||||
newcontent := make([]uint16, 0, maxPossibleCardinality)
|
||||
// doubling the capacity reduces new slice allocations in the case of
|
||||
// repeated calls to iorArray().
|
||||
newSize := 2 * maxPossibleCardinality
|
||||
// the second check is to handle overly large array containers
|
||||
// and should not occur in normal usage,
|
||||
// as all array containers should be at most arrayDefaultMaxSize
|
||||
if newSize > 2*arrayDefaultMaxSize && maxPossibleCardinality <= 2*arrayDefaultMaxSize {
|
||||
newSize = 2 * arrayDefaultMaxSize
|
||||
}
|
||||
newcontent := make([]uint16, 0, newSize)
|
||||
copy(newcontent[len2:maxPossibleCardinality], ac.content[0:len1])
|
||||
ac.content = newcontent
|
||||
} else {
|
||||
|
|
@ -388,6 +377,13 @@ func (ac *arrayContainer) iorArray(value2 *arrayContainer) container {
|
|||
}
|
||||
nl := union2by2(value1.content[len2:maxPossibleCardinality], value2.content, ac.content)
|
||||
ac.content = ac.content[:nl] // reslice to match actual used capacity
|
||||
|
||||
if nl > arrayDefaultMaxSize {
|
||||
// Only converting to a bitmap when arrayDefaultMaxSize
|
||||
// is actually exceeded minimizes conversions in the case of repeated
|
||||
// calls to iorArray().
|
||||
return ac.toBitmapContainer()
|
||||
}
|
||||
return ac
|
||||
}
|
||||
|
||||
|
|
@ -876,6 +872,41 @@ func (ac *arrayContainer) loadData(bitmapContainer *bitmapContainer) {
|
|||
ac.content = make([]uint16, bitmapContainer.cardinality, bitmapContainer.cardinality)
|
||||
bitmapContainer.fillArray(ac.content)
|
||||
}
|
||||
|
||||
func (ac *arrayContainer) resetTo(a container) {
|
||||
switch x := a.(type) {
|
||||
case *arrayContainer:
|
||||
ac.realloc(len(x.content))
|
||||
copy(ac.content, x.content)
|
||||
|
||||
case *bitmapContainer:
|
||||
ac.realloc(x.cardinality)
|
||||
x.fillArray(ac.content)
|
||||
|
||||
case *runContainer16:
|
||||
card := int(x.cardinality())
|
||||
ac.realloc(card)
|
||||
cur := 0
|
||||
for _, r := range x.iv {
|
||||
for val := r.start; val <= r.last(); val++ {
|
||||
ac.content[cur] = val
|
||||
cur++
|
||||
}
|
||||
}
|
||||
|
||||
default:
|
||||
panic("unsupported container type")
|
||||
}
|
||||
}
|
||||
|
||||
func (ac *arrayContainer) realloc(size int) {
|
||||
if cap(ac.content) < size {
|
||||
ac.content = make([]uint16, size)
|
||||
} else {
|
||||
ac.content = ac.content[:size]
|
||||
}
|
||||
}
|
||||
|
||||
func newArrayContainer() *arrayContainer {
|
||||
p := new(arrayContainer)
|
||||
return p
|
||||
|
|
|
|||
53
vendor/github.com/RoaringBitmap/roaring/bitmapcontainer.go
generated
vendored
53
vendor/github.com/RoaringBitmap/roaring/bitmapcontainer.go
generated
vendored
|
|
@ -203,6 +203,33 @@ func (bcmi *bitmapContainerManyIterator) nextMany(hs uint32, buf []uint32) int {
|
|||
return n
|
||||
}
|
||||
|
||||
func (bcmi *bitmapContainerManyIterator) nextMany64(hs uint64, buf []uint64) int {
|
||||
n := 0
|
||||
base := bcmi.base
|
||||
bitset := bcmi.bitset
|
||||
|
||||
for n < len(buf) {
|
||||
if bitset == 0 {
|
||||
base++
|
||||
if base >= len(bcmi.ptr.bitmap) {
|
||||
bcmi.base = base
|
||||
bcmi.bitset = bitset
|
||||
return n
|
||||
}
|
||||
bitset = bcmi.ptr.bitmap[base]
|
||||
continue
|
||||
}
|
||||
t := bitset & -bitset
|
||||
buf[n] = uint64(((base * 64) + int(popcount(t-1)))) | hs
|
||||
n = n + 1
|
||||
bitset ^= t
|
||||
}
|
||||
|
||||
bcmi.base = base
|
||||
bcmi.bitset = bitset
|
||||
return n
|
||||
}
|
||||
|
||||
func newBitmapContainerManyIterator(a *bitmapContainer) *bitmapContainerManyIterator {
|
||||
return &bitmapContainerManyIterator{a, -1, 0}
|
||||
}
|
||||
|
|
@ -934,6 +961,32 @@ func (bc *bitmapContainer) loadData(arrayContainer *arrayContainer) {
|
|||
}
|
||||
}
|
||||
|
||||
func (bc *bitmapContainer) resetTo(a container) {
|
||||
switch x := a.(type) {
|
||||
case *arrayContainer:
|
||||
fill(bc.bitmap, 0)
|
||||
bc.loadData(x)
|
||||
|
||||
case *bitmapContainer:
|
||||
bc.cardinality = x.cardinality
|
||||
copy(bc.bitmap, x.bitmap)
|
||||
|
||||
case *runContainer16:
|
||||
bc.cardinality = len(x.iv)
|
||||
lastEnd := 0
|
||||
for _, r := range x.iv {
|
||||
bc.cardinality += int(r.length)
|
||||
resetBitmapRange(bc.bitmap, lastEnd, int(r.start))
|
||||
lastEnd = int(r.start+r.length) + 1
|
||||
setBitmapRange(bc.bitmap, int(r.start), lastEnd)
|
||||
}
|
||||
resetBitmapRange(bc.bitmap, lastEnd, maxCapacity)
|
||||
|
||||
default:
|
||||
panic("unsupported container type")
|
||||
}
|
||||
}
|
||||
|
||||
func (bc *bitmapContainer) toArrayContainer() *arrayContainer {
|
||||
ac := &arrayContainer{}
|
||||
ac.loadData(bc)
|
||||
|
|
|
|||
117
vendor/github.com/RoaringBitmap/roaring/fastaggregation.go
generated
vendored
117
vendor/github.com/RoaringBitmap/roaring/fastaggregation.go
generated
vendored
|
|
@ -213,3 +213,120 @@ func HeapXor(bitmaps ...*Bitmap) *Bitmap {
|
|||
}
|
||||
return heap.Pop(&pq).(*item).value
|
||||
}
|
||||
|
||||
// AndAny provides a result equivalent to x1.And(FastOr(bitmaps)).
|
||||
// It's optimized to minimize allocations. It also might be faster than separate calls.
|
||||
func (x1 *Bitmap) AndAny(bitmaps ...*Bitmap) {
|
||||
if len(bitmaps) == 0 {
|
||||
return
|
||||
} else if len(bitmaps) == 1 {
|
||||
x1.And(bitmaps[0])
|
||||
return
|
||||
}
|
||||
|
||||
type withPos struct {
|
||||
bitmap *roaringArray
|
||||
pos int
|
||||
key uint16
|
||||
}
|
||||
filters := make([]withPos, 0, len(bitmaps))
|
||||
|
||||
for _, b := range bitmaps {
|
||||
if b.highlowcontainer.size() > 0 {
|
||||
filters = append(filters, withPos{
|
||||
bitmap: &b.highlowcontainer,
|
||||
pos: 0,
|
||||
key: b.highlowcontainer.getKeyAtIndex(0),
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
basePos := 0
|
||||
intersections := 0
|
||||
keyContainers := make([]container, 0, len(filters))
|
||||
var (
|
||||
tmpArray *arrayContainer
|
||||
tmpBitmap *bitmapContainer
|
||||
minNextKey uint16
|
||||
)
|
||||
|
||||
for basePos < x1.highlowcontainer.size() && len(filters) > 0 {
|
||||
baseKey := x1.highlowcontainer.getKeyAtIndex(basePos)
|
||||
|
||||
// accumulate containers for current key, find next minimal key in filters
|
||||
// and exclude filters that do not have related values anymore
|
||||
i := 0
|
||||
maxPossibleOr := 0
|
||||
minNextKey = MaxUint16
|
||||
for _, f := range filters {
|
||||
if f.key < baseKey {
|
||||
f.pos = f.bitmap.advanceUntil(baseKey, f.pos)
|
||||
if f.pos == f.bitmap.size() {
|
||||
continue
|
||||
}
|
||||
f.key = f.bitmap.getKeyAtIndex(f.pos)
|
||||
}
|
||||
|
||||
if f.key == baseKey {
|
||||
cont := f.bitmap.getContainerAtIndex(f.pos)
|
||||
keyContainers = append(keyContainers, cont)
|
||||
maxPossibleOr += cont.getCardinality()
|
||||
|
||||
f.pos++
|
||||
if f.pos == f.bitmap.size() {
|
||||
continue
|
||||
}
|
||||
f.key = f.bitmap.getKeyAtIndex(f.pos)
|
||||
}
|
||||
|
||||
minNextKey = minOfUint16(minNextKey, f.key)
|
||||
filters[i] = f
|
||||
i++
|
||||
}
|
||||
filters = filters[:i]
|
||||
|
||||
if len(keyContainers) == 0 {
|
||||
basePos = x1.highlowcontainer.advanceUntil(minNextKey, basePos)
|
||||
continue
|
||||
}
|
||||
|
||||
var ored container
|
||||
|
||||
if len(keyContainers) == 1 {
|
||||
ored = keyContainers[0]
|
||||
} else {
|
||||
//TODO: special case for run containers?
|
||||
if maxPossibleOr > arrayDefaultMaxSize {
|
||||
if tmpBitmap == nil {
|
||||
tmpBitmap = newBitmapContainer()
|
||||
}
|
||||
tmpBitmap.resetTo(keyContainers[0])
|
||||
for _, c := range keyContainers[1:] {
|
||||
tmpBitmap.ior(c)
|
||||
}
|
||||
ored = tmpBitmap
|
||||
} else {
|
||||
if tmpArray == nil {
|
||||
tmpArray = newArrayContainerCapacity(maxPossibleOr)
|
||||
}
|
||||
tmpArray.realloc(maxPossibleOr)
|
||||
tmpArray.resetTo(keyContainers[0])
|
||||
for _, c := range keyContainers[1:] {
|
||||
tmpArray.ior(c)
|
||||
}
|
||||
ored = tmpArray
|
||||
}
|
||||
}
|
||||
|
||||
result := x1.highlowcontainer.getWritableContainerAtIndex(basePos).iand(ored)
|
||||
if result.getCardinality() > 0 {
|
||||
x1.highlowcontainer.replaceKeyAndContainerAtIndex(intersections, baseKey, result, false)
|
||||
intersections++
|
||||
}
|
||||
|
||||
keyContainers = keyContainers[:0]
|
||||
basePos = x1.highlowcontainer.advanceUntil(minNextKey, basePos)
|
||||
}
|
||||
|
||||
x1.highlowcontainer.resize(intersections)
|
||||
}
|
||||
|
|
|
|||
4
vendor/github.com/RoaringBitmap/roaring/go.mod
generated
vendored
4
vendor/github.com/RoaringBitmap/roaring/go.mod
generated
vendored
|
|
@ -1,6 +1,6 @@
|
|||
module github.com/RoaringBitmap/roaring
|
||||
|
||||
go 1.12
|
||||
go 1.14
|
||||
|
||||
require (
|
||||
github.com/glycerine/go-unsnap-stream v0.0.0-20181221182339-f9677308dec2
|
||||
|
|
@ -13,4 +13,6 @@ require (
|
|||
github.com/stretchr/testify v1.4.0
|
||||
github.com/tinylib/msgp v1.1.0
|
||||
github.com/willf/bitset v1.1.10
|
||||
golang.org/x/lint v0.0.0-20200302205851-738671d3881b // indirect
|
||||
golang.org/x/tools v0.0.0-20200928182047-19e03678916f // indirect
|
||||
)
|
||||
|
|
|
|||
25
vendor/github.com/RoaringBitmap/roaring/go.sum
generated
vendored
25
vendor/github.com/RoaringBitmap/roaring/go.sum
generated
vendored
|
|
@ -24,6 +24,31 @@ github.com/tinylib/msgp v1.1.0 h1:9fQd+ICuRIu/ue4vxJZu6/LzxN0HwMds2nq/0cFvxHU=
|
|||
github.com/tinylib/msgp v1.1.0/go.mod h1:+d+yLhGm8mzTaHzB+wgMYrodPfmZrzkirds8fDWklFE=
|
||||
github.com/willf/bitset v1.1.10 h1:NotGKqX0KwQ72NUzqrjZq5ipPNDQex9lo3WpaS8L2sc=
|
||||
github.com/willf/bitset v1.1.10/go.mod h1:RjeCKbqT1RxIR/KWY6phxZiaY1IyutSBfGjNPySAYV4=
|
||||
github.com/yuin/goldmark v1.2.1/go.mod h1:3hX8gzYuyVAZsxl0MRgGTJEmQBFcNTphYh9decYSb74=
|
||||
golang.org/x/crypto v0.0.0-20190308221718-c2843e01d9a2/go.mod h1:djNgcEr1/C05ACkg1iLfiJU5Ep61QUkGW8qpdssI0+w=
|
||||
golang.org/x/crypto v0.0.0-20191011191535-87dc89f01550/go.mod h1:yigFU9vqHzYiE8UmvKecakEJjdnWj3jj499lnFckfCI=
|
||||
golang.org/x/crypto v0.0.0-20200622213623-75b288015ac9/go.mod h1:LzIPMQfyMNhhGPhUkYOs5KpL4U8rLKemX1yGLhDgUto=
|
||||
golang.org/x/lint v0.0.0-20200302205851-738671d3881b h1:Wh+f8QHJXR411sJR8/vRBTZ7YapZaRvUcLFFJhusH0k=
|
||||
golang.org/x/lint v0.0.0-20200302205851-738671d3881b/go.mod h1:3xt1FjdF8hUf6vQPIChWIBhFzV8gjjsPE/fR3IyQdNY=
|
||||
golang.org/x/mod v0.1.1-0.20191105210325-c90efee705ee/go.mod h1:QqPTAvyqsEbceGzBzNggFXnrqF1CaUcvgkdR5Ot7KZg=
|
||||
golang.org/x/mod v0.3.0/go.mod h1:s0Qsj1ACt9ePp/hMypM3fl4fZqREWJwdYDEqhRiZZUA=
|
||||
golang.org/x/net v0.0.0-20190404232315-eb5bcb51f2a3/go.mod h1:t9HGtf8HONx5eT2rtn7q6eTqICYqUVnKs3thJo3Qplg=
|
||||
golang.org/x/net v0.0.0-20190620200207-3b0461eec859/go.mod h1:z5CRVTTTmAJ677TzLLGU+0bjPO0LkuOLi4/5GtJWs/s=
|
||||
golang.org/x/net v0.0.0-20200822124328-c89045814202/go.mod h1:/O7V0waA8r7cgGh81Ro3o1hOxt32SMVPicZroKQ2sZA=
|
||||
golang.org/x/sync v0.0.0-20190423024810-112230192c58/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=
|
||||
golang.org/x/sync v0.0.0-20200625203802-6e8e738ad208/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=
|
||||
golang.org/x/sys v0.0.0-20190215142949-d0b11bdaac8a/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY=
|
||||
golang.org/x/sys v0.0.0-20190412213103-97732733099d/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
|
||||
golang.org/x/sys v0.0.0-20200323222414-85ca7c5b95cd/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
|
||||
golang.org/x/text v0.3.0/go.mod h1:NqM8EUOU14njkJ3fqMW+pc6Ldnwhi/IjpwHt7yyuwOQ=
|
||||
golang.org/x/tools v0.0.0-20191119224855-298f0cb1881e/go.mod h1:b+2E5dAYhXwXZwtnZ6UAqBI28+e2cm9otk0dWdXHAEo=
|
||||
golang.org/x/tools v0.0.0-20200130002326-2f3ba24bd6e7 h1:EBZoQjiKKPaLbPrbpssUfuHtwM6KV/vb4U85g/cigFY=
|
||||
golang.org/x/tools v0.0.0-20200130002326-2f3ba24bd6e7/go.mod h1:TB2adYChydJhpapKDTa4BR/hXlZSLoq2Wpct/0txZ28=
|
||||
golang.org/x/tools v0.0.0-20200928182047-19e03678916f h1:VwGa2Wf+rHGIxvsssCkUNIyFv8jQY0VCBCNWtikoWq0=
|
||||
golang.org/x/tools v0.0.0-20200928182047-19e03678916f/go.mod h1:z6u4i615ZeAfBE4XtMziQW1fSVJXACjjbWkB/mvPzlU=
|
||||
golang.org/x/xerrors v0.0.0-20190717185122-a985d3407aa7/go.mod h1:I/5z698sn9Ka8TeJc9MKroUUfqBBauWjQqLJ2OPfmY0=
|
||||
golang.org/x/xerrors v0.0.0-20191011141410-1b5146add898/go.mod h1:I/5z698sn9Ka8TeJc9MKroUUfqBBauWjQqLJ2OPfmY0=
|
||||
golang.org/x/xerrors v0.0.0-20200804184101-5ec99f83aff1/go.mod h1:I/5z698sn9Ka8TeJc9MKroUUfqBBauWjQqLJ2OPfmY0=
|
||||
gopkg.in/check.v1 v0.0.0-20161208181325-20d25e280405 h1:yhCVgyC4o1eVCa2tZl7eS0r+SDo693bJlVdllGtEeKM=
|
||||
gopkg.in/check.v1 v0.0.0-20161208181325-20d25e280405/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0=
|
||||
gopkg.in/yaml.v2 v2.2.2 h1:ZCJp+EgiOT7lHqUV2J862kp8Qj64Jo6az82+3Td9dZw=
|
||||
|
|
|
|||
14
vendor/github.com/RoaringBitmap/roaring/manyiterator.go
generated
vendored
14
vendor/github.com/RoaringBitmap/roaring/manyiterator.go
generated
vendored
|
|
@ -2,6 +2,7 @@ package roaring
|
|||
|
||||
type manyIterable interface {
|
||||
nextMany(hs uint32, buf []uint32) int
|
||||
nextMany64(hs uint64, buf []uint64) int
|
||||
}
|
||||
|
||||
func (si *shortIterator) nextMany(hs uint32, buf []uint32) int {
|
||||
|
|
@ -16,3 +17,16 @@ func (si *shortIterator) nextMany(hs uint32, buf []uint32) int {
|
|||
si.loc = l
|
||||
return n
|
||||
}
|
||||
|
||||
func (si *shortIterator) nextMany64(hs uint64, buf []uint64) int {
|
||||
n := 0
|
||||
l := si.loc
|
||||
s := si.slice
|
||||
for n < len(buf) && l < len(s) {
|
||||
buf[n] = uint64(s[l]) | hs
|
||||
l++
|
||||
n++
|
||||
}
|
||||
si.loc = l
|
||||
return n
|
||||
}
|
||||
|
|
|
|||
36
vendor/github.com/RoaringBitmap/roaring/roaring.go
generated
vendored
36
vendor/github.com/RoaringBitmap/roaring/roaring.go
generated
vendored
|
|
@ -345,8 +345,10 @@ func newIntReverseIterator(a *Bitmap) *intReverseIterator {
|
|||
|
||||
// ManyIntIterable allows you to iterate over the values in a Bitmap
|
||||
type ManyIntIterable interface {
|
||||
// pass in a buffer to fill up with values, returns how many values were returned
|
||||
NextMany([]uint32) int
|
||||
// NextMany fills buf up with values, returns how many values were returned
|
||||
NextMany(buf []uint32) int
|
||||
// NextMany64 fills up buf with 64 bit values, uses hs as a mask (OR), returns how many values were returned
|
||||
NextMany64(hs uint64, buf []uint64) int
|
||||
}
|
||||
|
||||
type manyIntIterator struct {
|
||||
|
|
@ -382,6 +384,25 @@ func (ii *manyIntIterator) NextMany(buf []uint32) int {
|
|||
return n
|
||||
}
|
||||
|
||||
func (ii *manyIntIterator) NextMany64(hs64 uint64, buf []uint64) int {
|
||||
n := 0
|
||||
for n < len(buf) {
|
||||
if ii.iter == nil {
|
||||
break
|
||||
}
|
||||
|
||||
hs := uint64(ii.hs) | hs64
|
||||
moreN := ii.iter.nextMany64(hs, buf[n:])
|
||||
n += moreN
|
||||
if moreN == 0 {
|
||||
ii.pos = ii.pos + 1
|
||||
ii.init()
|
||||
}
|
||||
}
|
||||
|
||||
return n
|
||||
}
|
||||
|
||||
func newManyIntIterator(a *Bitmap) *manyIntIterator {
|
||||
p := new(manyIntIterator)
|
||||
p.pos = 0
|
||||
|
|
@ -678,7 +699,10 @@ func (rb *Bitmap) GetCardinality() uint64 {
|
|||
return size
|
||||
}
|
||||
|
||||
// Rank returns the number of integers that are smaller or equal to x (Rank(infinity) would be GetCardinality())
|
||||
// Rank returns the number of integers that are smaller or equal to x (Rank(infinity) would be GetCardinality()).
|
||||
// If you pass the smallest value, you get the value 1. If you pass a value that is smaller than the smallest
|
||||
// value, you get 0. Note that this function differs in convention from the Select function since it
|
||||
// return 1 and not 0 on the smallest value.
|
||||
func (rb *Bitmap) Rank(x uint32) uint64 {
|
||||
size := uint64(0)
|
||||
for i := 0; i < rb.highlowcontainer.size(); i++ {
|
||||
|
|
@ -695,7 +719,9 @@ func (rb *Bitmap) Rank(x uint32) uint64 {
|
|||
return size
|
||||
}
|
||||
|
||||
// Select returns the xth integer in the bitmap
|
||||
// Select returns the xth integer in the bitmap. If you pass 0, you get
|
||||
// the smallest element. Note that this function differs in convention from
|
||||
// the Rank function which returns 1 on the smallest value.
|
||||
func (rb *Bitmap) Select(x uint32) (uint32, error) {
|
||||
if rb.GetCardinality() <= uint64(x) {
|
||||
return 0, fmt.Errorf("can't find %dth integer in a bitmap with only %d items", x, rb.GetCardinality())
|
||||
|
|
@ -980,7 +1006,7 @@ main:
|
|||
}
|
||||
s2 = x2.highlowcontainer.getKeyAtIndex(pos2)
|
||||
} else {
|
||||
rb.highlowcontainer.replaceKeyAndContainerAtIndex(pos1, s1, rb.highlowcontainer.getWritableContainerAtIndex(pos1).ior(x2.highlowcontainer.getContainerAtIndex(pos2)), false)
|
||||
rb.highlowcontainer.replaceKeyAndContainerAtIndex(pos1, s1, rb.highlowcontainer.getUnionedWritableContainer(pos1, x2.highlowcontainer.getContainerAtIndex(pos2)), false)
|
||||
pos1++
|
||||
pos2++
|
||||
if (pos1 == length1) || (pos2 == length2) {
|
||||
|
|
|
|||
15
vendor/github.com/RoaringBitmap/roaring/roaringarray.go
generated
vendored
15
vendor/github.com/RoaringBitmap/roaring/roaringarray.go
generated
vendored
|
|
@ -328,6 +328,17 @@ func (ra *roaringArray) getFastContainerAtIndex(i int, needsWriteable bool) cont
|
|||
return c
|
||||
}
|
||||
|
||||
// getUnionedWritableContainer switches behavior for in-place Or
|
||||
// depending on whether the container requires a copy on write.
|
||||
// If it does using the non-inplace or() method leads to fewer allocations.
|
||||
func (ra *roaringArray) getUnionedWritableContainer(pos int, other container) container {
|
||||
if ra.needCopyOnWrite[pos] {
|
||||
return ra.getContainerAtIndex(pos).or(other)
|
||||
}
|
||||
return ra.getContainerAtIndex(pos).ior(other)
|
||||
|
||||
}
|
||||
|
||||
func (ra *roaringArray) getWritableContainerAtIndex(i int) container {
|
||||
if ra.needCopyOnWrite[i] {
|
||||
ra.containers[i] = ra.containers[i].clone()
|
||||
|
|
@ -491,11 +502,11 @@ func (ra *roaringArray) writeTo(w io.Writer) (n int64, err error) {
|
|||
binary.LittleEndian.PutUint16(buf[2:], uint16(len(ra.keys)-1))
|
||||
nw += 2
|
||||
// compute isRun bitmap without temporary allocation
|
||||
var runbitmapslice = buf[nw:nw+isRunSizeInBytes]
|
||||
var runbitmapslice = buf[nw : nw+isRunSizeInBytes]
|
||||
for i, c := range ra.containers {
|
||||
switch c.(type) {
|
||||
case *runContainer16:
|
||||
runbitmapslice[i / 8] |= 1<<(uint(i)%8)
|
||||
runbitmapslice[i/8] |= 1 << (uint(i) % 8)
|
||||
}
|
||||
}
|
||||
nw += isRunSizeInBytes
|
||||
|
|
|
|||
41
vendor/github.com/RoaringBitmap/roaring/runcontainer.go
generated
vendored
41
vendor/github.com/RoaringBitmap/roaring/runcontainer.go
generated
vendored
|
|
@ -1321,6 +1321,47 @@ func (ri *runIterator16) nextMany(hs uint32, buf []uint32) int {
|
|||
return n
|
||||
}
|
||||
|
||||
func (ri *runIterator16) nextMany64(hs uint64, buf []uint64) int {
|
||||
n := 0
|
||||
|
||||
if !ri.hasNext() {
|
||||
return n
|
||||
}
|
||||
|
||||
// start and end are inclusive
|
||||
for n < len(buf) {
|
||||
moreVals := 0
|
||||
|
||||
if ri.rc.iv[ri.curIndex].length >= ri.curPosInIndex {
|
||||
// add as many as you can from this seq
|
||||
moreVals = minOfInt(int(ri.rc.iv[ri.curIndex].length-ri.curPosInIndex)+1, len(buf)-n)
|
||||
base := uint64(ri.rc.iv[ri.curIndex].start+ri.curPosInIndex) | hs
|
||||
|
||||
// allows BCE
|
||||
buf2 := buf[n : n+moreVals]
|
||||
for i := range buf2 {
|
||||
buf2[i] = base + uint64(i)
|
||||
}
|
||||
|
||||
// update values
|
||||
n += moreVals
|
||||
}
|
||||
|
||||
if moreVals+int(ri.curPosInIndex) > int(ri.rc.iv[ri.curIndex].length) {
|
||||
ri.curPosInIndex = 0
|
||||
ri.curIndex++
|
||||
|
||||
if ri.curIndex == int64(len(ri.rc.iv)) {
|
||||
break
|
||||
}
|
||||
} else {
|
||||
ri.curPosInIndex += uint16(moreVals) //moreVals always fits in uint16
|
||||
}
|
||||
}
|
||||
|
||||
return n
|
||||
}
|
||||
|
||||
// remove removes key from the container.
|
||||
func (rc *runContainer16) removeKey(key uint16) (wasPresent bool) {
|
||||
|
||||
|
|
|
|||
2
vendor/github.com/RoaringBitmap/roaring/serialization_generic.go
generated
vendored
2
vendor/github.com/RoaringBitmap/roaring/serialization_generic.go
generated
vendored
|
|
@ -1,4 +1,4 @@
|
|||
// +build !amd64,!386 appengine
|
||||
// +build !amd64,!386,!arm,!arm64,!ppc64le,!mipsle,!mips64le,!mips64p32le,!wasm appengine
|
||||
|
||||
package roaring
|
||||
|
||||
|
|
|
|||
2
vendor/github.com/RoaringBitmap/roaring/serialization_littleendian.go
generated
vendored
2
vendor/github.com/RoaringBitmap/roaring/serialization_littleendian.go
generated
vendored
|
|
@ -1,4 +1,4 @@
|
|||
// +build 386 amd64,!appengine
|
||||
// +build 386,!appengine amd64,!appengine arm,!appengine arm64,!appengine ppc64le,!appengine mipsle,!appengine mips64le,!appengine mips64p32le,!appengine wasm,!appengine
|
||||
|
||||
package roaring
|
||||
|
||||
|
|
|
|||
60
vendor/github.com/RoaringBitmap/roaring/setutil.go
generated
vendored
60
vendor/github.com/RoaringBitmap/roaring/setutil.go
generated
vendored
|
|
@ -135,66 +135,6 @@ func exclusiveUnion2by2(set1 []uint16, set2 []uint16, buffer []uint16) int {
|
|||
return pos
|
||||
}
|
||||
|
||||
func union2by2(set1 []uint16, set2 []uint16, buffer []uint16) int {
|
||||
pos := 0
|
||||
k1 := 0
|
||||
k2 := 0
|
||||
if 0 == len(set2) {
|
||||
buffer = buffer[:len(set1)]
|
||||
copy(buffer, set1[:])
|
||||
return len(set1)
|
||||
}
|
||||
if 0 == len(set1) {
|
||||
buffer = buffer[:len(set2)]
|
||||
copy(buffer, set2[:])
|
||||
return len(set2)
|
||||
}
|
||||
s1 := set1[k1]
|
||||
s2 := set2[k2]
|
||||
buffer = buffer[:cap(buffer)]
|
||||
for {
|
||||
if s1 < s2 {
|
||||
buffer[pos] = s1
|
||||
pos++
|
||||
k1++
|
||||
if k1 >= len(set1) {
|
||||
copy(buffer[pos:], set2[k2:])
|
||||
pos += len(set2) - k2
|
||||
break
|
||||
}
|
||||
s1 = set1[k1]
|
||||
} else if s1 == s2 {
|
||||
buffer[pos] = s1
|
||||
pos++
|
||||
k1++
|
||||
k2++
|
||||
if k1 >= len(set1) {
|
||||
copy(buffer[pos:], set2[k2:])
|
||||
pos += len(set2) - k2
|
||||
break
|
||||
}
|
||||
if k2 >= len(set2) {
|
||||
copy(buffer[pos:], set1[k1:])
|
||||
pos += len(set1) - k1
|
||||
break
|
||||
}
|
||||
s1 = set1[k1]
|
||||
s2 = set2[k2]
|
||||
} else { // if (set1[k1]>set2[k2])
|
||||
buffer[pos] = s2
|
||||
pos++
|
||||
k2++
|
||||
if k2 >= len(set2) {
|
||||
copy(buffer[pos:], set1[k1:])
|
||||
pos += len(set1) - k1
|
||||
break
|
||||
}
|
||||
s2 = set2[k2]
|
||||
}
|
||||
}
|
||||
return pos
|
||||
}
|
||||
|
||||
func union2by2Cardinality(set1 []uint16, set2 []uint16) int {
|
||||
pos := 0
|
||||
k1 := 0
|
||||
|
|
|
|||
6
vendor/github.com/RoaringBitmap/roaring/setutil_arm64.go
generated
vendored
Normal file
6
vendor/github.com/RoaringBitmap/roaring/setutil_arm64.go
generated
vendored
Normal file
|
|
@ -0,0 +1,6 @@
|
|||
// +build arm64,!gccgo,!appengine
|
||||
|
||||
package roaring
|
||||
|
||||
//go:noescape
|
||||
func union2by2(set1 []uint16, set2 []uint16, buffer []uint16) (size int)
|
||||
132
vendor/github.com/RoaringBitmap/roaring/setutil_arm64.s
generated
vendored
Normal file
132
vendor/github.com/RoaringBitmap/roaring/setutil_arm64.s
generated
vendored
Normal file
|
|
@ -0,0 +1,132 @@
|
|||
// +build arm64,!gccgo,!appengine
|
||||
|
||||
#include "textflag.h"
|
||||
|
||||
|
||||
// This implements union2by2 using golang's version of arm64 assembly
|
||||
// The algorithm is very similar to the generic one,
|
||||
// but makes better use of arm64 features so is notably faster.
|
||||
// The basic algorithm structure is as follows:
|
||||
// 1. If either set is empty, copy the other set into the buffer and return the length
|
||||
// 2. Otherwise, load the first element of each set into a variable (s1 and s2).
|
||||
// 3. a. Compare the values of s1 and s2.
|
||||
// b. add the smaller one to the buffer.
|
||||
// c. perform a bounds check before incrementing.
|
||||
// If one set is finished, copy the rest of the other set over.
|
||||
// d. update s1 and or s2 to the next value, continue loop.
|
||||
//
|
||||
// Past the fact of the algorithm, this code makes use of several arm64 features
|
||||
// Condition Codes:
|
||||
// arm64's CMP operation sets 4 bits that can be used for branching,
|
||||
// rather than just true or false.
|
||||
// As a consequence, a single comparison gives enough information to distinguish the three cases
|
||||
//
|
||||
// Post-increment pointers after load/store:
|
||||
// Instructions like `MOVHU.P 2(R0), R6`
|
||||
// increment the register by a specified amount, in this example 2.
|
||||
// Because uint16's are exactly 2 bytes and the length of the slices
|
||||
// is part of the slice header,
|
||||
// there is no need to separately track the index into the slice.
|
||||
// Instead, the code can calculate the final read value and compare against that,
|
||||
// using the post-increment reads to move the pointers along.
|
||||
//
|
||||
// TODO: CALL out to memmove once the list is exhausted.
|
||||
// Right now it moves the necessary shorts so that the remaining count
|
||||
// is a multiple of 4 and then copies 64 bits at a time.
|
||||
|
||||
TEXT ·union2by2(SB), NOSPLIT, $0-80
|
||||
// R0, R1, and R2 for the pointers to the three slices
|
||||
MOVD set1+0(FP), R0
|
||||
MOVD set2+24(FP), R1
|
||||
MOVD buffer+48(FP), R2
|
||||
|
||||
//R3 and R4 will be the values at which we will have finished reading set1 and set2.
|
||||
// R3 should be R0 + 2 * set1_len+8(FP)
|
||||
MOVD set1_len+8(FP), R3
|
||||
MOVD set2_len+32(FP), R4
|
||||
|
||||
ADD R3<<1, R0, R3
|
||||
ADD R4<<1, R1, R4
|
||||
|
||||
|
||||
//Rather than counting the number of elements added separately
|
||||
//Save the starting register of buffer.
|
||||
MOVD buffer+48(FP), R5
|
||||
|
||||
// set1 is empty, just flush set2
|
||||
CMP R0, R3
|
||||
BEQ flush_right
|
||||
|
||||
// set2 is empty, just flush set1
|
||||
CMP R1, R4
|
||||
BEQ flush_left
|
||||
|
||||
// R6, R7 are the working space for s1 and s2
|
||||
MOVD ZR, R6
|
||||
MOVD ZR, R7
|
||||
|
||||
MOVHU.P 2(R0), R6
|
||||
MOVHU.P 2(R1), R7
|
||||
loop:
|
||||
|
||||
CMP R6, R7
|
||||
BEQ pop_both // R6 == R7
|
||||
BLS pop_right // R6 > R7
|
||||
//pop_left: // R6 < R7
|
||||
MOVHU.P R6, 2(R2)
|
||||
CMP R0, R3
|
||||
BEQ pop_then_flush_right
|
||||
MOVHU.P 2(R0), R6
|
||||
JMP loop
|
||||
pop_both:
|
||||
MOVHU.P R6, 2(R2) //could also use R7, since they are equal
|
||||
CMP R0, R3
|
||||
BEQ flush_right
|
||||
CMP R1, R4
|
||||
BEQ flush_left
|
||||
MOVHU.P 2(R0), R6
|
||||
MOVHU.P 2(R1), R7
|
||||
JMP loop
|
||||
pop_right:
|
||||
MOVHU.P R7, 2(R2)
|
||||
CMP R1, R4
|
||||
BEQ pop_then_flush_left
|
||||
MOVHU.P 2(R1), R7
|
||||
JMP loop
|
||||
|
||||
pop_then_flush_right:
|
||||
MOVHU.P R7, 2(R2)
|
||||
flush_right:
|
||||
MOVD R1, R0
|
||||
MOVD R4, R3
|
||||
JMP flush_left
|
||||
pop_then_flush_left:
|
||||
MOVHU.P R6, 2(R2)
|
||||
flush_left:
|
||||
CMP R0, R3
|
||||
BEQ return
|
||||
//figure out how many bytes to slough off. Must be a multiple of two
|
||||
SUB R0, R3, R4
|
||||
ANDS $6, R4
|
||||
BEQ long_flush //handles the 0 mod 8 case
|
||||
SUBS $4, R4, R4 // since possible values are 2, 4, 6, this splits evenly
|
||||
BLT pop_single // exactly the 2 case
|
||||
MOVW.P 4(R0), R6
|
||||
MOVW.P R6, 4(R2)
|
||||
BEQ long_flush // we're now aligned by 64 bits, as R4==4, otherwise 2 more
|
||||
pop_single:
|
||||
MOVHU.P 2(R0), R6
|
||||
MOVHU.P R6, 2(R2)
|
||||
long_flush:
|
||||
// at this point we know R3 - R0 is a multiple of 8.
|
||||
CMP R0, R3
|
||||
BEQ return
|
||||
MOVD.P 8(R0), R6
|
||||
MOVD.P R6, 8(R2)
|
||||
JMP long_flush
|
||||
return:
|
||||
// number of shorts written is (R5 - R2) >> 1
|
||||
SUB R5, R2
|
||||
LSR $1, R2, R2
|
||||
MOVD R2, size+72(FP)
|
||||
RET
|
||||
63
vendor/github.com/RoaringBitmap/roaring/setutil_generic.go
generated
vendored
Normal file
63
vendor/github.com/RoaringBitmap/roaring/setutil_generic.go
generated
vendored
Normal file
|
|
@ -0,0 +1,63 @@
|
|||
// +build !arm64 gccgo appengine
|
||||
|
||||
package roaring
|
||||
|
||||
func union2by2(set1 []uint16, set2 []uint16, buffer []uint16) int {
|
||||
pos := 0
|
||||
k1 := 0
|
||||
k2 := 0
|
||||
if 0 == len(set2) {
|
||||
buffer = buffer[:len(set1)]
|
||||
copy(buffer, set1[:])
|
||||
return len(set1)
|
||||
}
|
||||
if 0 == len(set1) {
|
||||
buffer = buffer[:len(set2)]
|
||||
copy(buffer, set2[:])
|
||||
return len(set2)
|
||||
}
|
||||
s1 := set1[k1]
|
||||
s2 := set2[k2]
|
||||
buffer = buffer[:cap(buffer)]
|
||||
for {
|
||||
if s1 < s2 {
|
||||
buffer[pos] = s1
|
||||
pos++
|
||||
k1++
|
||||
if k1 >= len(set1) {
|
||||
copy(buffer[pos:], set2[k2:])
|
||||
pos += len(set2) - k2
|
||||
break
|
||||
}
|
||||
s1 = set1[k1]
|
||||
} else if s1 == s2 {
|
||||
buffer[pos] = s1
|
||||
pos++
|
||||
k1++
|
||||
k2++
|
||||
if k1 >= len(set1) {
|
||||
copy(buffer[pos:], set2[k2:])
|
||||
pos += len(set2) - k2
|
||||
break
|
||||
}
|
||||
if k2 >= len(set2) {
|
||||
copy(buffer[pos:], set1[k1:])
|
||||
pos += len(set1) - k1
|
||||
break
|
||||
}
|
||||
s1 = set1[k1]
|
||||
s2 = set2[k2]
|
||||
} else { // if (set1[k1]>set2[k2])
|
||||
buffer[pos] = s2
|
||||
pos++
|
||||
k2++
|
||||
if k2 >= len(set2) {
|
||||
copy(buffer[pos:], set1[k1:])
|
||||
pos += len(set1) - k1
|
||||
break
|
||||
}
|
||||
s2 = set2[k2]
|
||||
}
|
||||
}
|
||||
return pos
|
||||
}
|
||||
5
vendor/github.com/RoaringBitmap/roaring/util.go
generated
vendored
5
vendor/github.com/RoaringBitmap/roaring/util.go
generated
vendored
|
|
@ -1,6 +1,7 @@
|
|||
package roaring
|
||||
|
||||
import (
|
||||
"math"
|
||||
"math/rand"
|
||||
"sort"
|
||||
)
|
||||
|
|
@ -15,7 +16,7 @@ const (
|
|||
noOffsetThreshold = 4
|
||||
|
||||
// MaxUint32 is the largest uint32 value.
|
||||
MaxUint32 = 4294967295
|
||||
MaxUint32 = math.MaxUint32
|
||||
|
||||
// MaxRange is One more than the maximum allowed bitmap bit index. For use as an upper
|
||||
// bound for ranges.
|
||||
|
|
@ -23,7 +24,7 @@ const (
|
|||
|
||||
// MaxUint16 is the largest 16 bit unsigned int.
|
||||
// This is the largest value an interval16 can store.
|
||||
MaxUint16 = 65535
|
||||
MaxUint16 = math.MaxUint16
|
||||
|
||||
// Compute wordSizeInBytes, the size of a word in bytes.
|
||||
_m = ^uint64(0)
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue