Skip to content

perf: reduce allocation in serializing for frequency sketch - #169

Merged
proost merged 1 commit into
apache:mainfrom
proost:perf-reduce-useless-allocation-serializing
Aug 31, 2026
Merged

perf: reduce allocation in serializing for frequency sketch#169
proost merged 1 commit into
apache:mainfrom
proost:perf-reduce-useless-allocation-serializing

Conversation

@proost

@proost proost commented Aug 29, 2026

Copy link
Copy Markdown
Member

During serialization, current implementation allocates same active items multiple times. It makes huge overhead because of heap allocation.

See the benchmark:

Current main branch:

goos: darwin
goarch: arm64
pkg: github.com/apache/datasketches-go/frequencies
cpu: Apple M1 Pro
BenchmarkItemsSketchToSlice
BenchmarkItemsSketchToSlice/int64/items=48
BenchmarkItemsSketchToSlice/int64/items=48-10         	  208099	      6027 ns/op	 132.73 MB/s	   20096 B/op	      51 allocs/op
BenchmarkItemsSketchToSlice/int64/items=192
BenchmarkItemsSketchToSlice/int64/items=192-10        	   14924	     80697 ns/op	  38.46 MB/s	  301185 B/op	     195 allocs/op
BenchmarkItemsSketchToSlice/int64/items=768
BenchmarkItemsSketchToSlice/int64/items=768-10        	    1072	   1082348 ns/op	  11.38 MB/s	 4744468 B/op	     771 allocs/op
BenchmarkItemsSketchToSlice/string/items=48
BenchmarkItemsSketchToSlice/string/items=48-10        	  200163	      5586 ns/op	 167.22 MB/s	   20928 B/op	      51 allocs/op
BenchmarkItemsSketchToSlice/string/items=192
BenchmarkItemsSketchToSlice/string/items=192-10       	   15355	     75696 ns/op	  49.70 MB/s	  304512 B/op	     195 allocs/op
BenchmarkItemsSketchToSlice/string/items=768
BenchmarkItemsSketchToSlice/string/items=768-10       	    1129	   1072233 ns/op	  14.25 MB/s	 4758036 B/op	     771 allocs/op

Change:

goos: darwin
goarch: arm64
pkg: github.com/apache/datasketches-go/frequencies
cpu: Apple M1 Pro
BenchmarkItemsSketchToSlice
BenchmarkItemsSketchToSlice/int64/items=48
BenchmarkItemsSketchToSlice/int64/items=48-10         	 2943316	       401.4 ns/op	1992.87 MB/s	    1664 B/op	       3 allocs/op
BenchmarkItemsSketchToSlice/int64/items=192
BenchmarkItemsSketchToSlice/int64/items=192-10        	  865233	      1399 ns/op	2218.19 MB/s	    6272 B/op	       3 allocs/op
BenchmarkItemsSketchToSlice/int64/items=768
BenchmarkItemsSketchToSlice/int64/items=768-10        	  211227	      5016 ns/op	2456.31 MB/s	   25856 B/op	       3 allocs/op
BenchmarkItemsSketchToSlice/string/items=48
BenchmarkItemsSketchToSlice/string/items=48-10        	 2000376	       599.6 ns/op	1557.73 MB/s	    2496 B/op	       3 allocs/op
BenchmarkItemsSketchToSlice/string/items=192
BenchmarkItemsSketchToSlice/string/items=192-10       	  575967	      2041 ns/op	1843.51 MB/s	    9600 B/op	       3 allocs/op
BenchmarkItemsSketchToSlice/string/items=768
BenchmarkItemsSketchToSlice/string/items=768-10       	  145569	      9110 ns/op	1677.53 MB/s	   39424 B/op	       3 allocs/op
PASS

@proost proost self-assigned this Aug 29, 2026
@proost proost changed the title perf: reduce allocation in serializing for frequency sketch. perf: reduce allocation in serializing for frequency sketch Aug 29, 2026
@proost
proost merged commit a01e78c into apache:main Aug 31, 2026
4 of 5 checks passed
@proost
proost deleted the perf-reduce-useless-allocation-serializing branch August 31, 2026 07:23
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant