allocate fewer binaries

[ruslandoga]

👋

This PR makes Zstream allocate fewer binaries where possible.

Benchee shows 10% difference in memory usage in this (admittedly very synthetic) benchmark:

Benchee.run(
  %{
    "zip" => fn entries ->
      Stream.run(Zstream.zip(entries))
    end
  },
  inputs: %{
    "empty" =>
      for i <- 1..10 do
        Zstream.entry("empty_file_#{i}", [], coder: Zstream.Coder.Stored)
      end
  },
  memory_time: 1,
  profile_after: true
)

results on master

$ MIX_ENV=bench mix run bench/zip.exs
Compiling 2 files (.ex)
Operating System: macOS
CPU Information: Apple M2
Number of Available Cores: 8
Available memory: 8 GB
Elixir 1.17.0-rc.0
Erlang 27.0
JIT enabled: true

Benchmark suite executing with the following configuration:
warmup: 2 s
time: 5 s
memory time: 1 s
reduction time: 0 ns
parallel: 1
inputs: empty
Estimated total run time: 8 s

Benchmarking zip with input empty ...
Calculating statistics...
Formatting results...

##### With input empty #####
Name           ips        average  deviation         median         99th %
zip        43.56 K       22.96 μs    ±12.94%       22.25 μs       28.38 μs

Memory usage statistics:

Name    Memory usage
zip         55.37 KB

**All measurements for memory usage were the same**

Profiling zip with eprof...

Profile results of #PID<0.20947.0>
#                                                          CALLS     % TIME µS/CALL
Total                                                       2463 100.0  323    0.13
:zlib.close_nif/1                                              1  0.00    0    0.00
:zlib.close/1                                                  1  0.00    0    0.00
:zlib.open/0                                                   1  0.00    0    0.00
Zstream.Coder.Stored.init/1                                   10  0.00    0    0.00
Zstream.Coder.Stored.close/1                                  10  0.00    0    0.00
Zstream.Zip.State.entry_initial_state/0                       10  0.00    0    0.00
Zstream.EncryptionCoder.None.encode/2                         10  0.00    0    0.00
Zstream.EncryptionCoder.None.close/1                          10  0.00    0    0.00
Enum.reverse/1                                                 1  0.00    0    0.00
Enum.map/2                                                     1  0.00    0    0.00
Zstream.Protocol.zip64_end_of_central_directory_record/4       1  0.00    0    0.00
Zstream.Protocol.zip64_end_of_central_directory_locator/2      1  0.00    0    0.00
Zstream.Protocol.external_file_attributes/0                   10  0.00    0    0.00
Zstream.Protocol.end_of_central_directory/4                    1  0.00    0    0.00
anonymous fn/1 in Stream.concat/1                              3  0.00    0    0.00
anonymous fn/3 in Stream.do_transform/7                       13  0.00    0    0.00
anonymous fn/3 in Stream.do_transform_result/6                11  0.00    0    0.00
anonymous fn/6 in Stream.transform/4                           1  0.00    0    0.00
Stream.transform/4                                             1  0.00    0    0.00
Stream.run/1                                                   1  0.00    0    0.00
Stream.flat_map/2                                             12  0.00    0    0.00
Stream.concat/2                                               10  0.00    0    0.00
Stream.concat/1                                                1  0.00    0    0.00
Zstream.zip/2                                                  1  0.00    0    0.00
Zstream.zip/1                                                  1  0.00    0    0.00
anonymous fn/1 in Zstream.Zip.close_entry/1                   10  0.00    0    0.00
anonymous fn/1 in Zstream.Zip.close_entry/1                   10  0.00    0    0.00
anonymous fn/1 in Zstream.Zip.close_entry/1                   10  0.00    0    0.00
anonymous fn/1 in Zstream.Zip.close_entry/1                   10  0.00    0    0.00
anonymous fn/2 in Zstream.Zip.close_entry/1                   10  0.00    0    0.00
anonymous fn/2 in Zstream.Zip.close_entry/1                   10  0.00    0    0.00
anonymous fn/2 in Zstream.Zip.close_entry/1                   10  0.00    0    0.00
anonymous fn/1 in Zstream.Zip.close_entry/1                   10  0.00    0    0.00
anonymous fn/2 in Zstream.Zip.construct/2                     10  0.00    0    0.00
anonymous fn/2 in Zstream.Zip.construct/2                     10  0.00    0    0.00
anonymous fn/2 in Zstream.Zip.construct/2                     10  0.00    0    0.00
anonymous fn/2 in Zstream.Zip.construct/2                     10  0.00    0    0.00
anonymous fn/1 in Zstream.Zip.construct/2                     10  0.00    0    0.00
anonymous fn/2 in Zstream.Zip.construct/2                     10  0.00    0    0.00
anonymous fn/2 in Zstream.Zip.construct/2                     10  0.00    0    0.00
anonymous fn/1 in Zstream.Zip.construct/2                      1  0.00    0    0.00
anonymous fn/1 in Zstream.Zip.construct/2                      1  0.00    0    0.00
anonymous fn/0 in Zstream.Zip.zip/2                            1  0.00    0    0.00
Zstream.Zip.zip/2                                              1  0.00    0    0.00
Zstream.Zip.free_resource/1                                    1  0.00    0    0.00
anonymous fn/2 in Benchee.Benchmark.Runner.main_function/2     1  0.00    0    0.00
Enumerable.Function.reduce/3                                  13  0.00    0    0.00
anonymous fn/1 in :elixir_compiler_3.__FILE__/1                1  0.00    0    0.00
:lists.reverse/2                                               1  0.00    0    0.00
:erlang.++/2                                                  10  0.31    1    0.10
:erlang.apply/2                                                1  0.31    1    1.00
:erlang.crc32/1                                               10  0.31    1    0.10
Zstream.Coder.Stored.compression_method/0                     20  0.31    1    0.05
Zstream.EncryptionCoder.None.init/1                           10  0.31    1    0.10
Zstream.EncryptionCoder.None.general_purpose_flag/0           20  0.31    1    0.05
Zstream.Protocol.data_descriptor/4                            10  0.31    1    0.10
anonymous fn/1 in Stream.concat/2                             20  0.31    1    0.05
anonymous fn/7 in Stream.do_transform_inner_enum/7            20  0.31    1    0.05
anonymous fn/7 in Stream.do_transform_inner_list/7            12  0.31    1    0.08
anonymous fn/3 in Stream.do_transform_result/6                23  0.31    1    0.04
anonymous fn/2 in Stream.run/1                                15  0.31    1    0.07
anonymous fn/1 in Stream.transform/3                          12  0.31    1    0.08
anonymous fn/1 in Stream.transform/3                          12  0.31    1    0.08
anonymous fn/5 in Stream.transform/3                          12  0.31    1    0.08
Stream.transform/3                                            12  0.31    1    0.08
Keyword.put/3                                                 10  0.31    1    0.10
Keyword.keyword?/1                                            20  0.31    1    0.05
anonymous fn/2 in Zstream.Zip.close_entry/1                   10  0.31    1    0.10
anonymous fn/3 in Zstream.Zip.construct/2                     10  0.31    1    0.10
anonymous fn/3 in Zstream.Zip.construct/2                     10  0.31    1    0.10
anonymous fn/2 in Zstream.Zip.construct/2                     10  0.31    1    0.10
anonymous fn/1 in Zstream.Zip.construct/2                     10  0.31    1    0.10
anonymous fn/2 in Zstream.Zip.zip/2                           10  0.31    1    0.10
Zstream.Zip.Extra.zip64_extended_info/3                       20  0.31    1    0.05
:zlib.open_nif/0                                               1  0.62    2    2.00
Enum."-map/2-lists^map/1-1-"/2                                11  0.62    2    0.18
Zstream.Protocol.dos_time/1                                   20  0.62    2    0.10
Zstream.Protocol.dos_date/1                                   20  0.62    2    0.10
Stream.do_transform_each/3                                    20  0.62    2    0.10
Stream.do_transform/7                                         13  0.62    2    0.15
anonymous fn/2 in Zstream.Zip.close_entry/1                   10  0.62    2    0.20
Enumerable.impl_for/1                                         25  0.62    2    0.08
:maps.merge/2                                                 10  0.93    3    0.30
Zstream.Protocol.compression_method/1                         20  0.93    3    0.15
Stream.do_transform_step/2                                    45  0.93    3    0.07
anonymous fn/3 in Keyword.merge/2                             30  0.93    3    0.10
Keyword.merge/2                                               11  0.93    3    0.27
anonymous fn/3 in Enumerable.List.reduce/3                    45  0.93    3    0.07
Enumerable.impl_for!/1                                        25  0.93    3    0.12
Zstream.Protocol.general_purpose_bit_flag/1                   20  1.24    4    0.20
anonymous fn/3 in Stream.do_transform/7                       20  1.24    4    0.20
anonymous fn/3 in Stream.flat_map/2                           33  1.24    4    0.12
Stream.do_transform_inner_enum/7                              31  1.24    4    0.13
Keyword.has_key?/2                                            30  1.24    4    0.13
Enumerable.reduce/3                                           25  1.24    4    0.16
:lists.reverse/1                                              58  1.24    4    0.07
Stream.do_transform_inner_list/7                              35  1.55    5    0.14
:lists.keymember/3                                            40  1.55    5    0.13
Stream.do_transform_result/6                                  45  1.86    6    0.13
Zstream.Protocol.local_file_header/3                          10  2.17    7    0.70
Keyword."-merge/2-lists^filter/1-0-"/2                        40  2.17    7    0.18
pick 0a7b2a6 allocate fewer binaries
:erlang.iolist_size/1                                         53  2.48    8    0.15
anonymous fn/2 in Zstream.Zip.construct/2                     10  2.48    8    0.80
Stream.do_transform_user/6                                   103  2.79    9    0.09
Zstream.Protocol.zip64?/3                                     86  3.41   11    0.13
Zstream.Zip.construct/2                                       12  4.02   13    1.08
Zstream.Zip.close_entry/1                                     11  4.02   13    1.18
Stream.do_transform/5                                         84  4.33   14    0.17
Zstream.Protocol.central_directory_header/1                   10  4.64   15    1.50
:lists.keyfind/3                                             208  5.26   17    0.08
Enumerable.List.reduce/3                                     140  6.81   22    0.16
Keyword.fetch!/2                                             208  9.91   32    0.15
Map.update!/3                                                222 12.69   41    0.18

Profile done over 113 matching functions

results on this branch

$ MIX_ENV=bench mix run bench/zip.exs
Operating System: macOS
CPU Information: Apple M2
Number of Available Cores: 8
Available memory: 8 GB
Elixir 1.17.0-rc.0
Erlang 27.0
JIT enabled: true

Benchmark suite executing with the following configuration:
warmup: 2 s
time: 5 s
memory time: 1 s
reduction time: 0 ns
parallel: 1
inputs: empty
Estimated total run time: 8 s

Benchmarking zip with input empty ...
Calculating statistics...
Formatting results...

##### With input empty #####
Name           ips        average  deviation         median         99th %
zip        47.05 K       21.26 μs    ±19.71%       20.58 μs       27.46 μs

Memory usage statistics:

Name    Memory usage
zip         44.61 KB

**All measurements for memory usage were the same**

Profiling zip with eprof...

Profile results of #PID<0.29872.0>
#                                                          CALLS     % TIME µS/CALL
Total                                                       2463 100.0  302    0.12
:lists.reverse/2                                               1  0.00    0    0.00
:zlib.close_nif/1                                              1  0.00    0    0.00
:zlib.close/1                                                  1  0.00    0    0.00
:zlib.open/0                                                   1  0.00    0    0.00
Zstream.Coder.Stored.init/1                                   10  0.00    0    0.00
Zstream.Coder.Stored.close/1                                  10  0.00    0    0.00
Zstream.Zip.State.entry_initial_state/0                       10  0.00    0    0.00
Zstream.EncryptionCoder.None.init/1                           10  0.00    0    0.00
Zstream.EncryptionCoder.None.close/1                          10  0.00    0    0.00
Enum.reverse/1                                                 1  0.00    0    0.00
Enum.map/2                                                     1  0.00    0    0.00
Zstream.Protocol.zip64_end_of_central_directory_record/4       1  0.00    0    0.00
Zstream.Protocol.zip64_end_of_central_directory_locator/2      1  0.00    0    0.00
Zstream.Protocol.end_of_central_directory/4                    1  0.00    0    0.00
anonymous fn/1 in Stream.concat/1                              3  0.00    0    0.00
anonymous fn/7 in Stream.do_transform_inner_list/7            12  0.00    0    0.00
anonymous fn/3 in Stream.do_transform_result/6                11  0.00    0    0.00
anonymous fn/1 in Stream.transform/3                          12  0.00    0    0.00
anonymous fn/1 in Stream.transform/3                          12  0.00    0    0.00
anonymous fn/5 in Stream.transform/3                          12  0.00    0    0.00
anonymous fn/6 in Stream.transform/4                           1  0.00    0    0.00
Stream.transform/4                                             1  0.00    0    0.00
Stream.transform/3                                            12  0.00    0    0.00
Stream.run/1                                                   1  0.00    0    0.00
Stream.concat/2                                               10  0.00    0    0.00
Stream.concat/1                                                1  0.00    0    0.00
Zstream.zip/2                                                  1  0.00    0    0.00
Zstream.zip/1                                                  1  0.00    0    0.00
anonymous fn/1 in Zstream.Zip.close_entry/1                   10  0.00    0    0.00
anonymous fn/1 in Zstream.Zip.close_entry/1                   10  0.00    0    0.00
anonymous fn/1 in Zstream.Zip.close_entry/1                   10  0.00    0    0.00
anonymous fn/1 in Zstream.Zip.close_entry/1                   10  0.00    0    0.00
anonymous fn/2 in Zstream.Zip.close_entry/1                   10  0.00    0    0.00
anonymous fn/2 in Zstream.Zip.close_entry/1                   10  0.00    0    0.00
anonymous fn/2 in Zstream.Zip.close_entry/1                   10  0.00    0    0.00
anonymous fn/2 in Zstream.Zip.construct/2                     10  0.00    0    0.00
anonymous fn/2 in Zstream.Zip.construct/2                     10  0.00    0    0.00
anonymous fn/2 in Zstream.Zip.construct/2                     10  0.00    0    0.00
anonymous fn/2 in Zstream.Zip.construct/2                     10  0.00    0    0.00
anonymous fn/2 in Zstream.Zip.construct/2                     10  0.00    0    0.00
anonymous fn/2 in Zstream.Zip.construct/2                     10  0.00    0    0.00
anonymous fn/1 in Zstream.Zip.construct/2                      1  0.00    0    0.00
anonymous fn/1 in Zstream.Zip.construct/2                      1  0.00    0    0.00
anonymous fn/0 in Zstream.Zip.zip/2                            1  0.00    0    0.00
Zstream.Zip.zip/2                                              1  0.00    0    0.00
Zstream.Zip.free_resource/1                                    1  0.00    0    0.00
anonymous fn/2 in Benchee.Benchmark.Runner.main_function/2     1  0.00    0    0.00
Enumerable.Function.reduce/3                                  13  0.00    0    0.00
anonymous fn/1 in :elixir_compiler_2.__FILE__/1                1  0.00    0    0.00
:erlang.++/2                                                  10  0.33    1    0.10
:erlang.apply/2                                                1  0.33    1    1.00
:erlang.crc32/1                                               10  0.33    1    0.10
:maps.merge/2                                                 10  0.33    1    0.10
Zstream.Coder.Stored.compression_method/0                     20  0.33    1    0.05
Zstream.EncryptionCoder.None.general_purpose_flag/0           20  0.33    1    0.05
Zstream.Protocol.external_file_attributes/0                   10  0.33    1    0.10
Zstream.Protocol.dos_date/1                                   20  0.33    1    0.05
Zstream.Protocol.data_descriptor/4                            10  0.33    1    0.10
anonymous fn/1 in Stream.concat/2                             20  0.33    1    0.05
anonymous fn/3 in Stream.do_transform/7                       20  0.33    1    0.05
anonymous fn/3 in Stream.do_transform/7                       13  0.33    1    0.08
anonymous fn/7 in Stream.do_transform_inner_enum/7            20  0.33    1    0.05
anonymous fn/3 in Stream.do_transform_result/6                23  0.33    1    0.04
anonymous fn/2 in Stream.run/1                                15  0.33    1    0.07
Stream.flat_map/2                                             12  0.33    1    0.08
Keyword.put/3                                                 10  0.33    1    0.10
Keyword.keyword?/1                                            20  0.33    1    0.05
anonymous fn/2 in Zstream.Zip.close_entry/1                   10  0.33    1    0.10
anonymous fn/2 in Zstream.Zip.close_entry/1                   10  0.33    1    0.10
anonymous fn/1 in Zstream.Zip.close_entry/1                   10  0.33    1    0.10
anonymous fn/3 in Zstream.Zip.construct/2                     10  0.33    1    0.10
anonymous fn/2 in Zstream.Zip.construct/2                     10  0.33    1    0.10
anonymous fn/1 in Zstream.Zip.construct/2                     10  0.33    1    0.10
anonymous fn/1 in Zstream.Zip.construct/2                     10  0.33    1    0.10
anonymous fn/2 in Zstream.Zip.construct/2                     10  0.33    1    0.10
Zstream.Zip.Extra.zip64_extended_info/3                       20  0.33    1    0.05
Enumerable.impl_for/1                                         25  0.33    1    0.04
Zstream.EncryptionCoder.None.encode/2                         10  0.66    2    0.20
Enum."-map/2-lists^map/1-1-"/2                                11  0.66    2    0.18
Zstream.Protocol.dos_time/1                                   20  0.66    2    0.10
Stream.do_transform_step/2                                    45  0.66    2    0.04
Stream.do_transform_each/3                                    20  0.66    2    0.10
Stream.do_transform/7                                         13  0.66    2    0.15
Keyword.merge/2                                               11  0.66    2    0.18
anonymous fn/2 in Zstream.Zip.zip/2                           10  0.66    2    0.20
anonymous fn/3 in Enumerable.List.reduce/3                    45  0.66    2    0.04
:lists.keymember/3                                            40  0.99    3    0.08
:zlib.open_nif/0                                               1  0.99    3    3.00
Zstream.Protocol.compression_method/1                         20  0.99    3    0.15
Stream.do_transform_result/6                                  45  0.99    3    0.07
anonymous fn/3 in Zstream.Zip.construct/2                     10  0.99    3    0.30
Enumerable.impl_for!/1                                        25  0.99    3    0.12
:lists.reverse/1                                              58  1.32    4    0.07
Zstream.Protocol.general_purpose_bit_flag/1                   20  1.32    4    0.20
anonymous fn/3 in Stream.flat_map/2                           33  1.32    4    0.12
anonymous fn/3 in Keyword.merge/2                             30  1.32    4    0.13
Keyword.has_key?/2                                            30  1.32    4    0.13
Enumerable.reduce/3                                           25  1.32    4    0.16
Stream.do_transform_inner_list/7                              35  1.66    5    0.14
:erlang.iolist_size/1                                         53  1.99    6    0.11
Zstream.Protocol.local_file_header/3                          10  2.32    7    0.70
Keyword."-merge/2-lists^filter/1-0-"/2                        40  2.98    9    0.23
Zstream.Protocol.central_directory_header/1                   10  3.31   10    1.00
Stream.do_transform_user/6                                   103  3.31   10    0.10
Stream.do_transform_inner_enum/7                              31  3.64   11    0.35
Zstream.Protocol.zip64?/3                                     86  3.97   12    0.14
Zstream.Zip.construct/2                                       12  3.97   12    1.00
Zstream.Zip.close_entry/1                                     11  3.97   12    1.09
Stream.do_transform/5                                         84  4.64   14    0.17
:lists.keyfind/3                                             208  6.29   19    0.09
Enumerable.List.reduce/3                                     140  6.62   20    0.14
Keyword.fetch!/2                                             208  8.94   27    0.13
Map.update!/3                                                222 13.25   40    0.18

Profile done over 113 matching functions

[ananthakumaran]

I will take a look the CI failure

[ananthakumaran]

Could you rebase, I have fixed the CI on master