www.fabiankeil.de/gehacktes/privoxy-tls-benchmarks/

Privoxy TLS benchmarks

Privoxy supports multiple TLS libraries for HTTPS inspection which lets Privoxy filter encrypted requests and responses.

On this page I'm collecting benchmarks to see how the implementations compare and to make sure Privoxy's performance does not regress.

The test setup currently isn't ideal as I have no dedicated benchmark hardware but it will do for now.

I'm running Privoxy and the system running the benchmarks in bhyve VMs. All systems including the host run ElectroBSD with the ELECTRO_BLOAT kernel configuration.

The host system has two cpu cores:

CPU: Intel(R) Pentium(R) CPU        G6950  @ 2.80GHz (2793.05-MHz K8-class CPU)
  Origin="GenuineIntel"  Id=0x20655  Family=0x6  Model=0x25  Stepping=5
  Features=0xbfebfbff<FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CLFLUSH,DTS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE>
  Features2=0x82e3bd<SSE3,DTES64,MON,DS_CPL,VMX,EST,TM2,SSSE3,CX16,xTPR,PDCM,PCID,POPCNT>
  AMD Features=0x28100800<SYSCALL,NX,RDTSCP,LM>
  AMD Features2=0x1<LAHF>
  VT-x: PAT,HLT,MTF,PAUSE,EPT,UG,VPID
  TSC: P-state invariant, performance statistics

Note that there's no AESNI support which is unfortunate as some of the TLS libraries could use it.

The VMs are configured to use a single core with 2 GB of memory. The host is also doing other tasks.

I started benchmarking with ElectroBSD 11.4-STABLE and upgraded to ElectroBSD 12.3-STABLE after Privoxy 3.0.33 was released.

Privoxy already supports OpenSSL, LibreSSL and mbedtls and I'm currently working on adding wolfSSL support. The work will be funded using donations made to the Privoxy project. You can donate to the Privoxy project to support this.

Privoxy 3.0.32 experimental 2021-02-16 679b708984 tested with ab-proxy

The Privoxy versions tested contain experimental patches that haven't been published yet.

Connection: keep-alive, concurrency level 1000

This benchmark requests the ElectroBSD homepage reusing connections for multiple requests, thus doing less handshakes. The concurrency level of 1000 results in a fair amount of parallel connections. The connections made by ab-proxy don't seem to be reused fairly, though.

privoxy-experimental 3.0.32.20210216 with OpenSSL 1.0.2t-ElectroBSD from base

[fk@benchmark-vm ~]$ go/bin/ab-proxy -s 1000 -c 1000 -n 10000 --bursts 3 --proxy http://172.16.1.7:8118/ --show-errors -H "Connection: keep-alive" https://www.electrobsd.org/
Benchmarking 'https://www.electrobsd.org/' using proxy 'http://172.16.1.7:8118/' with a total of 30000 GET requests:

Number of bursts:             3
Number of request per burst   10000
Concurrency level:            1000
Time taken for tests:         4m2.844750331s

Total initiated requests:     30000
   Completed requests:        30000
      HTTP-200 completed:     29999
      HTTP-503 completed:     1
   Failed requests:           0

Total transferred:            15306525 bytes
Requests per second:          123.536
Time per request:             8.094825ms

privoxy-experimental-libressl 3.0.32.20210216 with LibreSSL 3.2.3_2

[fk@benchmark-vm ~]$ go/bin/ab-proxy -s 1000 -c 1000 -n 10000 --bursts 3 --proxy http://172.16.1.7:8118/ --show-errors -H "Connection: keep-alive" https://www.electrobsd.org/
Benchmarking 'https://www.electrobsd.org/' using proxy 'http://172.16.1.7:8118/' with a total of 30000 GET requests:

Number of bursts:             3
Number of request per burst   10000
Concurrency level:            1000
Time taken for tests:         3m57.554025599s

Total initiated requests:     30000
   Completed requests:        30000
      HTTP-200 completed:     30000
   Failed requests:           0

Total transferred:            15300000 bytes
Requests per second:          126.287
Time per request:             7.918467ms

privoxy-experimental-libressl-devel 3.0.32.20210216 with LibreSSL devel 3.3.1

[fk@benchmark-vm ~]$ go/bin/ab-proxy -s 1000 -c 1000 -n 10000 --bursts 3 --proxy http://172.16.1.7:8118/ --show-errors -H "Connection: keep-alive" https://www.electrobsd.org/
Benchmarking 'https://www.electrobsd.org/' using proxy 'http://172.16.1.7:8118/' with a total of 30000 GET requests:

Number of bursts:             3
Number of request per burst   10000
Concurrency level:            1000
Time taken for tests:         3m42.1314853s

Total initiated requests:     30000
   Completed requests:        30000
      HTTP-200 completed:     30000
   Failed requests:           0

Total transferred:            15300000 bytes
Requests per second:          135.055
Time per request:             7.404382ms

privoxy-experimental-ports-openssl 3.0.32.20210216 with OpenSSL 1.1.1i,1

[fk@benchmark-vm ~]$ go/bin/ab-proxy -s 1000 -c 1000 -n 10000 --bursts 3 --proxy http://172.16.1.7:8118/ --show-errors -H "Connection: keep-alive" https://www.electrobsd.org/
Benchmarking 'https://www.electrobsd.org/' using proxy 'http://172.16.1.7:8118/' with a total of 30000 GET requests:

Number of bursts:             3
Number of request per burst   10000
Concurrency level:            1000
Time taken for tests:         11m0.781294501s

Total initiated requests:     30000
   Completed requests:        29997
      HTTP-200 completed:     29988
      HTTP-502 completed:     5
      HTTP-503 completed:     4
   Failed requests:           3
      Timeout failures:       3

Total transferred:            15360060 bytes
Requests per second:          45.401
Time per request:             22.026043ms

Errors:
 3x  Get "https://www.electrobsd.org/": proxyconnect tcp: dial tcp 172.16.1.7:8118: connect: operation timed out

privoxy-experimental-ports-openssl-devel 3.0.32.20210216 with OpenSSL devel 3.0.0.a11

[fk@benchmark-vm ~]$ go/bin/ab-proxy -s 1000 -c 1000 -n 10000 --bursts 3 --proxy http://172.16.1.7:8118/ --show-errors -H "Connection: keep-alive" https://www.electrobsd.org/
Benchmarking 'https://www.electrobsd.org/' using proxy 'http://172.16.1.7:8118/' with a total of 30000 GET requests:

Number of bursts:             3
Number of request per burst   10000
Concurrency level:            1000
Time taken for tests:         8m2.315674827s

Total initiated requests:     30000
   Completed requests:        29080
      HTTP-200 completed:     29080
   Failed requests:           920
      Timeout failures:       806

Total transferred:            14830800 bytes
Requests per second:          62.200
Time per request:             16.077189ms

Errors:
 806x  Get "https://www.electrobsd.org/": proxyconnect tcp: dial tcp 172.16.1.7:8118: connect: operation timed out
 114x  Get "https://www.electrobsd.org/": Too many open connections

privoxy-experimental-wolfssl 3.0.32.20210216 with wolfSSL 4.7.0_1

[fk@benchmark-vm ~]$ go/bin/ab-proxy -s 1000 -c 1000 -n 10000 --bursts 3 --proxy http://172.16.1.7:8118/ --show-errors -H "Connection: keep-alive" https://www.electrobsd.org/
Benchmarking 'https://www.electrobsd.org/' using proxy 'http://172.16.1.7:8118/' with a total of 30000 GET requests:

Number of bursts:             3
Number of request per burst   10000
Concurrency level:            1000
Time taken for tests:         4m37.326142464s

Total initiated requests:     30000
   Completed requests:        30000
      HTTP-200 completed:     30000
   Failed requests:           0

Total transferred:            15300000 bytes
Requests per second:          108.176
Time per request:             9.244204ms

privoxy-experimental-mbedtls 3.0.32.20210216 with mbedtls 2.16.9_6

[fk@benchmark-vm ~]$ go/bin/ab-proxy -s 1000 -c 1000 -n 10000 --bursts 3 --proxy http://172.16.1.7:8118/ --show-errors -H "Connection: keep-alive" https://www.electrobsd.org/
Benchmarking 'https://www.electrobsd.org/' using proxy 'http://172.16.1.7:8118/' with a total of 30000 GET requests:

Number of bursts:             3
Number of request per burst   10000
Concurrency level:            1000
Time taken for tests:         12m25.87273415s

Total initiated requests:     30000
   Completed requests:        28375
      HTTP-200 completed:     28375
   Failed requests:           1625

Total transferred:            14471250 bytes
Requests per second:          40.221
Time per request:             24.862424ms

Errors:
 1625x  Get "https://www.electrobsd.org/": Too many open connections

Privoxy CPU time and resident memory use according to top after the test

TLS libraryCPU timeMemory size
OpenSSL 1.0.2t3:08692M
LibreSSL 3.2.3_23:12713M
LibreSSL devel 3.3.12:57690M
OpenSSL 1.1.1i,19:50788M
OpenSSL devel 3.0.0.a116:58603M
wolfSSL 4.7.0_13:22828M
mbedtls 2.16.9_68:32510M

Cipher suites and TLS versions used

Note that the ElectroBSD.org server does not support TLS 1.3 yet so the libraries that support it can't use it.

TLS libraryClient side TLS version and cipher suiteServer side TLS version and cipher suite
OpenSSL 1.0.2tTLSv1.2 AES128-GCM-SHA256TLSv1.2 ECDHE-RSA-AES128-GCM-SHA256
LibreSSL 3.2.3_2TLSv1.3 AEAD-CHACHA20-POLY1305-SHA256TLSv1.2 ECDHE-RSA-AES128-GCM-SHA256
LibreSSL devel 3.3.1TLSv1.3 AEAD-CHACHA20-POLY1305-SHA256TLSv1.2 ECDHE-RSA-AES128-GCM-SHA256
OpenSSL 1.1.1i,1TLSv1.3 TLS_CHACHA20_POLY1305_SHA256TLSv1.2 ECDHE-RSA-AES128-GCM-SHA256
OpenSSL devel 3.0.0.a11TLSv1.3 TLS_CHACHA20_POLY1305_SHA256TLSv1.2 ECDHE-RSA-AES128-GCM-SHA256
wolfSSL 4.7.0_1TLSv1.3 TLS13-AES128-GCM-SHA256TLSv1.2 ECDHE-RSA-AES128-GCM-SHA256
mbedtls 2.16.9_6TLSv1.2 TLS-ECDHE-RSA-WITH-CHACHA20-POLY1305-SHA256TLSv1.2 TLS-ECDHE-RSA-WITH-AES-128-GCM-SHA256

Connection: close, concurrency level 1000

This benchmark requests the ElectroBSD homepage without reusing connections, thus requiring lots of handshakes.

privoxy-experimental 3.0.32.20210216 with OpenSSL 1.0.2t-ElectroBSD from base

[fk@benchmark-vm ~]$ go/bin/ab-proxy -s 1000 -c 1000 -n 10000 --bursts 3 --proxy http://172.16.1.7:8118/ --show-errors -H "Connection: close" https://www.electrobsd.org/
Benchmarking 'https://www.electrobsd.org/' using proxy 'http://172.16.1.7:8118/' with a total of 30000 GET requests:

Number of bursts:             3
Number of request per burst   10000
Concurrency level:            1000
Time taken for tests:         21m16.715068967s

Total initiated requests:     30000
   Completed requests:        29719
      HTTP-200 completed:     29270
      HTTP-503 completed:     449
   Failed requests:           281
      Timeout failures:       281

Total transferred:            18086415 bytes
Requests per second:          23.498
Time per request:             42.557168ms
Errors:
 281x  Get "https://www.electrobsd.org/": proxyconnect tcp: dial tcp 172.16.1.7:8118: connect: operation timed out

privoxy-experimental-libressl 3.0.32.20210216 with LibreSSL 3.2.3_2

[fk@benchmark-vm ~]$ go/bin/ab-proxy -s 1000 -c 1000 -n 10000 --bursts 3 --proxy http://172.16.1.7:8118/ --show-errors -H "Connection: close" https://www.electrobsd.org/
Benchmarking 'https://www.electrobsd.org/' using proxy 'http://172.16.1.7:8118/' with a total of 30000 GET requests:

Number of bursts:             3
Number of request per burst   10000
Concurrency level:            1000
Time taken for tests:         23m27.798118651s

Total initiated requests:     30000
   Completed requests:        29668
      HTTP-200 completed:     29151
      HTTP-503 completed:     517
   Failed requests:           332
      Timeout failures:       332

Total transferred:            18504105 bytes
Requests per second:          21.310
Time per request:             46.926603ms

Errors:
 332x  Get "https://www.electrobsd.org/": proxyconnect tcp: dial tcp 172.16.1.7:8118: connect: operation timed out

privoxy-experimental-libressl-devel 3.0.32.20210216 with LibreSSL devel 3.3.1

[fk@benchmark-vm ~]$ go/bin/ab-proxy -s 1000 -c 1000 -n 10000 --bursts 3 --proxy http://172.16.1.7:8118/ --show-errors -H "Connection: close" https://www.electrobsd.org/
Benchmarking 'https://www.electrobsd.org/' using proxy 'http://172.16.1.7:8118/' with a total of 30000 GET requests:

Number of bursts:             3
Number of request per burst   10000
Concurrency level:            1000
Time taken for tests:         25m4.91063846s

Total initiated requests:     30000
   Completed requests:        29672
      HTTP-200 completed:     29341
      HTTP-503 completed:     331
   Failed requests:           328
      Timeout failures:       328

Total transferred:            17292495 bytes
Requests per second:          19.935
Time per request:             50.163687ms

Errors:
 328x  Get "https://www.electrobsd.org/": proxyconnect tcp: dial tcp 172.16.1.7:8118: connect: operation timed out

privoxy-experimental-ports-openssl 3.0.32.20210216 with OpenSSL 1.1.1i,1

[fk@benchmark-vm ~]$ go/bin/ab-proxy -s 1000 -c 1000 -n 10000 --bursts 3 --proxy http://172.16.1.7:8118/ --show-errors -H "Connection: close" https://www.electrobsd.org/
Benchmarking 'https://www.electrobsd.org/' using proxy 'http://172.16.1.7:8118/' with a total of 30000 GET requests:

Number of bursts:             3
Number of request per burst   10000
Concurrency level:            1000
Time taken for tests:         22m0.965955015s

Total initiated requests:     30000
   Completed requests:        29355
      HTTP-200 completed:     28919
      HTTP-503 completed:     436
   Failed requests:           645
      Timeout failures:       645

Total transferred:            17815950 bytes
Requests per second:          22.711
Time per request:             44.032198ms

Errors:
 645x  Get "https://www.electrobsd.org/": proxyconnect tcp: dial tcp 172.16.1.7:8118: connect: operation timed out

privoxy-experimental-ports-openssl-devel 3.0.32.20210216 with OpenSSL devel 3.0.0.a11

[fk@benchmark-vm ~]$ go/bin/ab-proxy -s 1000 -c 1000 -n 10000 --bursts 3 --proxy http://172.16.1.7:8118/ --show-errors -H "Connection: close" https://www.electrobsd.org/
Benchmarking 'https://www.electrobsd.org/' using proxy 'http://172.16.1.7:8118/' with a total of 30000 GET requests:

Number of bursts:             3
Number of request per burst   10000
Concurrency level:            1000
Time taken for tests:         26m55.960774332s

Total initiated requests:     30000
   Completed requests:        29871
      HTTP-200 completed:     29635
      HTTP-503 completed:     236
   Failed requests:           129
      Timeout failures:       129

Total transferred:            16774110 bytes
Requests per second:          18.565
Time per request:             53.865359ms

Errors:
 129x  Get "https://www.electrobsd.org/": proxyconnect tcp: dial tcp 172.16.1.7:8118: connect: operation timed out

privoxy-experimental-wolfssl 3.0.32.20210216 with wolfSSL 4.7.0_1

[fk@benchmark-vm ~]$ go/bin/ab-proxy -s 1000 -c 1000 -n 10000 --bursts 3 --proxy http://172.16.1.7:8118/ --show-errors -H "Connection: close" https://www.electrobsd.org/
Benchmarking 'https://www.electrobsd.org/' using proxy 'http://172.16.1.7:8118/' with a total of 30000 GET requests:

Number of bursts:             3
Number of request per burst   10000
Concurrency level:            1000
Time taken for tests:         17m46.797803308s

Total initiated requests:     30000
   Completed requests:        25512
      HTTP-200 completed:     25119
      HTTP-503 completed:     393
   Failed requests:           4488
      Timeout failures:       4488

Total transferred:            15575445 bytes
Requests per second:          28.122
Time per request:             35.559926ms

Errors:
 4488x  Get "https://www.electrobsd.org/": proxyconnect tcp: dial tcp 172.16.1.7:8118: connect: operation timed out

privoxy-experimental-mbedtls 3.0.32.20210216 with mbedtls 2.16.9_6

[fk@benchmark-vm ~]$ go/bin/ab-proxy -s 1000 -c 1000 -n 10000 --bursts 3 --proxy http://172.16.1.7:8118/ --show-errors -H "Connection: close" https://www.electrobsd.org/
Benchmarking 'https://www.electrobsd.org/' using proxy 'http://172.16.1.7:8118/' with a total of 30000 GET requests:

Number of bursts:             3
Number of request per burst   10000
Concurrency level:            1000
Time taken for tests:         34m44.465265753s

Total initiated requests:     30000
   Completed requests:        29935
      HTTP-200 completed:     29789
      HTTP-503 completed:     146
   Failed requests:           65
      Timeout failures:       65

Total transferred:            16219500 bytes
Requests per second:          14.392
Time per request:             69.482175ms

Errors:
 65x  Get "https://www.electrobsd.org/": proxyconnect tcp: dial tcp 172.16.1.7:8118: connect: operation timed out

RSA key generation

For this test I added and enabled an invalidate-all-certificates directive that causes Privoxy to generate a new X509 host certificate for each request.

Normally Privoxy generates X509 certificates that are valid for 90 days.

The benchmark is requesting a page from the Privoxy user manual which Privoxy serves itself, therefore there are no outgoing connections. Most of the time is spend generating the 2048 bit RSA key for the certificate.

privoxy-experimental 3.0.32.20210216 with OpenSSL 1.0.2t-ElectroBSD from base

[fk@benchmark-vm ~]$ go/bin/ab-proxy -s 1000 -c 1 -n 1000 --proxy http://172.16.1.7:8118/ --show-errors -H "Connection: close" https://p.p/user-manual/
Benchmarking 'https://p.p/user-manual/' using proxy 'http://172.16.1.7:8118/' with a total of 1000 GET requests:

Number of bursts:             1
Number of request per burst   1000
Concurrency level:            1
Time taken for tests:         3m51.207656674s

Total initiated requests:     1000
   Completed requests:        1000
      HTTP-200 completed:     1000
   Failed requests:           0

Total transferred:            21587000 bytes
Requests per second:          4.325
Time per request:             231.207656ms

privoxy-experimental-libressl 3.0.32.20210216 with LibreSSL 3.2.3_2

[fk@benchmark-vm ~]$ go/bin/ab-proxy -s 5000 -c 1 -n 1000 --proxy http://172.16.1.7:8118/ --show-errors -H "Connection: close" https://p.p/user-manual/
Benchmarking 'https://p.p/user-manual/' using proxy 'http://172.16.1.7:8118/' with a total of 1000 GET requests:

Number of bursts:             1
Number of request per burst   1000
Concurrency level:            1
Time taken for tests:         3m54.853337356s

Total initiated requests:     1000
   Completed requests:        1000
      HTTP-200 completed:     1000
   Failed requests:           0

Total transferred:            21587000 bytes
Requests per second:          4.258
Time per request:             234.853337ms

privoxy-experimental-libressl-devel 3.0.32.20210216 with LibreSSL devel 3.3.1

[fk@benchmark-vm ~]$ go/bin/ab-proxy -s 5000 -c 1 -n 1000 --proxy http://172.16.1.7:8118/ --show-errors -H "Connection: close" https://p.p/user-manual/
Benchmarking 'https://p.p/user-manual/' using proxy 'http://172.16.1.7:8118/' with a total of 1000 GET requests:

Number of bursts:             1
Number of request per burst   1000
Concurrency level:            1
Time taken for tests:         4m18.215755982s

Total initiated requests:     1000
   Completed requests:        1000
      HTTP-200 completed:     1000
   Failed requests:           0

Total transferred:            21587000 bytes
Requests per second:          3.873
Time per request:             258.215755ms

privoxy-experimental-ports-openssl 3.0.32.20210216 with OpenSSL 1.1.1i,1

[fk@benchmark-vm ~]$ go/bin/ab-proxy -s 5000 -c 1 -n 1000 --proxy http://172.16.1.7:8118/ --show-errors -H "Connection: close" https://p.p/user-manual/
Benchmarking 'https://p.p/user-manual/' using proxy 'http://172.16.1.7:8118/' with a total of 1000 GET requests:

Number of bursts:             1
Number of request per burst   1000
Concurrency level:            1
Time taken for tests:         5m59.344970629s

Total initiated requests:     1000
   Completed requests:        999
      HTTP-200 completed:     999
   Failed requests:           1
      Timeout failures:       1

Total transferred:            21565413 bytes
Requests per second:          2.783
Time per request:             359.34497ms

Errors:
 1x  Get "https://p.p/user-manual/": proxyconnect tcp: dial tcp 172.16.1.7:8118: connect: operation timed out

privoxy-experimental-ports-openssl-devel 3.0.32.20210216 with OpenSSL devel 3.0.0.a11

[fk@benchmark-vm ~]$ go/bin/ab-proxy -s 5000 -c 1 -n 1000 --proxy http://172.16.1.7:8118/ --show-errors -H "Connection: close" https://p.p/user-manual/
Benchmarking 'https://p.p/user-manual/' using proxy 'http://172.16.1.7:8118/' with a total of 1000 GET requests:

Number of bursts:             1
Number of request per burst   1000
Concurrency level:            1
Time taken for tests:         14m8.243950082s

Total initiated requests:     1000
   Completed requests:        999
      HTTP-200 completed:     999
   Failed requests:           1
      Timeout failures:       1

Total transferred:            21565413 bytes
Requests per second:          1.179
Time per request:             848.24395ms

Errors:
 1x  Get "https://p.p/user-manual/": proxyconnect tcp: dial tcp 172.16.1.7:8118: connect: operation timed out

privoxy-experimental-wolfssl 3.0.32.20210216 with wolfSSL 4.7.0_1

[fk@benchmark-vm ~]$ go/bin/ab-proxy -s 5000 -c 1 -n 1000 --proxy http://172.16.1.7:8118/ --show-errors -H "Connection: close" https://p.p/user-manual/
Benchmarking 'https://p.p/user-manual/' using proxy 'http://172.16.1.7:8118/' with a total of 1000 GET requests:

Number of bursts:             1
Number of request per burst   1000
Concurrency level:            1
Time taken for tests:         10m6.475444546s

Total initiated requests:     1000
   Completed requests:        1000
      HTTP-200 completed:     1000
   Failed requests:           0

Total transferred:            21587000 bytes
Requests per second:          1.649
Time per request:             606.475444ms

privoxy-experimental-mbedtls 3.0.32.20210216 with mbedtls 2.16.9_6

[fk@benchmark-vm ~]$ go/bin/ab-proxy -s 5000 -c 1 -n 1000 --proxy http://172.16.1.7:8118/ --show-errors -H "Connection: close" https://p.p/user-manual/
Benchmarking 'https://p.p/user-manual/' using proxy 'http://172.16.1.7:8118/' with a total of 1000 GET requests:

Number of bursts:             1
Number of request per burst   1000
Concurrency level:            1
Time taken for tests:         6m7.916393792s

Total initiated requests:     1000
   Completed requests:        1000
      HTTP-200 completed:     1000
   Failed requests:           0

Total transferred:            21587000 bytes
Requests per second:          2.718
Time per request:             367.916393ms

ab-proxy errors

I have not yet properly investigated the ab-proxy errors.

It's suspicious that proxyconnect tcp: dial tcp 172.16.1.7:8118: connect: operation timed out errors were reported even for some of the RSA key generation tests with concurrency level 1.

It's not obvious to me which limit is being hit when ab-proxy complaints about Too many open connections.

Privoxy 3.0.32 experimental 2021-02-16 679b708984 tested with hey

Connection: keep-alive, concurrency level 1000

privoxy-experimental 3.0.32.20210216 with OpenSSL 1.0.2t-ElectroBSD from base

[fk@benchmark-vm ~]$ hey -x http://172.16.1.7:8118 -n 10000 -c 1000 -t 0 https://www.electrobsd.org/

Summary:
  Total:        70.8044 secs
  Slowest:      39.0221 secs
  Fastest:      0.0100 secs
  Average:      6.5599 secs
  Requests/sec: 141.2341


Response time histogram:
  0.010 [1]     |
  3.911 [4166]  |________________________________________
  7.812 [3032]  |_____________________________
  11.714 [1363] |_____________
  15.615 [633]  |______
  19.516 [329]  |___
  23.417 [269]  |___
  27.318 [87]   |_
  31.220 [38]   |
  35.121 [61]   |_
  39.022 [21]   |


Latency distribution:
  10% in 2.3938 secs
  25% in 3.1952 secs
  50% in 4.2323 secs
  75% in 8.4616 secs
  90% in 13.7324 secs
  95% in 18.9752 secs
  99% in 29.2693 secs

Details (average, fastest, slowest):
  DNS+dialup:   0.1407 secs, 0.0100 secs, 39.0221 secs
  DNS-lookup:   0.0000 secs, 0.0000 secs, 0.0000 secs
  req write:    0.0005 secs, 0.0000 secs, 0.1165 secs
  resp wait:    1.6825 secs, 0.0080 secs, 27.2391 secs
  resp read:    0.0021 secs, 0.0001 secs, 0.5911 secs

Status code distribution:
  [200] 10000 responses

privoxy-experimental-libressl 3.0.32.20210216 with LibreSSL 3.2.3_2

[fk@benchmark-vm ~]$ hey -x http://172.16.1.7:8118 -n 10000 -c 1000 -t 0 https://www.electrobsd.org/

Summary:
  Total:        58.0847 secs
  Slowest:      29.9009 secs
  Fastest:      0.0104 secs
  Average:      5.1935 secs
  Requests/sec: 172.1623


Response time histogram:
  0.010 [1]     |
  2.999 [1384]  |________
  5.989 [6588]  |________________________________________
  8.978 [770]   |_____
  11.967 [436]  |___
  14.956 [333]  |__
  17.945 [220]  |_
  20.934 [183]  |_
  23.923 [43]   |
  26.912 [27]   |
  29.901 [15]   |


Latency distribution:
  10% in 2.7917 secs
  25% in 3.1824 secs
  50% in 3.5952 secs
  75% in 4.7549 secs
  90% in 10.4189 secs
  95% in 14.8431 secs
  99% in 20.5910 secs

Details (average, fastest, slowest):
  DNS+dialup:   0.0543 secs, 0.0104 secs, 29.9009 secs
  DNS-lookup:   0.0000 secs, 0.0000 secs, 0.0000 secs
  req write:    0.0005 secs, 0.0000 secs, 0.5658 secs
  resp wait:    0.8956 secs, 0.0082 secs, 26.5739 secs
  resp read:    0.0018 secs, 0.0001 secs, 0.6613 secs

Status code distribution:
  [200] 10000 responses

privoxy-experimental-libressl-devel 3.0.32.20210216 with LibreSSL devel 3.3.1

[fk@benchmark-vm ~]$ hey -x http://172.16.1.7:8118 -n 10000 -c 1000 -t 0 https://www.electrobsd.org/

Summary:
  Total:        45.4728 secs
  Slowest:      45.1093 secs
  Fastest:      0.0089 secs
  Average:      3.9195 secs
  Requests/sec: 219.9116


Response time histogram:
  0.009 [1]     |
  4.519 [8462]  |________________________________________
  9.029 [1172]  |______
  13.539 [234]  |_
  18.049 [3]    |
  22.559 [9]    |
  27.069 [6]    |
  31.579 [17]   |
  36.089 [13]   |
  40.599 [8]    |
  45.109 [75]   |


Latency distribution:
  10% in 0.7545 secs
  25% in 2.8709 secs
  50% in 3.3514 secs
  75% in 3.8533 secs
  90% in 5.9146 secs
  95% in 8.7582 secs
  99% in 30.5981 secs

Details (average, fastest, slowest):
  DNS+dialup:   0.0396 secs, 0.0089 secs, 45.1093 secs
  DNS-lookup:   0.0000 secs, 0.0000 secs, 0.0000 secs
  req write:    0.0003 secs, 0.0000 secs, 0.0647 secs
  resp wait:    0.7306 secs, 0.0079 secs, 44.5228 secs
  resp read:    0.0024 secs, 0.0001 secs, 8.5983 secs

Status code distribution:
  [200] 10000 responses

privoxy-experimental-ports-openssl 3.0.32.20210216 with OpenSSL 1.1.1i,1

[fk@benchmark-vm ~]$ hey -x http://172.16.1.7:8118 -n 10000 -c 1000 -t 0 https://www.electrobsd.org/

Summary:
  Total:        53.5002 secs
  Slowest:      53.2844 secs
  Fastest:      0.0084 secs
  Average:      4.6653 secs
  Requests/sec: 186.9150


Response time histogram:
  0.008 [1]     |
  5.336 [7242]  |________________________________________
  10.664 [2562] |______________
  15.991 [111]  |_
  21.319 [25]   |
  26.646 [3]    |
  31.974 [1]    |
  37.302 [2]    |
  42.629 [0]    |
  47.957 [8]    |
  53.284 [45]   |


Latency distribution:
  10% in 2.6186 secs
  25% in 3.1237 secs
  50% in 4.2889 secs
  75% in 5.5426 secs
  90% in 7.2864 secs
  95% in 8.0298 secs
  99% in 14.1800 secs

Details (average, fastest, slowest):
  DNS+dialup:   0.0127 secs, 0.0084 secs, 53.2844 secs
  DNS-lookup:   0.0000 secs, 0.0000 secs, 0.0000 secs
  req write:    0.0003 secs, 0.0000 secs, 0.0691 secs
  resp wait:    0.5642 secs, 0.0082 secs, 52.4999 secs
  resp read:    0.0018 secs, 0.0001 secs, 0.8801 secs

Status code distribution:
  [200] 10000 responses

privoxy-experimental-ports-openssl-devel 3.0.32.20210216 with OpenSSL devel 3.0.0.a11

[fk@benchmark-vm ~]$ hey -x http://172.16.1.7:8118 -n 10000 -c 1000 -t 0 https://www.electrobsd.org/

Summary:
  Total:        45.2568 secs
  Slowest:      44.3002 secs
  Fastest:      0.0109 secs
  Average:      4.1396 secs
  Requests/sec: 220.9610


Response time histogram:
  0.011 [1]     |
  4.440 [7903]  |________________________________________
  8.869 [1985]  |__________
  13.298 [14]   |
  17.727 [23]   |
  22.156 [25]   |
  26.584 [11]   |
  31.013 [10]   |
  35.442 [8]    |
  39.871 [14]   |
  44.300 [6]    |


Latency distribution:
  10% in 3.0057 secs
  25% in 3.4220 secs
  50% in 3.8525 secs
  75% in 4.3496 secs
  90% in 5.6387 secs
  95% in 7.0523 secs
  99% in 12.3051 secs

Details (average, fastest, slowest):
  DNS+dialup:   0.0347 secs, 0.0109 secs, 44.3002 secs
  DNS-lookup:   0.0000 secs, 0.0000 secs, 0.0000 secs
  req write:    0.0007 secs, 0.0000 secs, 0.4245 secs
  resp wait:    0.4287 secs, 0.0082 secs, 41.7396 secs
  resp read:    0.0023 secs, 0.0001 secs, 2.8815 secs

Status code distribution:
  [200] 10000 responses

privoxy-experimental-wolfssl 3.0.32.20210216 with wolfSSL 4.7.0_1

[fk@benchmark-vm ~]$ hey -x http://172.16.1.7:8118 -n 10000 -c 1000 -t 0 https://www.electrobsd.org/

Summary:
  Total:        70.0007 secs
  Slowest:      41.1610 secs
  Fastest:      0.0085 secs
  Average:      5.8105 secs
  Requests/sec: 142.8557


Response time histogram:
  0.009 [1]     |
  4.124 [5269]  |________________________________________
  8.239 [3145]  |________________________
  12.354 [695]  |_____
  16.470 [348]  |___
  20.585 [117]  |_
  24.700 [225]  |__
  28.815 [19]   |
  32.931 [42]   |
  37.046 [31]   |
  41.161 [108]  |_


Latency distribution:
  10% in 2.0746 secs
  25% in 3.2209 secs
  50% in 4.0434 secs
  75% in 5.4152 secs
  90% in 11.4100 secs
  95% in 17.8118 secs
  99% in 37.3581 secs

Details (average, fastest, slowest):
  DNS+dialup:   0.0093 secs, 0.0085 secs, 41.1610 secs
  DNS-lookup:   0.0000 secs, 0.0000 secs, 0.0000 secs
  req write:    0.0004 secs, 0.0000 secs, 0.4865 secs
  resp wait:    1.2999 secs, 0.0079 secs, 34.2348 secs
  resp read:    0.0021 secs, 0.0001 secs, 2.0635 secs

Status code distribution:
  [200] 10000 responses

privoxy-experimental-mbedtls 3.0.32.20210216 with mbedtls 2.16.9_6

[fk@benchmark-vm ~]$ hey -x http://172.16.1.7:8118 -n 10000 -c 1000 -t 0 https://www.electrobsd.org/

Summary:
  Total:        369.2357 secs
  Slowest:      166.0667 secs
  Fastest:      0.0091 secs
  Average:      30.3497 secs
  Requests/sec: 27.0830


Response time histogram:
  0.009 [1]     |
  16.615 [6120] |________________________________________
  33.221 [185]  |_
  49.826 [575]  |____
  66.432 [413]  |___
  83.038 [170]  |_
  99.644 [173]  |_
  116.249 [266] |__
  132.855 [630] |____
  149.461 [192] |_
  166.067 [192] |_


Latency distribution:
  10% in 0.9116 secs
  25% in 2.9378 secs
  50% in 4.5298 secs
  75% in 44.1925 secs
  90% in 121.4544 secs
  95% in 130.6333 secs
  99% in 155.0877 secs

Details (average, fastest, slowest):
  DNS+dialup:   0.9649 secs, 0.0091 secs, 166.0667 secs
  DNS-lookup:   0.0000 secs, 0.0000 secs, 0.0000 secs
  req write:    0.2049 secs, 0.0000 secs, 27.5575 secs
  resp wait:    14.5920 secs, 0.0086 secs, 111.2285 secs
  resp read:    0.2987 secs, 0.0001 secs, 25.8274 secs

Status code distribution:
  [200] 8917 responses

Error distribution:
  [1083]        Get "https://www.electrobsd.org/": context deadline exceeded

Connection: close, concurrency level 1000

privoxy-experimental 3.0.32.20210216 with OpenSSL 1.0.2t-ElectroBSD from base

[fk@benchmark-vm ~]$ hey -x http://172.16.1.7:8118 -n 10000 -c 1000 -t 0 -disable-keepalive https://www.electrobsd.org/

Summary:
  Total:        586.4203 secs
  Slowest:      385.5857 secs
  Fastest:      0.0861 secs
  Average:      37.4564 secs
  Requests/sec: 17.0526

  Total data:   429135 bytes
  Size/request: 45 bytes

Response time histogram:
  0.086 [1]     |
  38.636 [6089] |________________________________________
  77.186 [3258] |_____________________
  115.736 [110] |_
  154.286 [1]   |
  192.836 [0]   |
  231.386 [0]   |
  269.936 [0]   |
  308.486 [0]   |
  347.036 [0]   |
  385.586 [20]  |


Latency distribution:
  10% in 25.5035 secs
  25% in 29.9598 secs
  50% in 34.5757 secs
  75% in 43.7889 secs
  90% in 50.2532 secs
  95% in 62.4555 secs
  99% in 89.7409 secs

Details (average, fastest, slowest):
  DNS+dialup:   29.9351 secs, 0.0861 secs, 385.5857 secs
  DNS-lookup:   0.0000 secs, 0.0000 secs, 0.0000 secs
  req write:    0.0005 secs, 0.0000 secs, 0.2965 secs
  resp wait:    7.5154 secs, 0.0719 secs, 362.7325 secs
  resp read:    0.0053 secs, 0.0001 secs, 6.6434 secs

Status code distribution:
  [200] 9418 responses
  [503] 61 responses

Error distribution:
  [363] Get "https://www.electrobsd.org/": context deadline exceeded
  [158] Get "https://www.electrobsd.org/": proxyconnect tcp: dial tcp 172.16.1.7:8118: connect: operation timed out

privoxy-experimental-libressl 3.0.32.20210216 with LibreSSL 3.2.3_2

[fk@benchmark-vm ~]$ hey -x http://172.16.1.7:8118 -n 10000 -c 1000 -t 0 -disable-keepalive https://www.electrobsd.org/

Summary:
  Total:        658.7700 secs
  Slowest:      358.0393 secs
  Fastest:      0.0796 secs
  Average:      44.1614 secs
  Requests/sec: 15.1798

  Total data:   729126 bytes
  Size/request: 75 bytes

Response time histogram:
  0.080 [1]     |
  35.876 [1482] |________
  71.672 [7671] |________________________________________
  107.468 [371] |__
  143.263 [58]  |
  179.059 [1]   |
  214.855 [0]   |
  250.651 [0]   |
  286.447 [0]   |
  322.243 [1]   |
  358.039 [9]   |


Latency distribution:
  10% in 34.0873 secs
  25% in 37.6683 secs
  50% in 41.8934 secs
  75% in 47.3630 secs
  90% in 59.8096 secs
  95% in 69.1311 secs
  99% in 93.7552 secs

Details (average, fastest, slowest):
  DNS+dialup:   37.2611 secs, 0.0796 secs, 358.0393 secs
  DNS-lookup:   0.0000 secs, 0.0000 secs, 0.0000 secs
  req write:    0.0002 secs, 0.0000 secs, 0.0114 secs
  resp wait:    6.8963 secs, 0.0701 secs, 330.3939 secs
  resp read:    0.0038 secs, 0.0001 secs, 7.4791 secs

Status code distribution:
  [200] 9495 responses
  [502] 57 responses
  [503] 42 responses

Error distribution:
  [364] Get "https://www.electrobsd.org/": context deadline exceeded
  [42]  Get "https://www.electrobsd.org/": proxyconnect tcp: dial tcp 172.16.1.7:8118: connect: operation timed out

privoxy-experimental-libressl-devel 3.0.32.20210216 with LibreSSL devel 3.3.1

[fk@benchmark-vm ~]$ hey -x http://172.16.1.7:8118 -n 10000 -c 1000 -t 0 -disable-keepalive https://www.electrobsd.org/

Summary:
  Total:        498.4878 secs
  Slowest:      123.9539 secs
  Fastest:      0.0959 secs
  Average:      44.9095 secs
  Requests/sec: 20.0607

  Total data:   1020075 bytes
  Size/request: 102 bytes

Response time histogram:
  0.096 [1]     |
  12.482 [275]  |__
  24.868 [100]  |_
  37.253 [1863] |_________________
  49.639 [4511] |________________________________________
  62.025 [2244] |____________________
  74.411 [869]  |________
  86.797 [53]   |
  99.182 [14]   |
  111.568 [10]  |
  123.954 [1]   |


Latency distribution:
  10% in 32.2111 secs
  25% in 37.8849 secs
  50% in 43.8031 secs
  75% in 52.6480 secs
  90% in 61.1590 secs
  95% in 67.6880 secs
  99% in 73.4740 secs

Details (average, fastest, slowest):
  DNS+dialup:   38.6913 secs, 0.0959 secs, 123.9539 secs
  DNS-lookup:   0.0000 secs, 0.0000 secs, 0.0000 secs
  req write:    0.0003 secs, 0.0000 secs, 0.0171 secs
  resp wait:    6.2129 secs, 0.0804 secs, 52.7336 secs
  resp read:    0.0050 secs, 0.0001 secs, 10.3844 secs

Status code distribution:
  [200] 9796 responses
  [503] 145 responses

Error distribution:
  [54]  Get "https://www.electrobsd.org/": context deadline exceeded
  [5]   Get "https://www.electrobsd.org/": proxyconnect tcp: dial tcp 172.16.1.7:8118: connect: operation timed out

privoxy-experimental-ports-openssl 3.0.32.20210216 with OpenSSL 1.1.1i,1

[fk@benchmark-vm ~]$ hey -x http://172.16.1.7:8118 -n 10000 -c 1000 -t 0 -disable-keepalive https://www.electrobsd.org/

Summary:
  Total:        448.7498 secs
  Slowest:      95.4216 secs
  Fastest:      0.1897 secs
  Average:      40.1372 secs
  Requests/sec: 22.2841

  Total data:   527625 bytes
  Size/request: 53 bytes

Response time histogram:
  0.190 [1]     |
  9.713 [116]   |_
  19.236 [375]  |___
  28.759 [120]  |_
  38.282 [5068] |________________________________________
  47.806 [1613] |_____________
  57.329 [1932] |_______________
  66.852 [672]  |_____
  76.375 [23]   |
  85.898 [25]   |
  95.422 [8]    |


Latency distribution:
  10% in 31.4769 secs
  25% in 34.1200 secs
  50% in 37.1113 secs
  75% in 49.0653 secs
  90% in 55.7705 secs
  95% in 58.4753 secs
  99% in 64.5587 secs

Details (average, fastest, slowest):
  DNS+dialup:   35.7682 secs, 0.1897 secs, 95.4216 secs
  DNS-lookup:   0.0000 secs, 0.0000 secs, 0.0000 secs
  req write:    0.0003 secs, 0.0000 secs, 0.1436 secs
  resp wait:    4.3613 secs, 0.0876 secs, 51.5034 secs
  resp read:    0.0074 secs, 0.0001 secs, 5.0321 secs

Status code distribution:
  [200] 9878 responses
  [503] 75 responses

Error distribution:
  [47]  Get "https://www.electrobsd.org/": proxyconnect tcp: dial tcp 172.16.1.7:8118: connect: operation timed out

privoxy-experimental-ports-openssl-devel 3.0.32.20210216 with OpenSSL devel 3.0.0.a11

[fk@benchmark-vm ~]$ hey -x http://172.16.1.7:8118 -n 10000 -c 1000 -t 0 -disable-keepalive https://www.electrobsd.org/

Summary:
  Total:        540.4711 secs
  Slowest:      107.6423 secs
  Fastest:      1.5339 secs
  Average:      48.4370 secs
  Requests/sec: 18.5024

  Total data:   196980 bytes
  Size/request: 20 bytes

Response time histogram:
  1.534 [1]     |
  12.145 [12]   |
  22.756 [46]   |
  33.366 [217]  |__
  43.977 [2850] |_____________________________
  54.588 [3893] |________________________________________
  65.199 [2472] |_________________________
  75.810 [175]  |__
  86.421 [17]   |
  97.031 [2]    |
  107.642 [1]   |


Latency distribution:
  10% in 39.5666 secs
  25% in 42.9131 secs
  50% in 46.9803 secs
  75% in 55.1250 secs
  90% in 58.6916 secs
  95% in 60.9301 secs
  99% in 70.8738 secs

Details (average, fastest, slowest):
  DNS+dialup:   43.6338 secs, 1.5339 secs, 107.6423 secs
  DNS-lookup:   0.0000 secs, 0.0000 secs, 0.0000 secs
  req write:    0.0002 secs, 0.0000 secs, 0.0225 secs
  resp wait:    4.8004 secs, 0.0896 secs, 43.6299 secs
  resp read:    0.0026 secs, 0.0001 secs, 1.1191 secs

Status code distribution:
  [200] 9658 responses
  [503] 28 responses

Error distribution:
  [275] Get "https://www.electrobsd.org/": context deadline exceeded
  [39]  Get "https://www.electrobsd.org/": proxyconnect tcp: dial tcp 172.16.1.7:8118: connect: operation timed out

privoxy-experimental-wolfssl 3.0.32.20210216 with wolfSSL 4.7.0_1

[fk@benchmark-vm ~]$ hey -x http://172.16.1.7:8118 -n 10000 -c 1000 -t 0 -disable-keepalive https://www.electrobsd.org/

Summary:
  Total:        651.9580 secs
  Slowest:      356.3045 secs
  Fastest:      0.0758 secs
  Average:      31.9959 secs
  Requests/sec: 15.3384

  Total data:   316575 bytes
  Size/request: 31 bytes

Response time histogram:
  0.076 [1]     |
  35.699 [7634] |________________________________________
  71.322 [2303] |____________
  106.944 [9]   |
  142.567 [0]   |
  178.190 [0]   |
  213.813 [0]   |
  249.436 [0]   |
  285.059 [0]   |
  320.682 [0]   |
  356.304 [4]   |


Latency distribution:
  10% in 24.4374 secs
  25% in 27.4192 secs
  50% in 30.1033 secs
  75% in 34.9318 secs
  90% in 48.9801 secs
  95% in 52.6747 secs
  99% in 56.6523 secs

Details (average, fastest, slowest):
  DNS+dialup:   26.3534 secs, 0.0758 secs, 356.3045 secs
  DNS-lookup:   0.0000 secs, 0.0000 secs, 0.0000 secs
  req write:    0.0007 secs, 0.0000 secs, 0.1846 secs
  resp wait:    5.6347 secs, 0.0587 secs, 331.2605 secs
  resp read:    0.0071 secs, 0.0001 secs, 2.1032 secs

Status code distribution:
  [200] 9902 responses
  [403] 4 responses
  [503] 45 responses

Error distribution:
    [49]  Get "https://www.electrobsd.org/": proxyconnect tcp: dial tcp 172.16.1.7:8118: connect: operation timed out

The 403 errors are the result of the certificate validation failing.

privoxy-experimental-mbedtls 3.0.32.20210216 with mbedtls 2.16.9_6

[fk@benchmark-vm ~]$ hey -x http://172.16.1.7:8118 -n 10000 -c 1000 -t 0 -disable-keepalive https://www.electrobsd.org/

Summary:
  Total:        745.4701 secs
  Slowest:      132.6298 secs
  Fastest:      0.1535 secs
  Average:      68.3000 secs
  Requests/sec: 13.4144

  Total data:   239190 bytes
  Size/request: 23 bytes

Response time histogram:
  0.154 [1]     |
  13.401 [39]   |_
  26.649 [137]  |__
  39.896 [885]  |____________
  53.144 [1675] |________________________
  66.392 [1406] |____________________
  79.639 [2068] |_____________________________
  92.887 [2845] |________________________________________
  106.135 [830] |____________
  119.382 [102] |_
  132.630 [2]   |


Latency distribution:
  10% in 39.3582 secs
  25% in 51.0014 secs
  50% in 72.5942 secs
  75% in 84.0988 secs
  90% in 92.1742 secs
  95% in 96.1419 secs
  99% in 106.3705 secs

Details (average, fastest, slowest):
  DNS+dialup:   58.9630 secs, 0.1535 secs, 132.6298 secs
  DNS-lookup:   0.0000 secs, 0.0000 secs, 0.0000 secs
  req write:    0.4718 secs, 0.0000 secs, 9.6484 secs
  resp wait:    7.9882 secs, 0.0836 secs, 58.6134 secs
  resp read:    0.8770 secs, 0.0002 secs, 18.9068 secs

Status code distribution:
  [200] 9956 responses
  [503] 34 responses

Error distribution:
  [10]  Get "https://www.electrobsd.org/": proxyconnect tcp: dial tcp 172.16.1.7:8118: connect: operation timed out

Privoxy CPU time and resident memory use according to top after the test

TLS libraryCPU timeMemory size
OpenSSL 1.0.2t5:56504M
LibreSSL 3.2.3_26:35586M
LibreSSL devel 3.3.16:55503M
OpenSSL 1.1.1i,16:12310M
OpenSSL devel 3.0.0.a117:30385M
wolfSSL 4.7.0_14:37302M
mbedtls 2.16.9_68:17304M

Cipher suites and TLS versions used (the same as with ab-proxy)

TLS libraryClient side TLS version and cipher suiteServer side TLS version and cipher suite
OpenSSL 1.0.2tTLSv1.2 AES128-GCM-SHA256TLSv1.2 ECDHE-RSA-AES128-GCM-SHA256
LibreSSL 3.2.3_2TLSv1.3 AEAD-CHACHA20-POLY1305-SHA256TLSv1.2 ECDHE-RSA-AES128-GCM-SHA256
LibreSSL devel 3.3.1TLSv1.3 AEAD-CHACHA20-POLY1305-SHA256TLSv1.2 ECDHE-RSA-AES128-GCM-SHA256
OpenSSL 1.1.1i,1TLSv1.3 TLS_CHACHA20_POLY1305_SHA256TLSv1.2 ECDHE-RSA-AES128-GCM-SHA256
OpenSSL devel 3.0.0.a11TLSv1.3 TLS_CHACHA20_POLY1305_SHA256TLSv1.2 ECDHE-RSA-AES128-GCM-SHA256
wolfSSL 4.7.0_1TLSv1.3 TLS13-AES128-GCM-SHA256TLSv1.2 ECDHE-RSA-AES128-GCM-SHA256
mbedtls 2.16.9_6TLSv1.2 TLS-ECDHE-RSA-WITH-CHACHA20-POLY1305-SHA256TLSv1.2 TLS-ECDHE-RSA-WITH-AES-128-GCM-SHA256

hey errors

I have not yet properly investigated the hey errors either. The -t option obviously does not prevent timeouts from occurring.

The response time histograms are probably worth investigating as well.

2021-02-18: Privoxy 3.0.32 experimental 2021-02-16 679b708984 tested with hey

The OpenSSL port has been updated to 1.1.1j. wolfSSL has been recompiled with --enable-sp-asm and --enable-sp-math-all added to the configure flags which significantly speeds up RSA key generation (and probably other things as well). privoxy-experimental-ports-openssl 3.0.32.20210216 and privoxy-experimental-wolfssl 3.0.32.20210216 have been recompiled, the other ports remain the same.

RSA key generation

Privoxy configured as described above to force RSA key regeneration.

privoxy-experimental 3.0.32.20210216 with OpenSSL 1.0.2t-ElectroBSD from base

[fk@benchmark-vm ~]$ hey -x http://172.16.1.7:8118 -n 1000 -c 1 -t 0 -disable-keepalive https://p.p/user-manual/

Summary:
  Total:        205.3925 secs
  Slowest:      6.3301 secs
  Fastest:      0.0301 secs
  Average:      0.2054 secs
  Requests/sec: 4.8687

  Total data:   21587000 bytes
  Size/request: 21587 bytes

Response time histogram:
  0.030 [1]     |
  0.660 [997]   |________________________________________
  1.290 [1]     |
  1.920 [0]     |
  2.550 [0]     |
  3.180 [0]     |
  3.810 [0]     |
  4.440 [0]     |
  5.070 [0]     |
  5.700 [0]     |
  6.330 [1]     |


Latency distribution:
  10% in 0.0731 secs
  25% in 0.1124 secs
  50% in 0.1740 secs
  75% in 0.2601 secs
  90% in 0.3597 secs
  95% in 0.4293 secs
  99% in 0.5754 secs

Details (average, fastest, slowest):
  DNS+dialup:   0.2021 secs, 0.0301 secs, 6.3301 secs
  DNS-lookup:   0.0000 secs, 0.0000 secs, 0.0000 secs
  req write:    0.0001 secs, 0.0001 secs, 0.0002 secs
  resp wait:    0.0012 secs, 0.0009 secs, 0.0857 secs
  resp read:    0.0019 secs, 0.0011 secs, 0.0655 secs

Status code distribution:
  [200] 1000 responses

privoxy-experimental-libressl 3.0.32.20210216 with LibreSSL 3.2.3_2

[fk@benchmark-vm ~]$ hey -x http://172.16.1.7:8118 -n 1000 -c 1 -t 0 -disable-keepalive https://p.p/user-manual/

Summary:
  Total:        285.2183 secs
  Slowest:      0.8314 secs
  Fastest:      0.0340 secs
  Average:      0.2104 secs
  Requests/sec: 3.5061

  Total data:   21565413 bytes
  Size/request: 21587 bytes

Response time histogram:
  0.034 [1]     |
  0.114 [226]   |____________________________
  0.193 [321]   |________________________________________
  0.273 [193]   |________________________
  0.353 [120]   |_______________
  0.433 [72]    |_________
  0.512 [36]    |____
  0.592 [20]    |__
  0.672 [6]     |_
  0.752 [1]     |
  0.831 [3]     |


Latency distribution:
  10% in 0.0806 secs
  25% in 0.1185 secs
  50% in 0.1793 secs
  75% in 0.2768 secs
  90% in 0.3825 secs
  95% in 0.4715 secs
  99% in 0.6071 secs

Details (average, fastest, slowest):
  DNS+dialup:   0.2078 secs, 0.0340 secs, 0.8314 secs
  DNS-lookup:   0.0000 secs, 0.0000 secs, 0.0000 secs
  req write:    0.0001 secs, 0.0001 secs, 0.0002 secs
  resp wait:    0.0011 secs, 0.0009 secs, 0.0167 secs
  resp read:    0.0014 secs, 0.0003 secs, 0.0145 secs

Status code distribution:
  [200] 999 responses

Error distribution:
  [1]   Get "https://p.p/user-manual/": proxyconnect tcp: dial tcp 172.16.1.7:8118: connect: operation timed out

privoxy-experimental-libressl-devel 3.0.32.20210216 with LibreSSL devel 3.3.1

[fk@benchmark-vm ~]$ hey -x http://172.16.1.7:8118 -n 1000 -c 1 -t 0 -disable-keepalive https://p.p/user-manual/

Summary:
  Total:        412.1682 secs
  Slowest:      59.0361 secs
  Fastest:      0.0310 secs
  Average:      0.2627 secs
  Requests/sec: 2.4262

  Total data:   21543826 bytes
  Size/request: 21587 bytes

Response time histogram:
  0.031 [1]     |
  5.932 [996]   |________________________________________
  11.832 [0]    |
  17.733 [0]    |
  23.633 [0]    |
  29.534 [0]    |
  35.434 [0]    |
  41.335 [0]    |
  47.235 [0]    |
  53.136 [0]    |
  59.036 [1]    |


Latency distribution:
  10% in 0.0743 secs
  25% in 0.1081 secs
  50% in 0.1772 secs
  75% in 0.2612 secs
  90% in 0.3695 secs
  95% in 0.4542 secs
  99% in 0.7016 secs

Details (average, fastest, slowest):
  DNS+dialup:   0.2583 secs, 0.0310 secs, 59.0361 secs
  DNS-lookup:   0.0000 secs, 0.0000 secs, 0.0000 secs
  req write:    0.0001 secs, 0.0001 secs, 0.0003 secs
  resp wait:    0.0029 secs, 0.0009 secs, 0.1069 secs
  resp read:    0.0014 secs, 0.0003 secs, 0.0221 secs

Status code distribution:
  [200] 998 responses

Error distribution:
  [2]   Get "https://p.p/user-manual/": proxyconnect tcp: dial tcp 172.16.1.7:8118: connect: operation timed out

privoxy-experimental-ports-openssl 3.0.32.20210216 with OpenSSL 1.1.1j,1

[fk@benchmark-vm ~]$ hey -x http://172.16.1.7:8118 -n 1000 -c 1 -t 0 -disable-keepalive https://p.p/user-manual/

Summary:
  Total:        331.6408 secs
  Slowest:      34.4816 secs
  Fastest:      0.0321 secs
  Average:      0.2569 secs
  Requests/sec: 3.0153

  Total data:   21565413 bytes
  Size/request: 21587 bytes

Response time histogram:
  0.032 [1]     |
  3.477 [996]   |________________________________________
  6.922 [0]     |
  10.367 [0]    |
  13.812 [0]    |
  17.257 [0]    |
  20.702 [0]    |
  24.147 [1]    |
  27.592 [0]    |
  31.037 [0]    |
  34.482 [1]    |


Latency distribution:
  10% in 0.0746 secs
  25% in 0.1114 secs
  50% in 0.1692 secs
  75% in 0.2572 secs
  90% in 0.3718 secs
  95% in 0.4698 secs
  99% in 0.6620 secs

Details (average, fastest, slowest):
  DNS+dialup:   0.2529 secs, 0.0321 secs, 34.4816 secs
  DNS-lookup:   0.0000 secs, 0.0000 secs, 0.0000 secs
  req write:    0.0001 secs, 0.0001 secs, 0.0002 secs
  resp wait:    0.0025 secs, 0.0009 secs, 0.9218 secs
  resp read:    0.0014 secs, 0.0003 secs, 0.0318 secs

Status code distribution:
  [200] 999 responses

Error distribution:
  [1]   Get "https://p.p/user-manual/": proxyconnect tcp: dial tcp 172.16.1.7:8118: connect: operation timed out

privoxy-experimental-ports-openssl-devel 3.0.32.20210216 with OpenSSL devel 3.0.0.a11

[fk@benchmark-vm ~]$ hey -x http://172.16.1.7:8118 -n 1000 -c 1 -t 0 -disable-keepalive https://p.p/user-manual/

Summary:
  Total:        694.1342 secs
  Slowest:      2.3307 secs
  Fastest:      0.1918 secs
  Average:      0.6941 secs
  Requests/sec: 1.4406

  Total data:   21587000 bytes
  Size/request: 21587 bytes

Response time histogram:
  0.192 [1]     |
  0.406 [219]   |_____________________________
  0.620 [306]   |________________________________________
  0.833 [187]   |________________________
  1.047 [146]   |___________________
  1.261 [60]    |________
  1.475 [44]    |______
  1.689 [16]    |__
  1.903 [10]    |_
  2.117 [8]     |_
  2.331 [3]     |


Latency distribution:
  10% in 0.3290 secs
  25% in 0.4316 secs
  50% in 0.5993 secs
  75% in 0.8770 secs
  90% in 1.1933 secs
  95% in 1.3963 secs
  99% in 1.9116 secs

Details (average, fastest, slowest):
  DNS+dialup:   0.6905 secs, 0.1918 secs, 2.3307 secs
  DNS-lookup:   0.0000 secs, 0.0000 secs, 0.0000 secs
  req write:    0.0001 secs, 0.0001 secs, 0.0002 secs
  resp wait:    0.0018 secs, 0.0010 secs, 0.0243 secs
  resp read:    0.0018 secs, 0.0002 secs, 0.0284 secs

Status code distribution:
  [200] 1000 responses

privoxy-experimental-wolfssl 3.0.32.20210216 with wolfSSL 4.7.0_2

[fk@benchmark-vm ~]$ hey -x http://172.16.1.7:8118 -n 1000 -c 1 -t 0 -disable-keepalive https://p.p/user-manual/

Summary:
  Total:        158.4484 secs
  Slowest:      1.3990 secs
  Fastest:      0.0402 secs
  Average:      0.1584 secs
  Requests/sec: 6.3112

  Total data:   21587000 bytes
  Size/request: 21587 bytes

Response time histogram:
  0.040 [1]     |
  0.176 [685]   |________________________________________
  0.312 [260]   |_______________
  0.448 [43]    |___
  0.584 [10]    |_
  0.720 [0]     |
  0.855 [0]     |
  0.991 [0]     |
  1.127 [0]     |
  1.263 [0]     |
  1.399 [1]     |


Latency distribution:
  10% in 0.0706 secs
  25% in 0.0961 secs
  50% in 0.1392 secs
  75% in 0.1981 secs
  90% in 0.2687 secs
  95% in 0.3204 secs
  99% in 0.4758 secs

Details (average, fastest, slowest):
  DNS+dialup:   0.1551 secs, 0.0402 secs, 1.3990 secs
  DNS-lookup:   0.0000 secs, 0.0000 secs, 0.0000 secs
  req write:    0.0001 secs, 0.0001 secs, 0.0008 secs
  resp wait:    0.0013 secs, 0.0009 secs, 0.0464 secs
  resp read:    0.0018 secs, 0.0006 secs, 0.0230 secs

Status code distribution:
  [200] 1000 responses

Clearly the configure flag changes for wolfSSL paid off.

privoxy-experimental-mbedtls 3.0.32.20210216 with mbedtls 2.16.9_6

[fk@benchmark-vm ~]$ hey -x http://172.16.1.7:8118 -n 1000 -c 1 -t 0 -disable-keepalive https://p.p/user-manual/

Summary:
  Total:        320.8540 secs
  Slowest:      2.2626 secs
  Fastest:      0.1000 secs
  Average:      0.3208 secs
  Requests/sec: 3.1167

  Total data:   21587000 bytes
  Size/request: 21587 bytes

Response time histogram:
  0.100 [1]     |
  0.316 [589]   |________________________________________
  0.532 [314]   |_____________________
  0.749 [72]    |_____
  0.965 [21]    |_
  1.181 [2]     |
  1.398 [0]     |
  1.614 [0]     |
  1.830 [0]     |
  2.046 [0]     |
  2.263 [1]     |


Latency distribution:
  10% in 0.1527 secs
  25% in 0.2001 secs
  50% in 0.2826 secs
  75% in 0.4036 secs
  90% in 0.5273 secs
  95% in 0.6218 secs
  99% in 0.8662 secs

Details (average, fastest, slowest):
  DNS+dialup:   0.3149 secs, 0.1000 secs, 2.2626 secs
  DNS-lookup:   0.0000 secs, 0.0000 secs, 0.0000 secs
  req write:    0.0009 secs, 0.0001 secs, 0.0431 secs
  resp wait:    0.0019 secs, 0.0009 secs, 0.1673 secs
  resp read:    0.0029 secs, 0.0002 secs, 0.1259 secs

Status code distribution:
  [200] 1000 responses

Connection: keep-alive, concurrency level 1000

privoxy-experimental 3.0.32.20210216 with OpenSSL 1.0.2t-ElectroBSD from base

[fk@benchmark-vm ~]$ hey -x http://172.16.1.7:8118 -n 10000 -c 1000 -t 0 https://www.electrobsd.org/

Summary:
  Total:        142.3040 secs
  Slowest:      60.2315 secs
  Fastest:      0.0119 secs
  Average:      9.2436 secs
  Requests/sec: 70.2721


Response time histogram:
  0.012 [1]     |
  6.034 [3364]  |____________________________
  12.056 [4818] |________________________________________
  18.078 [1301] |___________
  24.100 [217]  |__
  30.122 [55]   |
  36.144 [10]   |
  42.166 [1]    |
  48.188 [0]    |
  54.210 [51]   |
  60.232 [182]  |__


Latency distribution:
  10% in 3.5249 secs
  25% in 4.1903 secs
  50% in 7.4471 secs
  75% in 10.2701 secs
  90% in 17.0103 secs
  95% in 18.5290 secs
  99% in 54.6801 secs

Details (average, fastest, slowest):
  DNS+dialup:   0.0981 secs, 0.0119 secs, 60.2315 secs
  DNS-lookup:   0.0000 secs, 0.0000 secs, 0.0000 secs
  req write:    0.0002 secs, 0.0000 secs, 0.1230 secs
  resp wait:    1.1306 secs, 0.0082 secs, 24.3279 secs
  resp read:    0.0019 secs, 0.0001 secs, 5.1493 secs

Status code distribution:
  [200] 10000 responses

privoxy-experimental-libressl 3.0.32.20210216 with LibreSSL 3.2.3_2

[fk@benchmark-vm ~]$ hey -x http://172.16.1.7:8118 -n 10000 -c 1000 -t 0 https://www.electrobsd.org/

Summary:
  Total:        42.1387 secs
  Slowest:      41.8936 secs
  Fastest:      0.0109 secs
  Average:      3.8552 secs
  Requests/sec: 237.3115


Response time histogram:
  0.011 [1]     |
  4.199 [8660]  |________________________________________
  8.387 [716]   |___
  12.576 [548]  |___
  16.764 [15]   |
  20.952 [5]    |
  25.141 [24]   |
  29.329 [2]    |
  33.517 [3]    |
  37.705 [11]   |
  41.894 [15]   |


Latency distribution:
  10% in 2.8191 secs
  25% in 3.0436 secs
  50% in 3.3098 secs
  75% in 3.5999 secs
  90% in 5.9010 secs
  95% in 9.0816 secs
  99% in 10.1696 secs

Details (average, fastest, slowest):
  DNS+dialup:   0.0110 secs, 0.0109 secs, 41.8936 secs
  DNS-lookup:   0.0000 secs, 0.0000 secs, 0.0000 secs
  req write:    0.0002 secs, 0.0000 secs, 0.1999 secs
  resp wait:    0.3520 secs, 0.0083 secs, 41.0545 secs
  resp read:    0.0016 secs, 0.0001 secs, 0.1989 secs

Status code distribution:
  [200] 10000 responses

privoxy-experimental-libressl-devel 3.0.32.20210216 with LibreSSL devel 3.3.1

[fk@benchmark-vm ~]$ hey -x http://172.16.1.7:8118 -n 10000 -c 1000 -t 0 https://www.electrobsd.org/

Summary:
  Total:        54.8621 secs
  Slowest:      35.7601 secs
  Fastest:      0.0175 secs
  Average:      4.5901 secs
  Requests/sec: 182.2751


Response time histogram:
  0.018 [1]     |
  3.592 [2239]  |_____________
  7.166 [6899]  |________________________________________
  10.740 [669]  |____
  14.315 [41]   |
  17.889 [88]   |_
  21.463 [16]   |
  25.037 [18]   |
  28.612 [10]   |
  32.186 [13]   |
  35.760 [6]    |


Latency distribution:
  10% in 3.1132 secs
  25% in 3.6593 secs
  50% in 3.9741 secs
  75% in 4.8641 secs
  90% in 6.8332 secs
  95% in 8.6111 secs
  99% in 16.4391 secs

Details (average, fastest, slowest):
  DNS+dialup:   0.0245 secs, 0.0175 secs, 35.7601 secs
  DNS-lookup:   0.0000 secs, 0.0000 secs, 0.0000 secs
  req write:    0.0003 secs, 0.0000 secs, 0.2310 secs
  resp wait:    0.4633 secs, 0.0083 secs, 34.4497 secs
  resp read:    0.0018 secs, 0.0001 secs, 0.3882 secs

Status code distribution:
  [200] 10000 responses

privoxy-experimental-ports-openssl 3.0.32.20210216 with OpenSSL 1.1.1j,1

[fk@benchmark-vm ~]$ hey -x http://172.16.1.7:8118 -n 10000 -c 1000 -t 0 https://www.electrobsd.org/

Summary:
  Total:        54.5644 secs
  Slowest:      25.7727 secs
  Fastest:      0.0150 secs
  Average:      4.7654 secs
  Requests/sec: 183.2696


Response time histogram:
  0.015 [1]     |
  2.591 [975]   |______
  5.167 [6311]  |________________________________________
  7.742 [1544]  |__________
  10.318 [768]  |_____
  12.894 [222]  |_
  15.470 [51]   |
  18.045 [50]   |
  20.621 [32]   |
  23.197 [23]   |
  25.773 [23]   |


Latency distribution:
  10% in 2.6062 secs
  25% in 3.2676 secs
  50% in 3.8216 secs
  75% in 5.3538 secs
  90% in 8.1412 secs
  95% in 10.0502 secs
  99% in 17.0874 secs

Details (average, fastest, slowest):
  DNS+dialup:   0.0199 secs, 0.0150 secs, 25.7727 secs
  DNS-lookup:   0.0000 secs, 0.0000 secs, 0.0000 secs
  req write:    0.0004 secs, 0.0000 secs, 0.4120 secs
  resp wait:    0.4434 secs, 0.0082 secs, 24.1244 secs
  resp read:    0.0016 secs, 0.0001 secs, 0.4653 secs

Status code distribution:
  [200] 10000 responses

privoxy-experimental-ports-openssl-devel 3.0.32.20210216 with OpenSSL devel 3.0.0.a11

[fk@benchmark-vm ~]$ hey -x http://172.16.1.7:8118 -n 10000 -c 1000 -t 0 https://www.electrobsd.org/

Summary:
  Total:        43.0776 secs
  Slowest:      32.3190 secs
  Fastest:      0.0118 secs
  Average:      4.0326 secs
  Requests/sec: 232.1393


Response time histogram:
  0.012 [1]     |
  3.243 [2945]  |_________________
  6.473 [6818]  |________________________________________
  9.704 [135]   |_
  12.935 [24]   |
  16.165 [20]   |
  19.396 [11]   |
  22.627 [24]   |
  25.858 [10]   |
  29.088 [7]    |
  32.319 [5]    |


Latency distribution:
  10% in 2.9814 secs
  25% in 3.1775 secs
  50% in 3.7742 secs
  75% in 4.6033 secs
  90% in 5.5741 secs
  95% in 6.0325 secs
  99% in 9.7961 secs

Details (average, fastest, slowest):
  DNS+dialup:   0.0092 secs, 0.0118 secs, 32.3190 secs
  DNS-lookup:   0.0000 secs, 0.0000 secs, 0.0000 secs
  req write:    0.0003 secs, 0.0000 secs, 0.4390 secs
  resp wait:    0.3160 secs, 0.0082 secs, 31.6838 secs
  resp read:    0.0025 secs, 0.0001 secs, 9.0801 secs

Status code distribution:
  [200] 10000 responses

privoxy-experimental-wolfssl 3.0.32.20210216 with wolfSSL 4.7.0_2

[fk@benchmark-vm ~]$ hey -x http://172.16.1.7:8118 -n 10000 -c 1000 -t 0 https://www.electrobsd.org/

Summary:
  Total:        78.1368 secs
  Slowest:      49.5334 secs
  Fastest:      0.0125 secs
  Average:      7.0439 secs
  Requests/sec: 127.9807


Response time histogram:
  0.012 [1]     |
  4.965 [5003]  |________________________________________
  9.917 [2875]  |_______________________
  14.869 [793]  |______
  19.821 [900]  |_______
  24.773 [80]   |_
  29.725 [161]  |_
  34.677 [57]   |
  39.629 [28]   |
  44.581 [21]   |
  49.533 [81]   |_


Latency distribution:
  10% in 1.6025 secs
  25% in 2.5724 secs
  50% in 4.9548 secs
  75% in 7.9658 secs
  90% in 16.5569 secs
  95% in 19.1906 secs
  99% in 39.6885 secs

Details (average, fastest, slowest):
  DNS+dialup:   0.0878 secs, 0.0125 secs, 49.5334 secs
  DNS-lookup:   0.0000 secs, 0.0000 secs, 0.0000 secs
  req write:    0.0002 secs, 0.0000 secs, 0.0688 secs
  resp wait:    1.5907 secs, 0.0081 secs, 46.9209 secs
  resp read:    0.0019 secs, 0.0001 secs, 1.6332 secs

Status code distribution:
  [200] 10000 responses

privoxy-experimental-mbedtls 3.0.32.20210216 with mbedtls 2.16.9_6

[fk@benchmark-vm ~]$ hey -x http://172.16.1.7:8118 -n 10000 -c 1000 -t 0 https://www.electrobsd.org/

Summary:
  Total:        387.9665 secs
  Slowest:      291.5755 secs
  Fastest:      0.0087 secs
  Average:      34.6771 secs
  Requests/sec: 25.7754

  Total data:   899589 bytes
  Size/request: 99 bytes

Response time histogram:
  0.009 [1]     |
  29.165 [6664] |________________________________________
  58.322 [549]  |___
  87.479 [727]  |____
  116.635 [160] |_
  145.792 [153] |_
  174.949 [79]  |
  204.105 [162] |_
  233.262 [506] |___
  262.419 [52]  |
  291.575 [18]  |


Latency distribution:
  10% in 0.5065 secs
  25% in 2.3675 secs
  50% in 4.2593 secs
  75% in 34.5339 secs
  90% in 127.3195 secs
  95% in 215.5636 secs
  99% in 229.7423 secs

Details (average, fastest, slowest):
  DNS+dialup:   4.1781 secs, 0.0087 secs, 291.5755 secs
  DNS-lookup:   0.0000 secs, 0.0000 secs, 0.0000 secs
  req write:    0.1664 secs, 0.0000 secs, 30.5812 secs
  resp wait:    19.2848 secs, 0.0083 secs, 206.5032 secs
  resp read:    0.0796 secs, 0.0001 secs, 19.7582 secs

Status code distribution:
  [200] 8945 responses
  [502] 23 responses
  [503] 103 responses

Error distribution:
  [929] Get "https://www.electrobsd.org/": context deadline exceeded

Privoxy CPU time and resident memory use according to top after the test

TLS libraryCPU timeMemory size
OpenSSL 1.0.2t1:33478M
LibreSSL 3.2.3_21:07415M
LibreSSL devel 3.3.11:17452M
OpenSSL 1.1.1j,11:13562M
OpenSSL devel 3.0.0.a112:12432M
wolfSSL 4.7.0_21:39704M
mbedtls 2.16.9_65:15800M

Connection: close, concurrency level 1000

privoxy-experimental 3.0.32.20210216 with OpenSSL 1.0.2t-ElectroBSD from base

[fk@benchmark-vm ~]$ hey -x http://172.16.1.7:8118 -n 10000 -c 1000 -t 0 -disable-keepalive https://www.electrobsd.org/

Summary:
  Total:        462.4011 secs
  Slowest:      100.0522 secs
  Fastest:      0.1048 secs
  Average:      41.1673 secs
  Requests/sec: 21.6262

  Total data:   204015 bytes
  Size/request: 20 bytes

Response time histogram:
  0.105 [1]     |
  10.100 [153]  |__
  20.094 [99]   |_
  30.089 [769]  |________
  40.084 [3313] |____________________________________
  50.078 [3722] |________________________________________
  60.073 [1339] |______________
  70.068 [293]  |___
  80.063 [64]   |_
  90.057 [5]    |
  100.052 [2]   |


Latency distribution:
  10% in 29.9606 secs
  25% in 34.2167 secs
  50% in 41.5538 secs
  75% in 47.0471 secs
  90% in 54.9308 secs
  95% in 59.2613 secs
  99% in 65.2045 secs

Details (average, fastest, slowest):
  DNS+dialup:   35.6897 secs, 0.1048 secs, 100.0522 secs
  DNS-lookup:   0.0000 secs, 0.0000 secs, 0.0000 secs
  req write:    0.0002 secs, 0.0000 secs, 0.0416 secs
  resp wait:    5.4755 secs, 0.0875 secs, 58.7199 secs
  resp read:    0.0020 secs, 0.0001 secs, 0.4180 secs

Status code distribution:
  [200] 9731 responses
  [503] 29 responses

Error distribution:
  [167] Get "https://www.electrobsd.org/": context deadline exceeded
  [73]  Get "https://www.electrobsd.org/": proxyconnect tcp: dial tcp 172.16.1.7:8118: connect: operation timed out

privoxy-experimental-libressl 3.0.32.20210216 with LibreSSL 3.2.3_2

[fk@benchmark-vm ~]$ hey -x http://172.16.1.7:8118 -n 10000 -c 1000 -t 0 -disable-keepalive https://www.electrobsd.org/

Summary:
  Total:        463.9217 secs
  Slowest:      109.9825 secs
  Fastest:      0.0975 secs
  Average:      41.0221 secs
  Requests/sec: 21.5554

  Total data:   225120 bytes
  Size/request: 23 bytes

Response time histogram:
  0.097 [1]     |
  11.086 [229]  |__
  22.075 [346]  |___
  33.063 [604]  |____
  44.052 [5508] |________________________________________
  55.040 [1948] |______________
  66.029 [858]  |______
  77.017 [145]  |_
  88.006 [8]    |
  98.994 [21]   |
  109.983 [1]   |


Latency distribution:
  10% in 31.8813 secs
  25% in 36.1928 secs
  50% in 39.0768 secs
  75% in 48.2456 secs
  90% in 55.4362 secs
  95% in 60.2266 secs
  99% in 69.5408 secs

Details (average, fastest, slowest):
  DNS+dialup:   36.5161 secs, 0.0975 secs, 109.9825 secs
  DNS-lookup:   0.0000 secs, 0.0000 secs, 0.0000 secs
  req write:    0.0002 secs, 0.0000 secs, 0.1034 secs
  resp wait:    4.5031 secs, 0.0834 secs, 62.3988 secs
  resp read:    0.0026 secs, 0.0001 secs, 0.7376 secs

Status code distribution:
  [200] 9637 responses
  [503] 32 responses

Error distribution:
  [271] Get "https://www.electrobsd.org/": context deadline exceeded
  [60]  Get "https://www.electrobsd.org/": proxyconnect tcp: dial tcp 172.16.1.7:8118: connect: operation timed out

privoxy-experimental-libressl-devel 3.0.32.20210216 with LibreSSL devel 3.3.1

[fk@benchmark-vm ~]$ hey -x http://172.16.1.7:8118 -n 10000 -c 1000 -t 0 -disable-keepalive https://www.electrobsd.org/

Summary:
  Total:        494.6845 secs
  Slowest:      114.8360 secs
  Fastest:      0.1857 secs
  Average:      44.0541 secs
  Requests/sec: 20.2149

  Total data:   710535 bytes
  Size/request: 73 bytes

Response time histogram:
  0.186 [1]     |
  11.651 [253]  |__
  23.116 [51]   |
  34.581 [1259] |__________
  46.046 [5174] |________________________________________
  57.511 [1329] |__________
  68.976 [1183] |_________
  80.441 [102]  |_
  91.906 [160]  |_
  103.371 [121] |_
  114.836 [9]   |


Latency distribution:
  10% in 32.7068 secs
  25% in 36.2907 secs
  50% in 40.5670 secs
  75% in 50.4742 secs
  90% in 62.2500 secs
  95% in 66.2989 secs
  99% in 95.0129 secs

Details (average, fastest, slowest):
  DNS+dialup:   38.1360 secs, 0.1857 secs, 114.8360 secs
  DNS-lookup:   0.0000 secs, 0.0000 secs, 0.0000 secs
  req write:    0.0004 secs, 0.0000 secs, 0.1633 secs
  resp wait:    5.9142 secs, 0.0813 secs, 62.8772 secs
  resp read:    0.0035 secs, 0.0001 secs, 0.6897 secs

Status code distribution:
  [200] 9541 responses
  [503] 101 responses

Error distribution:
  [303] Get "https://www.electrobsd.org/": context deadline exceeded
  [55]  Get "https://www.electrobsd.org/": proxyconnect tcp: dial tcp 172.16.1.7:8118: connect: operation timed out

privoxy-experimental-ports-openssl 3.0.32.20210216 with OpenSSL 1.1.1j,1

[fk@benchmark-vm ~]$ hey -x http://172.16.1.7:8118 -n 10000 -c 1000 -t 0 -disable-keepalive https://www.electrobsd.org/

Summary:
  Total:        532.5850 secs
  Slowest:      129.8773 secs
  Fastest:      0.1238 secs
  Average:      47.0465 secs
  Requests/sec: 18.7763

  Total data:   7260120 bytes
  Size/request: 729 bytes

Response time histogram:
  0.124 [1]     |
  13.099 [264]  |___
  26.074 [606]  |______
  39.050 [4150] |________________________________________
  52.025 [3007] |_____________________________
  65.001 [789]  |________
  77.976 [139]  |_
  90.951 [10]   |
  103.927 [111] |_
  116.902 [133] |_
  129.877 [746] |_______


Latency distribution:
  10% in 28.7386 secs
  25% in 33.3900 secs
  50% in 38.8671 secs
  75% in 48.7103 secs
  90% in 81.2245 secs
  95% in 128.1317 secs
  99% in 129.3828 secs

Details (average, fastest, slowest):
  DNS+dialup:   32.5841 secs, 0.1238 secs, 129.8773 secs
  DNS-lookup:   0.0000 secs, 0.0000 secs, 0.0000 secs
  req write:    0.0002 secs, 0.0000 secs, 0.1551 secs
  resp wait:    14.4527 secs, 0.0760 secs, 129.1943 secs
  resp read:    0.0094 secs, 0.0001 secs, 1.1798 secs

Status code distribution:
  [200] 8924 responses
  [503] 1032 responses

Error distribution:
  [23]  Get "https://www.electrobsd.org/": context deadline exceeded
  [21]  Get "https://www.electrobsd.org/": proxyconnect tcp: dial tcp 172.16.1.7:8118: connect: operation timed out

privoxy-experimental-ports-openssl-devel 3.0.32.20210216 with OpenSSL devel 3.0.0.a11

[fk@benchmark-vm ~]$ hey -x http://172.16.1.7:8118 -n 10000 -c 1000 -t 0 -disable-keepalive https://www.electrobsd.org/

Summary:
  Total:        523.1596 secs
  Slowest:      117.2104 secs
  Fastest:      1.1062 secs
  Average:      47.6696 secs
  Requests/sec: 19.1146

  Total data:   316575 bytes
  Size/request: 32 bytes

Response time histogram:
  1.106 [1]     |
  12.717 [153]  |_
  24.327 [296]  |___
  35.937 [159]  |_
  47.548 [4488] |________________________________________
  59.158 [3844] |__________________________________
  70.769 [817]  |_______
  82.379 [60]   |_
  93.990 [8]    |
  105.600 [3]   |
  117.210 [2]   |


Latency distribution:
  10% in 39.0949 secs
  25% in 42.7823 secs
  50% in 47.1008 secs
  75% in 55.3380 secs
  90% in 58.9738 secs
  95% in 60.5852 secs
  99% in 68.4617 secs

Details (average, fastest, slowest):
  DNS+dialup:   43.6173 secs, 1.1062 secs, 117.2104 secs
  DNS-lookup:   0.0000 secs, 0.0000 secs, 0.0000 secs
  req write:    0.0002 secs, 0.0000 secs, 0.0275 secs
  resp wait:    4.0496 secs, 0.1013 secs, 41.0136 secs
  resp read:    0.0024 secs, 0.0001 secs, 0.7430 secs

Status code distribution:
  [200] 9786 responses
  [503] 45 responses

Error distribution:
  [88]  Get "https://www.electrobsd.org/": context deadline exceeded
  [81]  Get "https://www.electrobsd.org/": proxyconnect tcp: dial tcp 172.16.1.7:8118: connect: operation timed out

privoxy-experimental-wolfssl 3.0.32.20210216 with wolfSSL 4.7.0_2

[fk@benchmark-vm ~]$ hey -x http://172.16.1.7:8118 -n 10000 -c 1000 -t 0 -disable-keepalive https://www.electrobsd.org/

Summary:
  Total:        347.0963 secs
  Slowest:      92.0937 secs
  Fastest:      0.0660 secs
  Average:      28.6340 secs
  Requests/sec: 28.8105

  Total data:   513555 bytes
  Size/request: 51 bytes

Response time histogram:
  0.066 [1]     |
  9.269 [379]   |__
  18.472 [92]   |_
  27.674 [6341] |________________________________________
  36.877 [1724] |___________
  46.080 [559]  |____
  55.283 [209]  |_
  64.485 [308]  |__
  73.688 [261]  |__
  82.891 [2]    |
  92.094 [7]    |


Latency distribution:
  10% in 21.9224 secs
  25% in 23.7595 secs
  50% in 25.6125 secs
  75% in 29.0265 secs
  90% in 41.5099 secs
  95% in 62.4491 secs
  99% in 65.2665 secs

Details (average, fastest, slowest):
  DNS+dialup:   23.0281 secs, 0.0660 secs, 92.0937 secs
  DNS-lookup:   0.0000 secs, 0.0000 secs, 0.0000 secs
  req write:    0.0039 secs, 0.0000 secs, 0.4656 secs
  resp wait:    5.5940 secs, 0.0575 secs, 41.0118 secs
  resp read:    0.0079 secs, 0.0001 secs, 1.0702 secs

Status code distribution:
  [200] 9810 responses
  [503] 73 responses

Error distribution:
  [117] Get "https://www.electrobsd.org/": proxyconnect tcp: dial tcp 172.16.1.7:8118: connect: operation timed out

privoxy-experimental-mbedtls 3.0.32.20210216 with mbedtls 2.16.9_6

[fk@benchmark-vm ~]$ hey -x http://172.16.1.7:8118 -n 10000 -c 1000 -t 0 -disable-keepalive https://www.electrobsd.org/

Summary:
  Total:        1066.2614 secs
  Slowest:      432.5674 secs
  Fastest:      0.1520 secs
  Average:      72.9039 secs
  Requests/sec: 9.3786

  Total data:   928620 bytes
  Size/request: 93 bytes

Response time histogram:
  0.152 [1]     |
  43.394 [1172] |_______
  86.635 [6598] |________________________________________
  129.877 [2104]        |_____________
  173.118 [4]   |
  216.360 [0]   |
  259.601 [0]   |
  302.843 [0]   |
  346.084 [0]   |
  389.326 [29]  |
  432.567 [53]  |


Latency distribution:
  10% in 40.3065 secs
  25% in 58.8633 secs
  50% in 75.8038 secs
  75% in 85.3442 secs
  90% in 92.2249 secs
  95% in 97.8203 secs
  99% in 112.2393 secs

Details (average, fastest, slowest):
  DNS+dialup:   58.6916 secs, 0.1520 secs, 432.5674 secs
  DNS-lookup:   0.0000 secs, 0.0000 secs, 0.0000 secs
  req write:    0.4757 secs, 0.0000 secs, 10.2244 secs
  resp wait:    13.3211 secs, 0.0702 secs, 363.8382 secs
  resp read:    0.4153 secs, 0.0002 secs, 12.5637 secs

Status code distribution:
  [200] 9829 responses
  [503] 132 responses

Error distribution:
  [39]  Get "https://www.electrobsd.org/": proxyconnect tcp: dial tcp 172.16.1.7:8118: connect: operation timed out

Privoxy CPU time and resident memory use according to top after the test

TLS libraryCPU timeMemory size
OpenSSL 1.0.2t6:21381M
LibreSSL 3.2.3_26:21413M
LibreSSL devel 3.3.16:56367M
OpenSSL 1.1.1j,17:21561M
OpenSSL devel 3.0.0.a117:23284M
wolfSSL 4.7.0_24:11278M
mbedtls 2.16.9_68:05315M

2021-02-19: Privoxy 3.0.32 experimental 2021-02-16 679b708984 tested with hey

The OpenSSL devel port has been updated to 3.0.0.a12 and privoxy-experimental-ports-openssl-devel 3.0.32.20210216 has been recompiled.

Connection: keep-alive, blocked page, concurrency level 500

In this test https://ads.electrobsd/ is requested and blocked by Privoxy. Connections are reused. Concurrency is reduced to 500 as generating the block page requires more memory.

privoxy-experimental 3.0.32.20210216 with OpenSSL 1.0.2t-ElectroBSD from base

[fk@benchmark-vm ~]$ hey -x http://172.16.1.7:8118 -n 10000 -c 500 -t 0 https://ads.electrobsd.org/

Summary:
  Total:        56.5757 secs
  Slowest:      17.3687 secs
  Fastest:      0.0022 secs
  Average:      1.5680 secs
  Requests/sec: 176.7542

  Total data:   92320000 bytes
  Size/request: 9232 bytes

Response time histogram:
  0.002 [1]     |
  1.739 [7401]  |________________________________________
  3.475 [877]   |_____
  5.212 [854]   |_____
  6.949 [326]   |__
  8.685 [286]   |__
  10.422 [163]  |_
  12.159 [63]   |
  13.895 [16]   |
  15.632 [6]    |
  17.369 [7]    |


Latency distribution:
  10% in 0.0062 secs
  25% in 0.0926 secs
  50% in 0.5230 secs
  75% in 1.9266 secs
  90% in 4.8700 secs
  95% in 7.3835 secs
  99% in 10.2803 secs

Details (average, fastest, slowest):
  DNS+dialup:   0.0112 secs, 0.0022 secs, 17.3687 secs
  DNS-lookup:   0.0000 secs, 0.0000 secs, 0.0000 secs
  req write:    0.0001 secs, 0.0000 secs, 0.0232 secs
  resp wait:    1.2270 secs, 0.0016 secs, 16.2375 secs
  resp read:    0.0092 secs, 0.0002 secs, 5.7104 secs

Status code distribution:
  [403] 10000 responses

privoxy-experimental-libressl 3.0.32.20210216 with LibreSSL 3.2.3_2

[fk@benchmark-vm ~]$ hey -x http://172.16.1.7:8118 -n 10000 -c 500 -t 0 https://ads.electrobsd.org/

Summary:
  Total:        32.2621 secs
  Slowest:      32.0170 secs
  Fastest:      0.0019 secs
  Average:      1.2910 secs
  Requests/sec: 309.9613

  Total data:   92320000 bytes
  Size/request: 9232 bytes

Response time histogram:
  0.002 [1]     |
  3.203 [9622]  |________________________________________
  6.405 [90]    |
  9.606 [154]   |_
  12.808 [27]   |
  16.009 [0]    |
  19.211 [20]   |
  22.412 [67]   |
  25.614 [1]    |
  28.815 [0]    |
  32.017 [18]   |


Latency distribution:
  10% in 0.0038 secs
  25% in 0.5269 secs
  50% in 0.9290 secs
  75% in 1.2685 secs
  90% in 1.9006 secs
  95% in 2.3871 secs
  99% in 18.1151 secs

Details (average, fastest, slowest):
  DNS+dialup:   0.0065 secs, 0.0019 secs, 32.0170 secs
  DNS-lookup:   0.0000 secs, 0.0000 secs, 0.0000 secs
  req write:    0.0002 secs, 0.0000 secs, 0.0366 secs
  resp wait:    0.5585 secs, 0.0015 secs, 31.4776 secs
  resp read:    0.0067 secs, 0.0000 secs, 8.0812 secs

Status code distribution:
  [403] 10000 responses

privoxy-experimental-libressl-devel 3.0.32.20210216 with LibreSSL devel 3.3.1

[fk@benchmark-vm ~]$ hey -x http://172.16.1.7:8118 -n 10000 -c 500 -t 0 https://ads.electrobsd.org/

Summary:
  Total:        29.8085 secs
  Slowest:      29.5272 secs
  Fastest:      0.0020 secs
  Average:      1.1402 secs
  Requests/sec: 335.4750

  Total data:   92320000 bytes
  Size/request: 9232 bytes

Response time histogram:
  0.002 [1]     |
  2.954 [9646]  |________________________________________
  5.907 [27]    |
  8.860 [29]    |
  11.812 [17]   |
  14.765 [46]   |
  17.717 [48]   |
  20.670 [86]   |
  23.622 [80]   |
  26.575 [12]   |
  29.527 [8]    |


Latency distribution:
  10% in 0.0023 secs
  25% in 0.0051 secs
  50% in 0.7164 secs
  75% in 0.9514 secs
  90% in 1.3964 secs
  95% in 2.1237 secs
  99% in 20.7004 secs

Details (average, fastest, slowest):
  DNS+dialup:   0.0112 secs, 0.0020 secs, 29.5272 secs
  DNS-lookup:   0.0000 secs, 0.0000 secs, 0.0000 secs
  req write:    0.0002 secs, 0.0000 secs, 0.0451 secs
  resp wait:    0.5848 secs, 0.0015 secs, 28.7065 secs
  resp read:    0.0021 secs, 0.0000 secs, 1.3987 secs

Status code distribution:
  [403] 10000 responses

privoxy-experimental-ports-openssl 3.0.32.20210216 with OpenSSL 1.1.1j,1

[fk@benchmark-vm ~]$ hey -x http://172.16.1.7:8118 -n 10000 -c 500 -t 0 https://ads.electrobsd.org/

Summary:
  Total:        43.1997 secs
  Slowest:      33.1239 secs
  Fastest:      0.0019 secs
  Average:      1.3374 secs
  Requests/sec: 231.4831

  Total data:   92320000 bytes
  Size/request: 9232 bytes

Response time histogram:
  0.002 [1]     |
  3.314 [9340]  |________________________________________
  6.626 [365]   |__
  9.939 [127]   |_
  13.251 [46]   |
  16.563 [44]   |
  19.875 [48]   |
  23.187 [1]    |
  26.499 [0]    |
  29.812 [3]    |
  33.124 [25]   |


Latency distribution:
  10% in 0.0036 secs
  25% in 0.3672 secs
  50% in 0.7277 secs
  75% in 1.2631 secs
  90% in 2.7113 secs
  95% in 3.8144 secs
  99% in 16.0184 secs

Details (average, fastest, slowest):
  DNS+dialup:   0.0108 secs, 0.0019 secs, 33.1239 secs
  DNS-lookup:   0.0000 secs, 0.0000 secs, 0.0000 secs
  req write:    0.0002 secs, 0.0000 secs, 0.1298 secs
  resp wait:    0.6927 secs, 0.0015 secs, 32.5261 secs
  resp read:    0.0086 secs, 0.0000 secs, 5.3432 secs

Status code distribution:
  [403] 10000 responses

privoxy-experimental-ports-openssl-devel 3.0.32.20210216 with OpenSSL devel 3.0.0.a12

[fk@benchmark-vm ~]$ hey -x http://172.16.1.7:8118 -n 10000 -c 500 -t 0 https://ads.electrobsd.org/

Summary:
  Total:        36.9314 secs
  Slowest:      36.2355 secs
  Fastest:      0.0019 secs
  Average:      1.5423 secs
  Requests/sec: 270.7725

  Total data:   92320000 bytes
  Size/request: 9232 bytes

Response time histogram:
  0.002 [1]     |
  3.625 [9474]  |________________________________________
  7.249 [180]   |_
  10.872 [275]  |_
  14.495 [0]    |
  18.119 [1]    |
  21.742 [0]    |
  25.365 [16]   |
  28.989 [0]    |
  32.612 [3]    |
  36.236 [50]   |


Latency distribution:
  10% in 0.0135 secs
  25% in 0.8419 secs
  50% in 1.0043 secs
  75% in 1.5145 secs
  90% in 2.0303 secs
  95% in 6.5174 secs
  99% in 8.0359 secs

Details (average, fastest, slowest):
  DNS+dialup:   0.0050 secs, 0.0019 secs, 36.2355 secs
  DNS-lookup:   0.0000 secs, 0.0000 secs, 0.0000 secs
  req write:    0.0002 secs, 0.0000 secs, 0.0205 secs
  resp wait:    0.4955 secs, 0.0015 secs, 35.6578 secs
  resp read:    0.0075 secs, 0.0000 secs, 2.2044 secs

Status code distribution:
  [403] 10000 responses

privoxy-experimental-wolfssl 3.0.32.20210216 with wolfSSL 4.7.0_2

[fk@benchmark-vm ~]$ hey -x http://172.16.1.7:8118 -n 10000 -c 500 -t 0 https://ads.electrobsd.org/

Summary:
  Total:        34.4870 secs
  Slowest:      20.2775 secs
  Fastest:      0.0022 secs
  Average:      1.4665 secs
  Requests/sec: 289.9647

  Total data:   92320000 bytes
  Size/request: 9232 bytes

Response time histogram:
  0.002 [1]     |
  2.030 [8535]  |________________________________________
  4.057 [781]   |____
  6.085 [315]   |_
  8.112 [107]   |_
  10.140 [115]  |_
  12.167 [33]   |
  14.195 [8]    |
  16.222 [47]   |
  18.250 [55]   |
  20.278 [3]    |


Latency distribution:
  10% in 0.1146 secs
  25% in 0.6212 secs
  50% in 0.9245 secs
  75% in 1.2583 secs
  90% in 3.2655 secs
  95% in 4.7252 secs
  99% in 14.7234 secs

Details (average, fastest, slowest):
  DNS+dialup:   0.0335 secs, 0.0022 secs, 20.2775 secs
  DNS-lookup:   0.0000 secs, 0.0000 secs, 0.0000 secs
  req write:    0.0001 secs, 0.0000 secs, 0.0173 secs
  resp wait:    0.6001 secs, 0.0015 secs, 19.3268 secs
  resp read:    0.0058 secs, 0.0002 secs, 2.8281 secs

Status code distribution:
  [403] 10000 responses

privoxy-experimental-mbedtls 3.0.32.20210216 with mbedtls 2.16.9_6

[fk@benchmark-vm ~]$ hey -x http://172.16.1.7:8118 -n 10000 -c 500 -t 0 https://ads.electrobsd.org/

Summary:
  Total:        59.4872 secs
  Slowest:      37.5913 secs
  Fastest:      0.0019 secs
  Average:      2.7007 secs
  Requests/sec: 168.1035

  Total data:   92320000 bytes
  Size/request: 9232 bytes

Response time histogram:
  0.002 [1]     |
  3.761 [8436]  |________________________________________
  7.520 [726]   |___
  11.279 [283]  |_
  15.038 [51]   |
  18.797 [28]   |
  22.556 [12]   |
  26.314 [60]   |
  30.073 [121]  |_
  33.832 [264]  |_
  37.591 [18]   |


Latency distribution:
  10% in 0.0027 secs
  25% in 0.0537 secs
  50% in 0.3232 secs
  75% in 2.1293 secs
  90% in 6.4967 secs
  95% in 17.1389 secs
  99% in 31.3368 secs

Details (average, fastest, slowest):
  DNS+dialup:   0.2968 secs, 0.0019 secs, 37.5913 secs
  DNS-lookup:   0.0000 secs, 0.0000 secs, 0.0000 secs
  req write:    0.0146 secs, 0.0000 secs, 2.6626 secs
  resp wait:    1.1153 secs, 0.0015 secs, 12.9543 secs
  resp read:    0.0015 secs, 0.0000 secs, 1.3732 secs

Status code distribution:
  [403] 10000 responses

Privoxy CPU time and resident memory use according to top after the test

TLS libraryCPU timeMemory size
OpenSSL 1.0.2t1:00569M
LibreSSL 3.2.3_20:51644M
LibreSSL devel 3.3.10:44670M
OpenSSL 1.1.1j,11:04728M
OpenSSL devel 3.0.0.a121:21599M
wolfSSL 4.7.0_20:32387M
mbedtls 2.16.9_60:45111M

Connection: close, blocked page, concurrency level 500

Again the blocked https://ads.electrobsd/ is requested, this time without reusing connections.

privoxy-experimental 3.0.32.20210216 with OpenSSL 1.0.2t-ElectroBSD from base (maybe not?)

[fk@benchmark-vm ~]$ hey -x http://172.16.1.7:8118 -n 10000 -c 500 -t 0 -disable-keepalive https://ads.electrobsd.org/

Summary:
  Total:        506.8149 secs
  Slowest:      74.6107 secs
  Fastest:      1.1547 secs
  Average:      24.9626 secs
  Requests/sec: 19.7311

  Total data:   92320000 bytes
  Size/request: 9232 bytes

Response time histogram:
  1.155 [1]     |
  8.500 [73]    |_
  15.846 [171]  |_
  23.192 [3563] |___________________________
  30.537 [5222] |________________________________________
  37.883 [351]  |___
  45.228 [348]  |___
  52.574 [246]  |__
  59.920 [10]   |
  67.265 [0]    |
  74.611 [15]   |


Latency distribution:
  10% in 20.3134 secs
  25% in 21.5414 secs
  50% in 24.1415 secs
  75% in 26.0044 secs
  90% in 30.3611 secs
  95% in 39.9107 secs
  99% in 49.1163 secs

Details (average, fastest, slowest):
  DNS+dialup:   20.1254 secs, 1.1547 secs, 74.6107 secs
  DNS-lookup:   0.0000 secs, 0.0000 secs, 0.0000 secs
  req write:    0.4114 secs, 0.0000 secs, 2.9489 secs
  resp wait:    4.0431 secs, 0.0037 secs, 24.9994 secs
  resp read:    0.3802 secs, 0.0001 secs, 3.8255 secs

Status code distribution:
  [403] 10000 responses

privoxy-experimental-libressl 3.0.32.20210216 with LibreSSL 3.2.3_2

[fk@benchmark-vm ~]$ hey -x http://172.16.1.7:8118 -n 10000 -c 500 -t 0 -disable-keepalive https://ads.electrobsd.org/

Summary:
  Total:        182.8982 secs
  Slowest:      58.4864 secs
  Fastest:      0.0087 secs
  Average:      5.0819 secs
  Requests/sec: 54.6752

  Total data:   92052272 bytes
  Size/request: 9232 bytes

Response time histogram:
  0.009 [1]     |
  5.856 [8742]  |________________________________________
  11.704 [1222] |______
  17.552 [0]    |
  23.400 [0]    |
  29.248 [2]    |
  35.095 [0]    |
  40.943 [2]    |
  46.791 [0]    |
  52.639 [0]    |
  58.486 [2]    |


Latency distribution:
  10% in 3.9290 secs
  25% in 5.0435 secs
  50% in 5.2697 secs
  75% in 5.5730 secs
  90% in 5.9476 secs
  95% in 6.2688 secs
  99% in 7.3882 secs

Details (average, fastest, slowest):
  DNS+dialup:   4.9526 secs, 0.0087 secs, 58.4864 secs
  DNS-lookup:   0.0000 secs, 0.0000 secs, 0.0000 secs
  req write:    0.0011 secs, 0.0000 secs, 0.3014 secs
  resp wait:    0.1250 secs, 0.0017 secs, 2.3706 secs
  resp read:    0.0032 secs, 0.0001 secs, 0.8135 secs

Status code distribution:
  [403] 9971 responses

Error distribution:
  [29]  Get "https://ads.electrobsd.org/": proxyconnect tcp: dial tcp 172.16.1.7:8118: connect: operation timed out

privoxy-experimental-libressl-devel 3.0.32.20210216 with LibreSSL devel 3.3.1

[fk@benchmark-vm ~]$ hey -x http://172.16.1.7:8118 -n 10000 -c 500 -t 0 -disable-keepalive https://ads.electrobsd.org/

Summary:
  Total:        183.2557 secs
  Slowest:      63.8003 secs
  Fastest:      0.0088 secs
  Average:      5.2874 secs
  Requests/sec: 54.5686

  Total data:   92190752 bytes
  Size/request: 9232 bytes

Response time histogram:
  0.009 [1]     |
  6.388 [8788]  |________________________________________
  12.767 [1192] |_____
  19.146 [1]    |
  25.525 [0]    |
  31.905 [0]    |
  38.284 [0]    |
  44.663 [3]    |
  51.042 [0]    |
  57.421 [0]    |
  63.800 [1]    |


Latency distribution:
  10% in 3.6121 secs
  25% in 4.9986 secs
  50% in 5.4536 secs
  75% in 5.7698 secs
  90% in 6.5397 secs
  95% in 7.2812 secs
  99% in 8.4554 secs

Details (average, fastest, slowest):
  DNS+dialup:   4.9393 secs, 0.0088 secs, 63.8003 secs
  DNS-lookup:   0.0000 secs, 0.0000 secs, 0.0000 secs
  req write:    0.0009 secs, 0.0000 secs, 0.0832 secs
  resp wait:    0.3439 secs, 0.0017 secs, 3.0059 secs
  resp read:    0.0032 secs, 0.0001 secs, 0.6683 secs

Status code distribution:
  [403] 9986 responses

Error distribution:
  [14]  Get "https://ads.electrobsd.org/": proxyconnect tcp: dial tcp 172.16.1.7:8118: connect: operation timed out

privoxy-experimental-ports-openssl 3.0.32.20210216 with OpenSSL 1.1.1j,1

[fk@benchmark-vm ~]$ hey -x http://172.16.1.7:8118 -n 10000 -c 500 -t 0 -disable-keepalive https://ads.electrobsd.org/

Summary:
  Total:        160.6947 secs
  Slowest:      58.4727 secs
  Fastest:      0.0092 secs
  Average:      5.3160 secs
  Requests/sec: 62.2298

  Total data:   92190752 bytes
  Size/request: 9232 bytes

Response time histogram:
  0.009 [1]     |
  5.856 [7818]  |________________________________________
  11.702 [2156] |___________
  17.548 [0]    |
  23.395 [3]    |
  29.241 [1]    |
  35.087 [0]    |
  40.934 [4]    |
  46.780 [0]    |
  52.626 [0]    |
  58.473 [3]    |


Latency distribution:
  10% in 3.8861 secs
  25% in 5.0280 secs
  50% in 5.4429 secs
  75% in 5.7679 secs
  90% in 6.7195 secs
  95% in 7.3885 secs
  99% in 9.1674 secs

Details (average, fastest, slowest):
  DNS+dialup:   4.8543 secs, 0.0092 secs, 58.4727 secs
  DNS-lookup:   0.0000 secs, 0.0000 secs, 0.0000 secs
  req write:    0.0012 secs, 0.0000 secs, 0.1361 secs
  resp wait:    0.4562 secs, 0.0020 secs, 4.1427 secs
  resp read:    0.0041 secs, 0.0001 secs, 1.0762 secs

Status code distribution:
  [403] 9986 responses

Error distribution:
  [14]  Get "https://ads.electrobsd.org/": proxyconnect tcp: dial tcp 172.16.1.7:8118: connect: operation timed out

privoxy-experimental-ports-openssl-devel 3.0.32.20210216 with OpenSSL devel 3.0.0.a12

[fk@benchmark-vm ~]$ hey -x http://172.16.1.7:8118 -n 10000 -c 500 -t 0 -disable-keepalive https://ads.electrobsd.org/

Summary:
  Total:        158.0386 secs
  Slowest:      41.6518 secs
  Fastest:      0.0137 secs
  Average:      7.6646 secs
  Requests/sec: 63.2757

  Total data:   92301536 bytes
  Size/request: 9232 bytes

Response time histogram:
  0.014 [1]     |
  4.178 [445]   |___
  8.341 [6622]  |________________________________________
  12.505 [2807] |_________________
  16.669 [122]  |_
  20.833 [0]    |
  24.997 [0]    |
  29.160 [0]    |
  33.324 [0]    |
  37.488 [0]    |
  41.652 [1]    |


Latency distribution:
  10% in 5.7120 secs
  25% in 6.7147 secs
  50% in 7.4759 secs
  75% in 8.5631 secs
  90% in 10.2541 secs
  95% in 11.2007 secs
  99% in 13.5611 secs

Details (average, fastest, slowest):
  DNS+dialup:   6.6690 secs, 0.0137 secs, 41.6518 secs
  DNS-lookup:   0.0000 secs, 0.0000 secs, 0.0000 secs
  req write:    0.0010 secs, 0.0000 secs, 0.1767 secs
  resp wait:    0.9755 secs, 0.0022 secs, 6.5447 secs
  resp read:    0.0190 secs, 0.0001 secs, 3.5642 secs

Status code distribution:
  [403] 9998 responses

Error distribution:
  [2]   Get "https://ads.electrobsd.org/": proxyconnect tcp: dial tcp 172.16.1.7:8118: connect: operation timed out

privoxy-experimental-wolfssl 3.0.32.20210216 with wolfSSL 4.7.0_2

[fk@benchmark-vm ~]$ hey -x http://172.16.1.7:8118 -n 10000 -c 500 -t 0 -disable-keepalive https://ads.electrobsd.org/

Summary:
  Total:        228.6437 secs
  Slowest:      58.4782 secs
  Fastest:      0.0079 secs
  Average:      4.2100 secs
  Requests/sec: 43.7362

  Total data:   90076624 bytes
  Size/request: 9232 bytes

Response time histogram:
  0.008 [1]     |
  5.855 [8476]  |________________________________________
  11.702 [1270] |______
  17.549 [1]    |
  23.396 [0]    |
  29.243 [0]    |
  35.090 [7]    |
  40.937 [0]    |
  46.784 [0]    |
  52.631 [0]    |
  58.478 [2]    |


Latency distribution:
  10% in 1.0798 secs
  25% in 3.2238 secs
  50% in 4.6062 secs
  75% in 5.3328 secs
  90% in 6.0933 secs
  95% in 6.6897 secs
  99% in 7.3967 secs

Details (average, fastest, slowest):
  DNS+dialup:   3.6402 secs, 0.0079 secs, 58.4782 secs
  DNS-lookup:   0.0000 secs, 0.0000 secs, 0.0000 secs
  req write:    0.0008 secs, 0.0000 secs, 0.1308 secs
  resp wait:    0.5608 secs, 0.0017 secs, 3.4118 secs
  resp read:    0.0082 secs, 0.0002 secs, 1.2066 secs

Status code distribution:
  [403] 9757 responses

Error distribution:
  [243] Get "https://ads.electrobsd.org/": proxyconnect tcp: dial tcp 172.16.1.7:8118: connect: operation timed out

privoxy-experimental-mbedtls 3.0.32.20210216 with mbedtls 2.16.9_6

[fk@benchmark-vm ~]$ hey -x http://172.16.1.7:8118 -n 10000 -c 500 -t 0 -disable-keepalive https://ads.electrobsd.org/

Summary:
  Total:        512.6566 secs
  Slowest:      62.0966 secs
  Fastest:      0.1080 secs
  Average:      25.0630 secs
  Requests/sec: 19.5062

  Total data:   92301536 bytes
  Size/request: 9232 bytes

Response time histogram:
  0.108 [1]     |
  6.307 [24]    |
  12.506 [109]  |_
  18.705 [284]  |__
  24.903 [5274] |________________________________________
  31.102 [3479] |__________________________
  37.301 [289]  |__
  43.500 [313]  |__
  49.699 [133]  |_
  55.898 [90]   |_
  62.097 [2]    |


Latency distribution:
  10% in 20.9160 secs
  25% in 21.7648 secs
  50% in 24.4792 secs
  75% in 25.8387 secs
  90% in 29.4425 secs
  95% in 38.0777 secs
  99% in 49.4553 secs

Details (average, fastest, slowest):
  DNS+dialup:   20.2104 secs, 0.1080 secs, 62.0966 secs
  DNS-lookup:   0.0000 secs, 0.0000 secs, 0.0000 secs
  req write:    0.4061 secs, 0.0000 secs, 2.5067 secs
  resp wait:    4.0624 secs, 0.0019 secs, 31.0060 secs
  resp read:    0.3785 secs, 0.0002 secs, 5.1205 secs

Status code distribution:
  [403] 9998 responses

Error distribution:
  [2]   Get "https://ads.electrobsd.org/": proxyconnect tcp: dial tcp 172.16.1.7:8118: connect: operation timed out

Privoxy CPU time and resident memory use according to top after the test

TLS libraryCPU timeMemory size
OpenSSL 1.0.2t (maybe not?)5:22187M
LibreSSL 3.2.3_21:38230M
LibreSSL devel 3.3.11:41264M
OpenSSL 1.1.1j,11:41191M
OpenSSL devel 3.0.0.a122:21277M
wolfSSL 4.7.0_21:23292M
mbedtls 2.16.9_64:42172M

The results for OpenSSL 1.0.2t are clearly suspect. I wonder if I they can be explained by changed background noise on the host of if I accidentally continued to test with mbedtls 2.16.9_6 installed from a previous run.

2021-02-20: Privoxy 3.0.32 experimental 2021-02-16 679b708984 tested with hey

The wolfssl port has been updated to include --enable-intelasm and then updated again to remove the configure flag as the resulting binaries don't work on the test system. privoxy-experimental-wolfssl 3.0.32.20210216_1 has been recompiled. Twice.

As an experiment I configured Privoxy to enable the HTTP accept filter to see if it makes a difference for CONNECT requests. The option can only impact new connections so I didn't runt the tests again with keep-alive enabled.

Connection: close, blocked page, concurrency level 500

privoxy-experimental 3.0.32.20210216 with OpenSSL 1.0.2t-ElectroBSD from base

[fk@benchmark-vm ~]$ hey -x http://172.16.1.7:8118 -n 10000 -c 500 -t 0 -disable-keepalive https://ads.electrobsd.org/

Summary:
  Total:        196.7352 secs
  Slowest:      67.1195 secs
  Fastest:      0.0106 secs
  Average:      8.3404 secs
  Requests/sec: 50.8297

  Total data:   92264608 bytes
  Size/request: 9232 bytes

Response time histogram:
  0.011 [1]     |
  6.721 [1330]  |______
  13.432 [8546] |________________________________________
  20.143 [115]  |_
  26.854 [0]    |
  33.565 [0]    |
  40.276 [0]    |
  46.987 [0]    |
  53.698 [0]    |
  60.409 [0]    |
  67.120 [2]    |


Latency distribution:
  10% in 6.2528 secs
  25% in 7.3807 secs
  50% in 8.3670 secs
  75% in 9.3534 secs
  90% in 10.7378 secs
  95% in 11.5283 secs
  99% in 13.5000 secs

Details (average, fastest, slowest):
  DNS+dialup:   7.5155 secs, 0.0106 secs, 67.1195 secs
  DNS-lookup:   0.0000 secs, 0.0000 secs, 0.0000 secs
  req write:    0.0021 secs, 0.0000 secs, 0.5894 secs
  resp wait:    0.8038 secs, 0.0019 secs, 4.2702 secs
  resp read:    0.0187 secs, 0.0002 secs, 1.8232 secs

Status code distribution:
  [403] 9994 responses

Error distribution:
  [6]   Get "https://ads.electrobsd.org/": proxyconnect tcp: dial tcp 172.16.1.7:8118: connect: operation timed out

privoxy-experimental-libressl 3.0.32.20210216 with LibreSSL 3.2.3_2

[fk@benchmark-vm ~]$ hey -x http://172.16.1.7:8118 -n 10000 -c 500 -t 0 -disable-keepalive https://ads.electrobsd.org/

Summary:
  Total:        219.6350 secs
  Slowest:      16.0899 secs
  Fastest:      0.0106 secs
  Average:      7.7134 secs
  Requests/sec: 45.5301

  Total data:   92209216 bytes
  Size/request: 9232 bytes

Response time histogram:
  0.011 [1]     |
  1.618 [114]   |_
  3.226 [282]   |___
  4.834 [208]   |__
  6.442 [694]   |______
  8.050 [4309]  |________________________________________
  9.658 [3419]  |________________________________
  11.266 [844]  |________
  12.874 [43]   |
  14.482 [50]   |
  16.090 [24]   |


Latency distribution:
  10% in 6.1966 secs
  25% in 7.0436 secs
  50% in 7.8535 secs
  75% in 8.5956 secs
  90% in 9.6273 secs
  95% in 10.1041 secs
  99% in 11.4976 secs

Details (average, fastest, slowest):
  DNS+dialup:   7.1040 secs, 0.0106 secs, 16.0899 secs
  DNS-lookup:   0.0000 secs, 0.0000 secs, 0.0000 secs
  req write:    0.0018 secs, 0.0000 secs, 0.2055 secs
  resp wait:    0.5928 secs, 0.0019 secs, 7.1156 secs
  resp read:    0.0146 secs, 0.0002 secs, 3.8758 secs

Status code distribution:
  [403] 9988 responses

Error distribution:
  [12]  Get "https://ads.electrobsd.org/": proxyconnect tcp: dial tcp 172.16.1.7:8118: connect: operation timed out

privoxy-experimental-libressl-devel 3.0.32.20210216 with LibreSSL devel 3.3.1

[fk@benchmark-vm ~]$ hey -x http://172.16.1.7:8118 -n 10000 -c 500 -t 0 -disable-keepalive https://ads.electrobsd.org/

Summary:
  Total:        229.1791 secs
  Slowest:      66.5862 secs
  Fastest:      0.0102 secs
  Average:      7.4500 secs
  Requests/sec: 43.6340

  Total data:   91895328 bytes
  Size/request: 9232 bytes

Response time histogram:
  0.010 [1]     |
  6.668 [2700]  |_______________
  13.325 [7116] |________________________________________
  19.983 [51]   |
  26.641 [11]   |
  33.298 [7]    |
  39.956 [9]    |
  46.613 [11]   |
  53.271 [2]    |
  59.929 [29]   |
  66.586 [17]   |


Latency distribution:
  10% in 3.6489 secs
  25% in 6.5623 secs
  50% in 7.5110 secs
  75% in 8.3671 secs
  90% in 9.3577 secs
  95% in 10.1053 secs
  99% in 19.6965 secs

Details (average, fastest, slowest):
  DNS+dialup:   6.9290 secs, 0.0102 secs, 66.5862 secs
  DNS-lookup:   0.0000 secs, 0.0000 secs, 0.0000 secs
  req write:    0.0019 secs, 0.0000 secs, 0.4935 secs
  resp wait:    0.5125 secs, 0.0019 secs, 7.8039 secs
  resp read:    0.0064 secs, 0.0001 secs, 3.2971 secs

Status code distribution:
  [403] 9954 responses

Error distribution:
  [46]  Get "https://ads.electrobsd.org/": proxyconnect tcp: dial tcp 172.16.1.7:8118: connect: operation timed out

privoxy-experimental-ports-openssl 3.0.32.20210216 with OpenSSL 1.1.1j,1

[fk@benchmark-vm ~]$ hey -x http://172.16.1.7:8118 -n 10000 -c 500 -t 0 -disable-keepalive https://ads.electrobsd.org/

Summary:
  Total:        175.7938 secs
  Slowest:      68.1560 secs
  Fastest:      0.0358 secs
  Average:      8.3704 secs
  Requests/sec: 56.8848

  Total data:   92227680 bytes
  Size/request: 9232 bytes

Response time histogram:
  0.036 [1]     |
  6.848 [1356]  |______
  13.660 [8580] |________________________________________
  20.472 [44]   |
  27.284 [1]    |
  34.096 [0]    |
  40.908 [0]    |
  47.720 [3]    |
  54.532 [0]    |
  61.344 [0]    |
  68.156 [5]    |


Latency distribution:
  10% in 6.3961 secs
  25% in 7.5922 secs
  50% in 8.4238 secs
  75% in 9.4106 secs
  90% in 10.5465 secs
  95% in 11.0188 secs
  99% in 12.6088 secs

Details (average, fastest, slowest):
  DNS+dialup:   7.6406 secs, 0.0358 secs, 68.1560 secs
  DNS-lookup:   0.0000 secs, 0.0000 secs, 0.0000 secs
  req write:    0.0055 secs, 0.0000 secs, 1.3088 secs
  resp wait:    0.7137 secs, 0.0022 secs, 7.5887 secs
  resp read:    0.0104 secs, 0.0001 secs, 4.9130 secs

Status code distribution:
  [403] 9990 responses

Error distribution:
  [10]  Get "https://ads.electrobsd.org/": proxyconnect tcp: dial tcp 172.16.1.7:8118: connect: operation timed out

privoxy-experimental-ports-openssl-devel 3.0.32.20210216 with OpenSSL devel 3.0.0.a12

[fk@benchmark-vm ~]$ hey -x http://172.16.1.7:8118 -n 10000 -c 500 -t 0 -disable-keepalive https://ads.electrobsd.org/

Summary:
  Total:        237.6310 secs
  Slowest:      58.4496 secs
  Fastest:      0.0145 secs
  Average:      11.3357 secs
  Requests/sec: 42.0820

  Total data:   92273840 bytes
  Size/request: 9232 bytes

Response time histogram:
  0.014 [1]     |
  5.858 [335]   |__
  11.702 [5422] |________________________________________
  17.545 [4122] |______________________________
  23.389 [111]  |_
  29.232 [3]    |
  35.076 [0]    |
  40.919 [0]    |
  46.763 [0]    |
  52.606 [0]    |
  58.450 [1]    |


Latency distribution:
  10% in 8.7541 secs
  25% in 9.9750 secs
  50% in 11.2747 secs
  75% in 12.7297 secs
  90% in 14.4656 secs
  95% in 15.5462 secs
  99% in 17.7227 secs

Details (average, fastest, slowest):
  DNS+dialup:   10.1225 secs, 0.0145 secs, 58.4496 secs
  DNS-lookup:   0.0000 secs, 0.0000 secs, 0.0000 secs
  req write:    0.0024 secs, 0.0000 secs, 0.2212 secs
  resp wait:    1.1980 secs, 0.0021 secs, 7.3346 secs
  resp read:    0.0126 secs, 0.0001 secs, 3.0961 secs

Status code distribution:
  [403] 9995 responses

Error distribution:
  [5]   Get "https://ads.electrobsd.org/": proxyconnect tcp: dial tcp 172.16.1.7:8118: connect: operation timed out

privoxy-experimental-wolfssl 3.0.32.20210216_1 with wolfSSL 4.7.0_4

[fk@benchmark-vm ~]$ hey -x http://172.16.1.7:8118 -n 10000 -c 500 -t 0 -disable-keepalive https://ads.electrobsd.org/

Summary:
  Total:        234.1005 secs
  Slowest:      12.5116 secs
  Fastest:      0.0100 secs
  Average:      7.4811 secs
  Requests/sec: 42.7167

  Total data:   92190752 bytes
  Size/request: 9232 bytes

Response time histogram:
  0.010 [1]     |
  1.260 [139]   |__
  2.510 [133]   |__
  3.760 [114]   |_
  5.011 [281]   |___
  6.261 [849]   |__________
  7.511 [3262]  |________________________________________
  8.761 [3283]  |________________________________________
  10.011 [1405] |_________________
  11.261 [495]  |______
  12.512 [24]   |


Latency distribution:
  10% in 5.7744 secs
  25% in 6.7763 secs
  50% in 7.5677 secs
  75% in 8.5206 secs
  90% in 9.4748 secs
  95% in 10.0378 secs
  99% in 10.8210 secs

Details (average, fastest, slowest):
  DNS+dialup:   6.4370 secs, 0.0100 secs, 12.5116 secs
  DNS-lookup:   0.0000 secs, 0.0000 secs, 0.0000 secs
  req write:    0.0040 secs, 0.0000 secs, 0.5011 secs
  resp wait:    1.0207 secs, 0.0020 secs, 4.1652 secs
  resp read:    0.0192 secs, 0.0003 secs, 0.9506 secs

Status code distribution:
  [403] 9986 responses

Error distribution:
  [14]  Get "https://ads.electrobsd.org/": proxyconnect tcp: dial tcp 172.16.1.7:8118: connect: operation timed out

privoxy-experimental-mbedtls 3.0.32.20210216 with mbedtls 2.16.9_6

[fk@benchmark-vm ~]$ hey -x http://172.16.1.7:8118 -n 10000 -c 500 -t 0 -disable-keepalive https://ads.electrobsd.org/

Summary:
  Total:        681.7736 secs
  Slowest:      90.7298 secs
  Fastest:      0.2496 secs
  Average:      33.5403 secs
  Requests/sec: 14.6676

  Total data:   92310768 bytes
  Size/request: 9232 bytes

Response time histogram:
  0.250 [1]     |
  9.298 [63]    |
  18.346 [151]  |_
  27.394 [1104] |_______
  36.442 [6191] |________________________________________
  45.490 [1965] |_____________
  54.538 [101]  |_
  63.586 [327]  |__
  72.634 [93]   |_
  81.682 [2]    |
  90.730 [1]    |


Latency distribution:
  10% in 26.1072 secs
  25% in 29.3204 secs
  50% in 33.0000 secs
  75% in 36.4320 secs
  90% in 40.0730 secs
  95% in 47.3563 secs
  99% in 62.9468 secs

Details (average, fastest, slowest):
  DNS+dialup:   27.9099 secs, 0.2496 secs, 90.7298 secs
  DNS-lookup:   0.0000 secs, 0.0000 secs, 0.0000 secs
  req write:    0.5864 secs, 0.0000 secs, 17.8840 secs
  resp wait:    4.4436 secs, 0.0028 secs, 35.6120 secs
  resp read:    0.5844 secs, 0.0001 secs, 9.0368 secs

Status code distribution:
  [403] 9999 responses

Error distribution:
  [1]   Get "https://ads.electrobsd.org/": proxyconnect tcp: dial tcp 172.16.1.7:8118: connect: operation timed out

Privoxy CPU time and resident memory use according to top after the test

TLS libraryCPU timeMemory size
OpenSSL 1.0.2t2:12316M
LibreSSL 3.2.3_23:44500M
LibreSSL devel 3.3.12:17279M
OpenSSL 1.1.1j,12:17204M
OpenSSL devel 3.0.0.a123:09259M
wolfSSL 4.7.0_42:05263M
mbedtls 2.16.9_65:27137M

The results for OpenSSL 1.0.2t look reasonable again.

Connection: close, blocked page, concurrency level 500. Once more, with feeling

I repeated the test with the same Privoxy configuration but shutdown a bhyve machine first that was started on 2020-02-19.

privoxy-experimental 3.0.32.20210216 with OpenSSL 1.0.2t-ElectroBSD from base

[fk@benchmark-vm ~]$ hey -x http://172.16.1.7:8118 -n 10000 -c 500 -t 0 -disable-keepalive https://ads.electrobsd.org/

Summary:
  Total:        229.1192 secs
  Slowest:      14.7169 secs
  Fastest:      0.0103 secs
  Average:      7.2841 secs
  Requests/sec: 43.6454

  Total data:   92107664 bytes
  Size/request: 9232 bytes

Response time histogram:
  0.010 [1]     |
  1.481 [204]   |__
  2.952 [117]   |_
  4.422 [246]   |___
  5.893 [883]   |__________
  7.364 [3435]  |________________________________________
  8.834 [3440]  |________________________________________
  10.305 [1224] |______________
  11.776 [356]  |____
  13.246 [55]   |_
  14.717 [16]   |


Latency distribution:
  10% in 5.1309 secs
  25% in 6.5465 secs
  50% in 7.3978 secs
  75% in 8.2579 secs
  90% in 9.3621 secs
  95% in 10.1418 secs
  99% in 11.3833 secs

Details (average, fastest, slowest):
  DNS+dialup:   6.5283 secs, 0.0103 secs, 14.7169 secs
  DNS-lookup:   0.0000 secs, 0.0000 secs, 0.0000 secs
  req write:    0.0022 secs, 0.0000 secs, 0.1999 secs
  resp wait:    0.7431 secs, 0.0019 secs, 6.5206 secs
  resp read:    0.0102 secs, 0.0002 secs, 0.7722 secs

Status code distribution:
  [403] 9977 responses

Error distribution:
  [23]  Get "https://ads.electrobsd.org/": proxyconnect tcp: dial tcp 172.16.1.7:8118: connect: operation timed out

privoxy-experimental-libressl 3.0.32.20210216 with LibreSSL 3.2.3_2

[fk@benchmark-vm ~]$ hey -x http://172.16.1.7:8118 -n 10000 -c 500 -t 0 -disable-keepalive https://ads.electrobsd.org/

Summary:
  Total:        201.5610 secs
  Slowest:      67.3539 secs
  Fastest:      0.0105 secs
  Average:      8.5295 secs
  Requests/sec: 49.6128

  Total data:   92209216 bytes
  Size/request: 9232 bytes

Response time histogram:
  0.010 [1]     |
  6.745 [1674]  |_________
  13.479 [7798] |________________________________________
  20.214 [174]  |_
  26.948 [338]  |__
  33.682 [1]    |
  40.417 [0]    |
  47.151 [0]    |
  53.885 [0]    |
  60.620 [0]    |
  67.354 [2]    |


Latency distribution:
  10% in 5.8359 secs
  25% in 7.2752 secs
  50% in 8.1401 secs
  75% in 9.0929 secs
  90% in 10.6214 secs
  95% in 14.3723 secs
  99% in 22.0539 secs

Details (average, fastest, slowest):
  DNS+dialup:   7.6579 secs, 0.0105 secs, 67.3539 secs
  DNS-lookup:   0.0000 secs, 0.0000 secs, 0.0000 secs
  req write:    0.0061 secs, 0.0000 secs, 0.4268 secs
  resp wait:    0.8436 secs, 0.0020 secs, 11.2957 secs
  resp read:    0.0217 secs, 0.0001 secs, 6.1869 secs

Status code distribution:
  [403] 9988 responses

Error distribution:
  [12]  Get "https://ads.electrobsd.org/": proxyconnect tcp: dial tcp 172.16.1.7:8118: connect: operation timed out

privoxy-experimental-libressl-devel 3.0.32.20210216 with LibreSSL devel 3.3.1

[fk@benchmark-vm ~]$ hey -x http://172.16.1.7:8118 -n 10000 -c 500 -t 0 -disable-keepalive https://ads.electrobsd.org/

Summary:
  Total:        224.9795 secs
  Slowest:      59.2200 secs
  Fastest:      0.0105 secs
  Average:      6.1113 secs
  Requests/sec: 44.4485

  Total data:   90547456 bytes
  Size/request: 9232 bytes

Response time histogram:
  0.011 [1]     |
  5.931 [3293]  |____________________
  11.852 [6458] |________________________________________
  17.773 [21]   |
  23.694 [0]    |
  29.615 [22]   |
  35.536 [4]    |
  41.457 [5]    |
  47.378 [0]    |
  53.299 [0]    |
  59.220 [4]    |


Latency distribution:
  10% in 0.8811 secs
  25% in 5.5868 secs
  50% in 6.5774 secs
  75% in 7.7550 secs
  90% in 8.7496 secs
  95% in 9.2552 secs
  99% in 11.1063 secs

Details (average, fastest, slowest):
  DNS+dialup:   5.6280 secs, 0.0105 secs, 59.2200 secs
  DNS-lookup:   0.0000 secs, 0.0000 secs, 0.0000 secs
  req write:    0.0030 secs, 0.0000 secs, 0.3329 secs
  resp wait:    0.4721 secs, 0.0020 secs, 6.7668 secs
  resp read:    0.0080 secs, 0.0001 secs, 0.6305 secs

Status code distribution:
  [403] 9808 responses

Error distribution:
  [192] Get "https://ads.electrobsd.org/": proxyconnect tcp: dial tcp 172.16.1.7:8118: connect: operation timed out

privoxy-experimental-ports-openssl 3.0.32.20210216 with OpenSSL 1.1.1j,1

[fk@benchmark-vm ~]$ hey -x http://172.16.1.7:8118 -n 10000 -c 500 -t 0 -disable-keepalive https://ads.electrobsd.org/

Summary:
  Total:        206.1265 secs
  Slowest:      67.3193 secs
  Fastest:      0.0117 secs
  Average:      8.4849 secs
  Requests/sec: 48.5139

  Total data:   92126128 bytes
  Size/request: 9232 bytes

Response time histogram:
  0.012 [1]     |
  6.742 [1077]  |_____
  13.473 [8823] |________________________________________
  20.204 [73]   |
  26.935 [1]    |
  33.666 [0]    |
  40.396 [0]    |
  47.127 [1]    |
  53.858 [0]    |
  60.589 [0]    |
  67.319 [3]    |


Latency distribution:
  10% in 6.6289 secs
  25% in 7.7299 secs
  50% in 8.6040 secs
  75% in 9.5682 secs
  90% in 10.3730 secs
  95% in 11.1952 secs
  99% in 13.1969 secs

Details (average, fastest, slowest):
  DNS+dialup:   7.7028 secs, 0.0117 secs, 67.3193 secs
  DNS-lookup:   0.0000 secs, 0.0000 secs, 0.0000 secs
  req write:    0.0043 secs, 0.0000 secs, 0.5182 secs
  resp wait:    0.7650 secs, 0.0020 secs, 8.2382 secs
  resp read:    0.0126 secs, 0.0001 secs, 0.8065 secs

Status code distribution:
  [403] 9979 responses

Error distribution:
  [21]  Get "https://ads.electrobsd.org/": proxyconnect tcp: dial tcp 172.16.1.7:8118: connect: operation timed out

privoxy-experimental-ports-openssl-devel 3.0.32.20210216 with OpenSSL devel 3.0.0.a12

[fk@benchmark-vm ~]$ hey -x http://172.16.1.7:8118 -n 10000 -c 500 -t 0 -disable-keepalive https://ads.electrobsd.org/

Summary:
  Total:        277.7110 secs
  Slowest:      73.4518 secs
  Fastest:      0.0144 secs
  Average:      9.9900 secs
  Requests/sec: 36.0087

  Total data:   91138304 bytes
  Size/request: 9232 bytes

Response time histogram:
  0.014 [1]     |
  7.358 [1778]  |_________
  14.702 [7571] |________________________________________
  22.046 [479]  |___
  29.389 [5]    |
  36.733 [19]   |
  44.077 [10]   |
  51.421 [1]    |
  58.764 [0]    |
  66.108 [0]    |
  73.452 [8]    |


Latency distribution:
  10% in 4.5968 secs
  25% in 8.2476 secs
  50% in 10.1133 secs
  75% in 12.2018 secs
  90% in 13.9013 secs
  95% in 14.7525 secs
  99% in 17.2616 secs

Details (average, fastest, slowest):
  DNS+dialup:   8.9541 secs, 0.0144 secs, 73.4518 secs
  DNS-lookup:   0.0000 secs, 0.0000 secs, 0.0000 secs
  req write:    0.0024 secs, 0.0000 secs, 0.2722 secs
  resp wait:    1.0195 secs, 0.0021 secs, 8.3752 secs
  resp read:    0.0137 secs, 0.0001 secs, 0.8761 secs

Status code distribution:
  [403] 9872 responses

Error distribution:
  [128] Get "https://ads.electrobsd.org/": proxyconnect tcp: dial tcp 172.16.1.7:8118: connect: operation timed out

privoxy-experimental-wolfssl 3.0.32.20210216_1 with wolfSSL 4.7.0_4

[fk@benchmark-vm ~]$ hey -x http://172.16.1.7:8118 -n 10000 -c 500 -t 0 -disable-keepalive https://ads.electrobsd.org/

Summary:
  Total:        243.3433 secs
  Slowest:      66.2667 secs
  Fastest:      0.0092 secs
  Average:      7.1743 secs
  Requests/sec: 41.0942

  Total data:   91941488 bytes
  Size/request: 9232 bytes

Response time histogram:
  0.009 [1]     |
  6.635 [3403]  |_____________________
  13.261 [6552] |________________________________________
  19.886 [0]    |
  26.512 [0]    |
  33.138 [0]    |
  39.764 [0]    |
  46.389 [1]    |
  53.015 [0]    |
  59.641 [1]    |
  66.267 [1]    |


Latency distribution:
  10% in 5.1104 secs
  25% in 6.2547 secs
  50% in 7.3163 secs
  75% in 8.3779 secs
  90% in 9.4554 secs
  95% in 10.1959 secs
  99% in 11.5535 secs

Details (average, fastest, slowest):
  DNS+dialup:   6.1148 secs, 0.0092 secs, 66.2667 secs
  DNS-lookup:   0.0000 secs, 0.0000 secs, 0.0000 secs
  req write:    0.0042 secs, 0.0000 secs, 0.5503 secs
  resp wait:    1.0366 secs, 0.0021 secs, 4.0304 secs
  resp read:    0.0186 secs, 0.0003 secs, 1.2941 secs

Status code distribution:
  [403] 9959 responses

Error distribution:
  [41]  Get "https://ads.electrobsd.org/": proxyconnect tcp: dial tcp 172.16.1.7:8118: connect: operation timed out

privoxy-experimental-mbedtls 3.0.32.20210216 with mbedtls 2.16.9_6

[fk@benchmark-vm ~]$ hey -x http://172.16.1.7:8118 -n 10000 -c 500 -t 0 -disable-keepalive https://ads.electrobsd.org/

Summary:
  Total:        651.4387 secs
  Slowest:      71.7645 secs
  Fastest:      0.1185 secs
  Average:      31.9575 secs
  Requests/sec: 15.3506

  Total data:   92273840 bytes
  Size/request: 9232 bytes

Response time histogram:
  0.118 [1]     |
  7.283 [42]    |
  14.448 [64]   |_
  21.612 [356]  |___
  28.777 [2539] |_____________________
  35.941 [4943] |________________________________________
  43.106 [1524] |____________
  50.271 [255]  |__
  57.435 [175]  |_
  64.600 [76]   |_
  71.764 [20]   |


Latency distribution:
  10% in 24.8700 secs
  25% in 28.1502 secs
  50% in 31.5823 secs
  75% in 35.0529 secs
  90% in 39.1698 secs
  95% in 43.6228 secs
  99% in 57.2514 secs

Details (average, fastest, slowest):
  DNS+dialup:   26.3336 secs, 0.1185 secs, 71.7645 secs
  DNS-lookup:   0.0000 secs, 0.0000 secs, 0.0000 secs
  req write:    0.5315 secs, 0.0000 secs, 5.1526 secs
  resp wait:    4.4777 secs, 0.0024 secs, 28.6529 secs
  resp read:    0.5911 secs, 0.0001 secs, 6.7086 secs

Status code distribution:
  [403] 9995 responses

Error distribution:
  [5]   Get "https://ads.electrobsd.org/": proxyconnect tcp: dial tcp 172.16.1.7:8118: connect: operation timed out

Privoxy CPU time and resident memory use according to top after the test

TLS libraryCPU timeMemory size
OpenSSL 1.0.2t2:04293M
LibreSSL 3.2.3_22:18299M
LibreSSL devel 3.3.12:21246M
OpenSSL 1.1.1j,12:21178M
OpenSSL devel 3.0.0.a123:09177M
wolfSSL 4.7.0_42:04272M
mbedtls 2.16.9_65:44170M

2021-02-22: Skewed request distribution thanks to go?

It's not obvious to me why both hey and ab-proxy claim that some responses are supposedly really slow and I'm not sure if it's Privoxy's fault.

Also this test (with OpenSSL 1.0.2t):

[fk@benchmark-vm ~]$ hey -x http://172.16.1.7:8118 -n 10000 -c 500 -t 0 https://ads.electrobsd.org/

Summary:
  Total:        60.8107 secs
  Slowest:      38.0051 secs
  Fastest:      0.0026 secs
  Average:      2.9465 secs
  Requests/sec: 164.4446

  Total data:   92320000 bytes
  Size/request: 9232 bytes

Response time histogram:
  0.003 [1]     |
  3.803 [8490]  |________________________________________
  7.603 [1497]  |_______
  11.403 [7]    |
  15.204 [1]    |
  19.004 [0]    |
  22.804 [3]    |
  26.604 [0]    |
  30.405 [0]    |
  34.205 [0]    |
  38.005 [1]    |


Latency distribution:
  10% in 2.0306 secs
  25% in 2.3515 secs
  50% in 2.7079 secs
  75% in 3.1667 secs
  90% in 4.2272 secs
  95% in 5.3383 secs
  99% in 5.7358 secs

Details (average, fastest, slowest):
  DNS+dialup:   0.0011 secs, 0.0026 secs, 38.0051 secs
  DNS-lookup:   0.0000 secs, 0.0000 secs, 0.0000 secs
  req write:    0.0001 secs, 0.0000 secs, 0.0337 secs
  resp wait:    0.1759 secs, 0.0017 secs, 35.1991 secs
  resp read:    0.0037 secs, 0.0002 secs, 2.3429 secs

Status code distribution:
  [403] 10000 responses

resulted in the following request distribution which does not seem ideal:

[fk@privoxy-vm ~]$ privoxy-log-parser.pl --statistics /var/log/privoxy/privoxy.log
Client requests total: 10000
Crunches: 10000 (100.00%)
Blocks: 10000 (100.00%)
Fast redirections: 0 (0.00%)
Connection timeouts: 0 (0.00%)
Connection failures: 0 (0.00%)
Outgoing requests: 0 (0.00%)
Server keep-alive offers: 0 (0.00%)
New outgoing connections: 0 (0.00%)
Reused connections: 0 (0.00%; server offers accepted: 0.00%)
Empty responses: 0 (0.00%)
Empty responses on new connections: 0 (0.00%)
Empty responses on reused connections: 0 (0.00%)
Client connections: 3034
Bytes of content transfered to the client: 92320000
Client requests per connection distribution:
    2995: 1
       2: 330
       1: 20
       1: 381
       1: 229
       1: 255
       1: 327
       1: 277
       1: 219
       1: 201
Enable --show-complete-request-distribution to get less common numbers as well.
Improperly accounted requests: ~2992
Method distribution:
   10000 : GET
Client HTTP versions:
   10000 : HTTP/1.1
HTTP status codes:
   10000 : 403
URL statistics are disabled. Increase --url-statistics-threshold to enable them.
Passed request statistics are disabled. Increase --passed-request-statistics-threshold to enable them.
Host statistics are disabled. Increase --host-statistics-threshold to enable them.

Apparently hey used a single connection to send 2995 of the 10000 requests instead of reusing the connections evenly. This could indicate that hey is preferring some of the connections when reading incoming requests which would skew the results.

I briefly looked at the hey code but I'm not used to debugging go programs and therefore didn't investigate this further.

I should probably try another benchmarking tool written in another language to see how the results differ.

2021-02-25: Privoxy 3.0.32 experimental 2021-02-25 8d72cc25e88

There's work in progress to release Privoxy 3.0.32 so I've updated the ports to include current commits from git a73211c9fa. The Privoxy code hasn't changed much. I've changed the Privoxy configuration to include regression-tests.action.

... tested with privoxy-regression-test

The test times Privoxy-Regression-Test loading and processing various CGI resources through https. Privoxy does not make outgoing connnections.

Privoxy-Regression-Test runs curl commands one at a time, the concurrency should therefore be one and connections are not reused.

The test is executed on the system running Privoxy. Two test failures are currently documented to be expected to occur when using --privoxy-cgi-prefix https://p.p/.

privoxy-experimental 3.0.32.20210225 with OpenSSL 1.0.2t-ElectroBSD from base

[fk@privoxy-vm ~]$ time privoxy-regression-test.pl --privoxy-cgi-prefix https://p.p/
2021-02-25 18:09:10: Asking Privoxy for the number of action files available ...
2021-02-25 18:09:10: Gathering regression tests from 4 action file(s) delivered by Privoxy 3.0.32.
2021-02-25 18:09:10: Executing regression tests ...
2021-02-25 18:09:32: Ooops. Expected removal but: 'Connection: close' is still there.
2021-02-25 18:09:32: Failure for test 574. Header 'Connection: close' and tag 'connection-sharing enabled'
2021-02-25 18:09:32: Ooops. Got: 'Host: p.p' while expecting: 'NO CHANGE'
2021-02-25 18:09:32: Failure for test 577. Header 'Host: whatever.example.org' and tag 'Host header other than the target host'
2021-02-25 18:09:36: Executed 700 regression tests. Skipped 91. 698 successes, 2 failures.

real    0m26.307s
user    0m13.953s
sys     0m5.295s
  

privoxy-experimental-libressl 3.0.32.20210225 with LibreSSL 3.2.3_2

[fk@privoxy-vm ~]$ time privoxy-regression-test.pl --privoxy-cgi-prefix https://p.p/
2021-02-25 18:13:07: Asking Privoxy for the number of action files available ...
2021-02-25 18:13:07: Gathering regression tests from 4 action file(s) delivered by Privoxy 3.0.32.
2021-02-25 18:13:07: Executing regression tests ...
2021-02-25 18:13:32: Ooops. Expected removal but: 'Connection: close' is still there.
2021-02-25 18:13:32: Failure for test 574. Header 'Connection: close' and tag 'connection-sharing enabled'
2021-02-25 18:13:32: Ooops. Got: 'Host: p.p' while expecting: 'NO CHANGE'
2021-02-25 18:13:32: Failure for test 577. Header 'Host: whatever.example.org' and tag 'Host header other than the target host'
2021-02-25 18:13:36: Executed 700 regression tests. Skipped 91. 698 successes, 2 failures.

real    0m29.660s
user    0m13.915s
sys     0m5.379s

privoxy-experimental-libressl-devel 3.0.32.20210225 with LibreSSL devel 3.3.1

[fk@privoxy-vm ~]$ time privoxy-regression-test.pl --privoxy-cgi-prefix https://p.p/
2021-02-25 18:13:07: Asking Privoxy for the number of action files available ...
2021-02-25 18:13:07: Gathering regression tests from 4 action file(s) delivered by Privoxy 3.0.32.
2021-02-25 18:13:07: Executing regression tests ...
2021-02-25 18:13:32: Ooops. Expected removal but: 'Connection: close' is still there.
2021-02-25 18:13:32: Failure for test 574. Header 'Connection: close' and tag 'connection-sharing enabled'
2021-02-25 18:13:32: Ooops. Got: 'Host: p.p' while expecting: 'NO CHANGE'
2021-02-25 18:13:32: Failure for test 577. Header 'Host: whatever.example.org' and tag 'Host header other than the target host'
2021-02-25 18:13:36: Executed 700 regression tests. Skipped 91. 698 successes, 2 failures.

real    0m29.660s
user    0m13.915s
sys     0m5.379s

privoxy-experimental-ports-openssl 3.0.32.20210225 with OpenSSL 1.1.1j,1

[fk@privoxy-vm ~]$ time privoxy-regression-test.pl --privoxy-cgi-prefix https://p.p/
2021-02-25 18:15:08: Asking Privoxy for the number of action files available ...
2021-02-25 18:15:09: Gathering regression tests from 4 action file(s) delivered by Privoxy 3.0.32.
2021-02-25 18:15:09: Executing regression tests ...
2021-02-25 18:15:38: Ooops. Expected removal but: 'Connection: close' is still there.
2021-02-25 18:15:38: Failure for test 574. Header 'Connection: close' and tag 'connection-sharing enabled'
2021-02-25 18:15:38: Ooops. Got: 'Host: p.p' while expecting: 'NO CHANGE'
2021-02-25 18:15:38: Failure for test 577. Header 'Host: whatever.example.org' and tag 'Host header other than the target host'
2021-02-25 18:15:42: Executed 700 regression tests. Skipped 91. 698 successes, 2 failures.

real    0m33.438s
user    0m15.856s
sys     0m5.449s

privoxy-experimental-ports-openssl-devel 3.0.32.20210225 with OpenSSL devel 3.0.0.a12

[fk@privoxy-vm ~]$ time privoxy-regression-test.pl --privoxy-cgi-prefix https://p.p/
2021-02-25 18:38:25: Asking Privoxy for the number of action files available ...
2021-02-25 18:38:25: Gathering regression tests from 4 action file(s) delivered by Privoxy 3.0.32.
2021-02-25 18:38:25: Executing regression tests ...
2021-02-25 18:38:52: Ooops. Expected removal but: 'Connection: close' is still there.
2021-02-25 18:38:52: Failure for test 574. Header 'Connection: close' and tag 'connection-sharing enabled'
2021-02-25 18:38:52: Ooops. Got: 'Host: p.p' while expecting: 'NO CHANGE'
2021-02-25 18:38:52: Failure for test 577. Header 'Host: whatever.example.org' and tag 'Host header other than the target host'
2021-02-25 18:38:56: Executed 700 regression tests. Skipped 91. 698 successes, 2 failures.

real    0m31.408s
user    0m15.492s
sys     0m5.897s

privoxy-experimental-wolfssl 3.0.32.20210225_1 with wolfSSL 4.7.0_4

[fk@privoxy-vm ~]$ time privoxy-regression-test.pl --privoxy-cgi-prefix https://p.p/
2021-02-25 18:18:48: Asking Privoxy for the number of action files available ...
2021-02-25 18:18:48: Gathering regression tests from 4 action file(s) delivered by Privoxy 3.0.32.
2021-02-25 18:18:48: Executing regression tests ...
2021-02-25 18:19:11: Ooops. Expected removal but: 'Connection: close' is still there.
2021-02-25 18:19:11: Failure for test 574. Header 'Connection: close' and tag 'connection-sharing enabled'
2021-02-25 18:19:11: Ooops. Got: 'Host: p.p' while expecting: 'NO CHANGE'
2021-02-25 18:19:11: Failure for test 577. Header 'Host: whatever.example.org' and tag 'Host header other than the target host'
2021-02-25 18:19:14: Executed 700 regression tests. Skipped 91. 698 successes, 2 failures.

real    0m26.378s
user    0m14.430s
sys     0m5.724s

privoxy-experimental-mbedtls 3.0.32.20210225 with mbedtls 2.16.9_6

[fk@privoxy-vm ~]$ time privoxy-regression-test.pl --privoxy-cgi-prefix https://p.p/
2021-02-25 18:20:17: Asking Privoxy for the number of action files available ...
2021-02-25 18:20:17: Gathering regression tests from 4 action file(s) delivered by Privoxy 3.0.32.
2021-02-25 18:20:17: Executing regression tests ...
2021-02-25 18:20:58: Ooops. Expected removal but: 'Connection: close' is still there.
2021-02-25 18:20:58: Failure for test 574. Header 'Connection: close' and tag 'connection-sharing enabled'
2021-02-25 18:20:58: Ooops. Got: 'Host: p.p' while expecting: 'NO CHANGE'
2021-02-25 18:20:58: Failure for test 577. Header 'Host: whatever.example.org' and tag 'Host header other than the target host'
2021-02-25 18:21:04: Executed 700 regression tests. Skipped 91. 698 successes, 2 failures.

real    0m47.105s
user    0m23.516s
sys     0m5.845s

Privoxy CPU time and resident memory use according to top after the test

TLS libraryCPU timeMemory size
OpenSSL 1.0.2t0:0661804K
LibreSSL 3.2.3_20:0962384K
LibreSSL devel 3.3.10:1063388K
OpenSSL 1.1.1j,10:0666108K
OpenSSL devel 3.0.0.a120:0964376K
wolfSSL 4.7.0_40:0563696K
mbedtls 2.16.9_60:1633848K

2021-03-17: Privoxy 3.0.33 experimental 2021-03-17 dfedbde3b4 tested with hey

The Privoxy ports have been updated. One experimental change worth mentioning is that the memory used to temporarily store the certificates used by servers is allocated dynamically which could reduce the peak memory usage for tests where Privoxy does not answer the request itself.

Privoxy's wolfSSL version is expected to profit less from this because it already contains another optimization and only stores the certificates if the certificate chain could not be validated (the only case where the certificates are needed so the user can inspect them).

The OpenSSL devel port has been updated to 3.0.0.a13. The mbedTLS port has been updated to 2.16.10. The wolfSSL port contains a patch to disable ciphers using 3DES.

The ports on the benchmark VM have been upgraded as well: hey 0.1.3 has been replaced with 0.1.4 and go went from 1.14.7,1 to 1.15.6,1.

Privoxy binary size

The binaries are dynamically linked so the sizes aren't significantly affected by the choice of the TLS library.

TLS libraryPrivoxy binary size
OpenSSL 1.0.2t321K
LibreSSL 3.2.3_2321K
LibreSSL devel 3.3.1321K
OpenSSL 1.1.1j,1321K
OpenSSL devel 3.0.0.a13321K
wolfSSL 4.7.0_6317K
mbedtls 2.16.10317K

Connection: close, concurrency level 1000

As hey and go have been upgraded I decided to give them another try to see if the skewed results issue still exists. Spoiler alert: it does.

privoxy-experimental 3.0.33.20210317 with OpenSSL 1.0.2t-ElectroBSD from base

[fk@benchmark-vm ~]$ hey -x http://172.16.1.7:8118 -n 10000 -c 1000 -disable-keepalive -t 0 https://www.electrobsd.org/

Summary:
  Total:        483.1075 secs
  Slowest:      86.0884 secs
  Fastest:      7.6978 secs
  Average:      43.8206 secs
  Requests/sec: 20.6993

  Total data:   204015 bytes
  Size/request: 21 bytes

Response time histogram:
  7.698 [1]     |
  15.537 [53]   |_
  23.376 [38]   |
  31.215 [506]  |_____
  39.054 [2296] |______________________
  46.893 [4093] |________________________________________
  54.732 [713]  |_______
  62.571 [1020] |__________
  70.410 [452]  |____
  78.249 [103]  |_
  86.088 [15]   |


Latency distribution:
  10% in 32.4097 secs
  25% in 37.3129 secs
  50% in 42.6137 secs
  75% in 46.8448 secs
  90% in 60.2343 secs
  95% in 63.6705 secs
  99% in 72.0302 secs

Details (average, fastest, slowest):
  DNS+dialup:   38.6265 secs, 7.6978 secs, 86.0884 secs
  DNS-lookup:   0.0000 secs, 0.0000 secs, 0.0000 secs
  req write:    0.0002 secs, 0.0000 secs, 0.0434 secs
  resp wait:    5.1896 secs, 0.1007 secs, 42.9234 secs
  resp read:    0.0043 secs, 0.0001 secs, 1.8764 secs

Status code distribution:
  [200] 9261 responses
  [503] 29 responses

Error distribution:
  [710] Get "https://www.electrobsd.org/": context deadline exceeded

privoxy-experimental-libressl 3.0.33.20210317 with LibreSSL 3.2.3_2

[fk@benchmark-vm ~]$ hey -x http://172.16.1.7:8118 -n 10000 -c 1000 -disable-keepalive -t 0 https://www.electrobsd.org/

Summary:
  Total:        507.0580 secs
  Slowest:      113.1922 secs
  Fastest:      0.7074 secs
  Average:      45.6618 secs
  Requests/sec: 19.7216

  Total data:   274365 bytes
  Size/request: 27 bytes

Response time histogram:
  0.707 [1]     |
  11.956 [160]  |_
  23.204 [357]  |___
  34.453 [352]  |___
  45.701 [4510] |________________________________________
  56.950 [3242] |_____________________________
  68.198 [1173] |__________
  79.447 [127]  |_
  90.695 [20]   |
  101.944 [9]   |
  113.192 [3]   |


Latency distribution:
  10% in 34.9347 secs
  25% in 39.9600 secs
  50% in 44.6978 secs
  75% in 54.6003 secs
  90% in 57.8763 secs
  95% in 60.9433 secs
  99% in 73.1241 secs

Details (average, fastest, slowest):
  DNS+dialup:   40.9759 secs, 0.7074 secs, 113.1922 secs
  DNS-lookup:   0.0000 secs, 0.0000 secs, 0.0000 secs
  req write:    0.0005 secs, 0.0000 secs, 0.1712 secs
  resp wait:    4.6827 secs, 0.1726 secs, 44.5401 secs
  resp read:    0.0026 secs, 0.0001 secs, 1.4885 secs

Status code distribution:
  [200] 9915 responses
  [503] 39 responses

Error distribution:
  [25]  Get "https://www.electrobsd.org/": context deadline exceeded
  [21]  Get "https://www.electrobsd.org/": proxyconnect tcp: dial tcp 172.16.1.7:8118: connect: operation timed out

privoxy-experimental-libressl-devel 3.0.33.20210317 with LibreSSL devel 3.3.1

[fk@benchmark-vm ~]$ hey -x http://172.16.1.7:8118 -n 10000 -c 1000 -disable-keepalive -t 0 https://www.electrobsd.org/

Summary:
  Total:        732.1350 secs
  Slowest:      326.2141 secs
  Fastest:      0.5917 secs
  Average:      59.4638 secs
  Requests/sec: 13.6587

  Total data:   2076981 bytes
  Size/request: 258 bytes

Response time histogram:
  0.592 [1]     |
  33.154 [572]  |___
  65.716 [6573] |________________________________________
  98.278 [436]  |___
  130.841 [1]   |
  163.403 [0]   |
  195.965 [1]   |
  228.527 [1]   |
  261.090 [1]   |
  293.652 [3]   |
  326.214 [435] |___


Latency distribution:
  10% in 34.8167 secs
  25% in 38.3919 secs
  50% in 45.0731 secs
  75% in 55.3488 secs
  90% in 68.4707 secs
  95% in 298.6087 secs
  99% in 309.1964 secs

Details (average, fastest, slowest):
  DNS+dialup:   41.8909 secs, 0.5917 secs, 326.2141 secs
  DNS-lookup:   0.0000 secs, 0.0000 secs, 0.0000 secs
  req write:    0.0003 secs, 0.0000 secs, 0.0389 secs
  resp wait:    17.5563 secs, 0.1327 secs, 290.8596 secs
  resp read:    0.0162 secs, 0.0001 secs, 26.0727 secs

Status code distribution:
  [200] 7733 responses
  [502] 52 responses
  [503] 239 responses

Error distribution:
  [1963]        Get "https://www.electrobsd.org/": context deadline exceeded
  [13]  Get "https://www.electrobsd.org/": proxyconnect tcp: dial tcp 172.16.1.7:8118: connect: operation timed out

privoxy-experimental-ports-openssl 3.0.33.20210317 with OpenSSL 1.1.1j,1

[fk@benchmark-vm ~]$ hey -x http://172.16.1.7:8118 -n 10000 -c 1000 -disable-keepalive -t 0 https://www.electrobsd.org/

Summary:
  Total:        490.4667 secs
  Slowest:      121.6341 secs
  Fastest:      0.4485 secs
  Average:      42.8682 secs
  Requests/sec: 20.3887

  Total data:   316575 bytes
  Size/request: 34 bytes

Response time histogram:
  0.449 [1]     |
  12.567 [187]  |__
  24.686 [195]  |__
  36.804 [2174] |_______________________
  48.923 [3838] |________________________________________
  61.041 [2325] |________________________
  73.160 [429]  |____
  85.278 [18]   |
  97.397 [13]   |
  109.516 [1]   |
  121.634 [3]   |


Latency distribution:
  10% in 32.1865 secs
  25% in 36.3391 secs
  50% in 41.3034 secs
  75% in 50.4415 secs
  90% in 54.9276 secs
  95% in 61.1024 secs
  99% in 69.0942 secs

Details (average, fastest, slowest):
  DNS+dialup:   37.7823 secs, 0.4485 secs, 121.6341 secs
  DNS-lookup:   0.0000 secs, 0.0000 secs, 0.0000 secs
  req write:    0.0003 secs, 0.0000 secs, 0.0138 secs
  resp wait:    5.0817 secs, 0.1188 secs, 55.3515 secs
  resp read:    0.0039 secs, 0.0001 secs, 3.7078 secs

Status code distribution:
  [200] 9139 responses
  [503] 45 responses

Error distribution:
  [770] Get "https://www.electrobsd.org/": context deadline exceeded
  [46]  Get "https://www.electrobsd.org/": proxyconnect tcp: dial tcp 172.16.1.7:8118: connect: operation timed out

privoxy-experimental-ports-openssl-devel 3.0.33.20210317 with OpenSSL devel 3.0.0.a13

[fk@benchmark-vm ~]$ hey -x http://172.16.1.7:8118 -n 10000 -c 1000 -disable-keepalive -t 0 https://www.electrobsd.org/

Summary:
  Total:        646.4045 secs
  Slowest:      224.0558 secs
  Fastest:      0.1815 secs
  Average:      58.4754 secs
  Requests/sec: 15.4702

  Total data:   961599 bytes
  Size/request: 107 bytes

Response time histogram:
  0.181 [1]     |
  22.569 [524]  |____
  44.956 [1446] |__________
  67.344 [5768] |________________________________________
  89.731 [356]  |__
  112.119 [218] |__
  134.506 [166] |_
  156.893 [136] |_
  179.281 [111] |_
  201.668 [89]  |_
  224.056 [99]  |_


Latency distribution:
  10% in 39.5655 secs
  25% in 45.9014 secs
  50% in 52.6002 secs
  75% in 59.6084 secs
  90% in 81.8000 secs
  95% in 133.0850 secs
  99% in 207.5297 secs

Details (average, fastest, slowest):
  DNS+dialup:   46.6857 secs, 0.1815 secs, 224.0558 secs
  DNS-lookup:   0.0000 secs, 0.0000 secs, 0.0000 secs
  req write:    0.0004 secs, 0.0000 secs, 0.0401 secs
  resp wait:    11.7872 secs, 0.0891 secs, 178.0161 secs
  resp read:    0.0021 secs, 0.0001 secs, 0.5959 secs

Status code distribution:
  [200] 8780 responses
  [502] 33 responses
  [503] 101 responses

Error distribution:
  [1065]        Get "https://www.electrobsd.org/": context deadline exceeded
  [21]  Get "https://www.electrobsd.org/": proxyconnect tcp: dial tcp 172.16.1.7:8118: connect: operation timed out

privoxy-experimental-wolfssl 3.0.33.20210317_1 with wolfSSL 4.7.0_6

[fk@benchmark-vm ~]$ hey -x http://172.16.1.7:8118 -n 10000 -c 1000 -disable-keepalive -t 0 https://www.electrobsd.org/

Summary:
  Total:        341.1197 secs
  Slowest:      112.2543 secs
  Fastest:      0.1199 secs
  Average:      30.2646 secs
  Requests/sec: 29.3152

  Total data:   302505 bytes
  Size/request: 30 bytes

Response time histogram:
  0.120 [1]     |
  11.333 [231]  |_
  22.547 [762]  |____
  33.760 [7305] |________________________________________
  44.974 [657]  |____
  56.187 [598]  |___
  67.401 [229]  |_
  78.614 [155]  |_
  89.827 [4]    |
  101.041 [0]   |
  112.254 [9]   |


Latency distribution:
  10% in 22.5617 secs
  25% in 25.1543 secs
  50% in 27.8003 secs
  75% in 30.8988 secs
  90% in 45.0318 secs
  95% in 55.3919 secs
  99% in 68.8072 secs

Details (average, fastest, slowest):
  DNS+dialup:   24.5586 secs, 0.1199 secs, 112.2543 secs
  DNS-lookup:   0.0000 secs, 0.0000 secs, 0.0000 secs
  req write:    0.0010 secs, 0.0000 secs, 0.0764 secs
  resp wait:    5.6970 secs, 0.0946 secs, 46.1816 secs
  resp read:    0.0079 secs, 0.0001 secs, 0.9150 secs

Status code distribution:
  [200] 9908 responses
  [503] 43 responses

Error distribution:
  [12]  Get "https://www.electrobsd.org/": context deadline exceeded
  [37]  Get "https://www.electrobsd.org/": proxyconnect tcp: dial tcp 172.16.1.7:8118: connect: operation timed out

privoxy-experimental-mbedtls 3.0.33.20210317 with mbedtls 2.16.10

[fk@benchmark-vm ~]$ hey -x http://172.16.1.7:8118 -n 10000 -c 1000 -disable-keepalive -t 0 https://www.electrobsd.org/

Summary:
  Total:        843.2849 secs
  Slowest:      181.8306 secs
  Fastest:      1.4541 secs
  Average:      81.1763 secs
  Requests/sec: 11.8584

  Total data:   211050 bytes
  Size/request: 40 bytes

Response time histogram:
  1.454 [1]     |
  19.492 [97]   |___
  37.529 [162]  |____
  55.567 [541]  |_______________
  73.605 [1447] |________________________________________
  91.642 [1223] |__________________________________
  109.680 [831] |_______________________
  127.718 [512] |______________
  145.755 [265] |_______
  163.793 [75]  |__
  181.831 [6]   |


Latency distribution:
  10% in 47.1757 secs
  25% in 64.4024 secs
  50% in 77.7478 secs
  75% in 99.0196 secs
  90% in 116.5997 secs
  95% in 133.8875 secs
  99% in 149.5484 secs

Details (average, fastest, slowest):
  DNS+dialup:   67.7622 secs, 1.4541 secs, 181.8306 secs
  DNS-lookup:   0.0000 secs, 0.0000 secs, 0.0000 secs
  req write:    0.2751 secs, 0.0000 secs, 4.6106 secs
  resp wait:    12.9506 secs, 0.1315 secs, 78.1341 secs
  resp read:    0.1851 secs, 0.0001 secs, 12.6156 secs

Status code distribution:
  [200] 5130 responses
  [503] 30 responses

Error distribution:
  [4734]        Get "https://www.electrobsd.org/": context deadline exceeded
  [106] Get "https://www.electrobsd.org/": proxyconnect tcp: dial tcp 172.16.1.7:8118: connect: operation timed out

Privoxy CPU time and resident memory use according to top after the test

TLS libraryCPU timeMemory size
OpenSSL 1.0.2t6:43319M
LibreSSL 3.2.3_26:45347M
LibreSSL devel 3.3.110:56446M
OpenSSL 1.1.1j,16:56319M
OpenSSL devel 3.0.0.a139:15348M
wolfSSL 4.7.0_64:21271M
mbedtls 2.16.1011:21260M

2021-03-18: Privoxy 3.0.33 experimental 2021-03-17 dfedbde3b4 tested with siege

I decided to try siege which is written in C to see if it also shows skewed results like the benchmarking tools written in go.

While siege doesn't seem to offer histograms it provides a Longest transaction value and as it's rather high it looks the go runtime wasn't responsible for the effect after all.

siege requires a configuration file. I modified the default configuration to use Privoxy as proxy, disabled JSON mode and disabled verbose mode (which results in an output line for each request made).

Connection: close, concurrency level 1000

privoxy-experimental 3.0.33.20210317 with OpenSSL 1.0.2t-ElectroBSD from base

[fk@benchmark-vm ~]$ siege --concurrent=1000 --reps=30 --benchmark "https://www.electrobsd.org/"
** SIEGE 4.0.7
** Preparing 1000 concurrent users for battle.
The server is now under siege...siege aborted due to excessive socket failure; you
can change the failure threshold in $HOME/.siegerc

Transactions:                  28372 hits
Availability:                  96.23 %
Elapsed time:                 938.55 secs
Data transferred:              16.66 MB
Response time:                 30.21 secs
Transaction rate:              30.23 trans/sec
Throughput:                     0.02 MB/sec
Concurrency:                  913.16
Successful transactions:       28372
Failed transactions:            1112
Longest transaction:           98.49
Shortest transaction:           0.09

privoxy-experimental-libressl 3.0.33.20210317 with LibreSSL 3.2.3_2

[fk@benchmark-vm ~]$ siege --concurrent=1000 --reps=30 --benchmark "https://www.electrobsd.org/"
** SIEGE 4.0.7
** Preparing 1000 concurrent users for battle.
The server is now under siege...
Transactions:                  29635 hits
Availability:                  98.78 %
Elapsed time:                1314.94 secs
Data transferred:              12.06 MB
Response time:                 41.17 secs
Transaction rate:              22.54 trans/sec
Throughput:                     0.01 MB/sec
Concurrency:                  927.84
Successful transactions:       29635
Failed transactions:             365
Longest transaction:           88.52
Shortest transaction:           0.09

privoxy-experimental-libressl-devel 3.0.33.20210317 with LibreSSL devel 3.3.1

[fk@benchmark-vm ~]$ siege --concurrent=1000 --reps=30 --benchmark "https://www.electrobsd.org/"
** SIEGE 4.0.7
** Preparing 1000 concurrent users for battle.
The server is now under siege...
Transactions:                  29722 hits
Availability:                  99.07 %
Elapsed time:                1491.27 secs
Data transferred:              11.50 MB
Response time:                 46.84 secs
Transaction rate:              19.93 trans/sec
Throughput:                     0.01 MB/sec
Concurrency:                  933.50
Successful transactions:       29722
Failed transactions:             278
Longest transaction:          143.56
Shortest transaction:           0.09

privoxy-experimental-ports-openssl 3.0.33.20210317 with OpenSSL 1.1.1j,1

[fk@benchmark-vm ~]$ siege --concurrent=1000 --reps=30 --benchmark "https://www.electrobsd.org/"
** SIEGE 4.0.7
** Preparing 1000 concurrent users for battle.
The server is now under siege...
Transactions:                  29431 hits
Availability:                  98.10 %
Elapsed time:                1217.58 secs
Data transferred:              13.37 MB
Response time:                 36.14 secs
Transaction rate:              24.17 trans/sec
Throughput:                     0.01 MB/sec
Concurrency:                  873.47
Successful transactions:       29430
Failed transactions:             569
Longest transaction:          398.73
Shortest transaction:           0.07

privoxy-experimental-ports-openssl-devel 3.0.33.20210317 with OpenSSL devel 3.0.0.a13

[fk@benchmark-vm ~]$ siege --concurrent=1000 --reps=30 --benchmark "https://www.electrobsd.org/"
** SIEGE 4.0.7
** Preparing 1000 concurrent users for battle.
The server is now under siege...
Transactions:                  29696 hits
Availability:                  98.99 %
Elapsed time:                1410.63 secs
Data transferred:              11.67 MB
Response time:                 44.16 secs
Transaction rate:              21.05 trans/sec
Throughput:                     0.01 MB/sec
Concurrency:                  929.60
Successful transactions:       29696
Failed transactions:             304
Longest transaction:          100.67
Shortest transaction:           0.10

privoxy-experimental-wolfssl 3.0.33.20210317_1 with wolfSSL 4.7.0_6

[fk@benchmark-vm ~]$ siege --concurrent=1000 --reps=30 --benchmark "https://www.electrobsd.org/"
** SIEGE 4.0.7
** Preparing 1000 concurrent users for battle.
The server is now under siege...siege aborted due to excessive socket failure; you
can change the failure threshold in $HOME/.siegerc

Transactions:                  24798 hits
Availability:                  94.48 %
Elapsed time:                1108.23 secs
Data transferred:              17.74 MB
Response time:                 24.02 secs
Transaction rate:              22.38 trans/sec
Throughput:                     0.02 MB/sec
Concurrency:                  537.52
Successful transactions:       24233
Failed transactions:            1448
Longest transaction:          426.51
Shortest transaction:           0.18

privoxy-experimental-mbedtls 3.0.33.20210317 with mbedtls 2.16.10

[fk@benchmark-vm ~]$ siege --concurrent=1000 --reps=30 --benchmark "https://www.electrobsd.org/"
** SIEGE 4.0.7
** Preparing 1000 concurrent users for battle.
The server is now under siege...
Transactions:                  29957 hits
Availability:                  99.86 %
Elapsed time:                3575.75 secs
Data transferred:              10.00 MB
Response time:                115.71 secs
Transaction rate:               8.38 trans/sec
Throughput:                     0.00 MB/sec
Concurrency:                  969.42
Successful transactions:       29957
Failed transactions:              43
Longest transaction:          165.54
Shortest transaction:           0.20

Privoxy CPU time and resident memory use according to top after the test

TLS libraryCPU timeMemory size
OpenSSL 1.0.2t13:50501M
LibreSSL 3.2.3_219:35478M
LibreSSL devel 3.3.122:24527M
OpenSSL 1.1.1j,117:11554M
OpenSSL devel 3.0.0.a1320:50372M
wolfSSL 4.7.0_610:11515M
mbedtls 2.16.1056:1996720K

2021-03-18: Privoxy 3.0.33 experimental 2021-03-18 f41ee87 tested with siege

All Privoxy ports have been updated, an experimental patch has been added that allows to change the scheduling policy using sched_setscheduler.

Connection: close, concurrency level 1000, various scheduling policies

privoxy-experimental 3.0.33.20210318 with OpenSSL 1.0.2t-ElectroBSD from base and scheduling policy 1 (SCHED_FIFO)

[fk@benchmark-vm ~]$ siege --concurrent=1000 --reps=10 --benchmark "https://www.electrobsd.org/"
** SIEGE 4.0.7
** Preparing 1000 concurrent users for battle.
The server is now under siege...
Transactions:                   9960 hits
Availability:                  99.60 %
Elapsed time:                 416.73 secs
Data transferred:               3.50 MB
Response time:                 29.56 secs
Transaction rate:              23.90 trans/sec
Throughput:                     0.01 MB/sec
Concurrency:                  706.39
Successful transactions:        9960
Failed transactions:              40
Longest transaction:          311.88
Shortest transaction:           0.09

While the test was running the ssh session to the Privoxy VM became unresponsive for several minutes, top showed a nice value of r31F most of the time. Given that and the 311.88 seconds it supposedly took to handle the longest transaction, SCHED_FIFO doesn't appear to be useful for Privoxy.

privoxy-experimental 3.0.33.20210318 with OpenSSL 1.0.2t-ElectroBSD from base and scheduling policy 2 (SCHED_OTHER)

[fk@benchmark-vm ~]$ siege --concurrent=1000 --reps=10 --benchmark "https://www.electrobsd.org/"
** SIEGE 4.0.7
** Preparing 1000 concurrent users for battle.
The server is now under siege...[alert] socket: select and discovered it's not ready sock.c:384: Operation timed out
[alert] socket: read check timed out(30) sock.c:273: Operation timed out
[alert] socket: select and discovered it's not ready sock.c:384: Operation timed out
[alert] socket: read check timed out(30) sock.c:273: Operation timed out
[alert] socket: select and discovered it's not ready sock.c:384: Operation timed out
[alert] socket: read check timed out(30) sock.c:273: Operation timed out
[alert] socket: select and discovered it's not ready sock.c:384: Operation timed out
[alert] socket: read check timed out(30) sock.c:273: Operation timed out
[alert] socket: select and discovered it's not ready sock.c:384: Operation timed out
[alert] socket: read check timed out(30) sock.c:273: Operation timed out
[alert] socket: select and discovered it's not ready sock.c:384: Operation timed out
[alert] socket: read check timed out(30) sock.c:273: Operation timed out
[alert] socket: select and discovered it's not ready sock.c:384: Operation timed out
[alert] socket: read check timed out(30) sock.c:273: Operation timed out
[alert] socket: select and discovered it's not ready sock.c:384: Operation timed out
[alert] socket: read check timed out(30) sock.c:273: Operation timed out
[alert] socket: select and discovered it's not ready sock.c:384: Operation timed out
[alert] socket: read check timed out(30) sock.c:273: Operation timed out
[alert] socket: select and discovered it's not ready sock.c:384: Operation timed out
[alert] socket: read check timed out(30) sock.c:273: Operation timed out
[alert] socket: select and discovered it's not ready sock.c:384: Operation timed out
[alert] socket: read check timed out(30) sock.c:273: Operation timed out
[alert] socket: select and discovered it's not ready sock.c:384: Operation timed out
[alert] socket: read check timed out(30) sock.c:273: Operation timed out
[alert] socket: select and discovered it's not ready sock.c:384: Operation timed out
[alert] socket: read check timed out(30) sock.c:273: Operation timed out
[alert] socket: select and discovered it's not ready sock.c:384: Operation timed out
[alert] socket: read check timed out(30) sock.c:273: Operation timed out
[alert] socket: select and discovered it's not ready sock.c:384: Operation timed out
[alert] socket: read check timed out(30) sock.c:273: Operation timed out
[alert] socket: select and discovered it's not ready sock.c:384: Operation timed out
[alert] socket: read check timed out(30) sock.c:273: Operation timed out
[alert] socket: select and discovered it's not ready sock.c:384: Operation timed out
[alert] socket: read check timed out(30) sock.c:273: Operation timed out

Transactions:                   9880 hits
Availability:                  98.80 %
Elapsed time:                 377.66 secs
Data transferred:               3.89 MB
Response time:                 29.81 secs
Transaction rate:              26.16 trans/sec
Throughput:                     0.01 MB/sec
Concurrency:                  779.75
Successful transactions:        9880
Failed transactions:             120
Longest transaction:           89.09
Shortest transaction:           0.09

privoxy-experimental 3.0.33.20210318 with OpenSSL 1.0.2t-ElectroBSD from base and scheduling policy 3 (SCHED_RR)

[fk@benchmark-vm ~]$ siege --concurrent=1000 --reps=10 --benchmark "https://www.electrobsd.org/"
** SIEGE 4.0.7
** Preparing 1000 concurrent users for battle.
The server is now under siege...
Transactions:                   9603 hits
Availability:                  96.03 %
Elapsed time:                 601.98 secs
Data transferred:               5.89 MB
Response time:                 29.62 secs
Transaction rate:              15.95 trans/sec
Throughput:                     0.01 MB/sec
Concurrency:                  472.53
Successful transactions:        9603
Failed transactions:             397
Longest transaction:          388.54
Shortest transaction:           0.08

Enabling SCHED_RR resulted in a temporary unresponsive ssh session as well. For the last minutes of the test top showed that Privoxy was using three threads, a nice value of r31 and was idle.

The Privoxy log confirmed that Privoxy didn't do any work for about four minutes near the end of the test:

15:15:52.791 818984200 Crunch: Connection failure: https://www.electrobsd.org/
172.16.1.6 - - [18/Mar/2021:15:15:52 +0100] "GET https://www.electrobsd.org/ HTTP/1.1" 503 7035
15:19:37.838 81787f300 Error: The TLS/SSL handshake with the server failed: no TLS/SSL errors detected

privoxy-experimental 3.0.33.20210318 with OpenSSL 1.0.2t-ElectroBSD from base and no scheduling policy adjustments

[fk@benchmark-vm ~]$ siege --concurrent=1000 --reps=10 --benchmark "https://www.electrobsd.org/"
** SIEGE 4.0.7
** Preparing 1000 concurrent users for battle.
The server is now under siege...
Transactions:                   9965 hits
Availability:                  99.65 %
Elapsed time:                 369.88 secs
Data transferred:               3.47 MB
Response time:                 28.65 secs
Transaction rate:              26.94 trans/sec
Throughput:                     0.01 MB/sec
Concurrency:                  771.94
Successful transactions:        9965
Failed transactions:              35
Longest transaction:           71.43
Shortest transaction:           0.08

This test was expected to get similar results as the one with scheduling policy 2 and it did.

Conclusion

As both non-default scheduling policies seem to be worse than the default, polishing the scheduling policy patch probably isn't worth it.

2021-03-20: Privoxy 3.0.33 experimental 2021-03-17 dfedbde3b4 tested with siege 4.0.7_3

I've patched siege to allow https requests over a proxy using keep-alive and enabled keep-alive in the siege configuration file, ignoring the TRIPLE CAUTION: We don't recommend you set this to keep-alive comment.

One obvious issue is that the concurrency seems to vary significantly. Something to investigate later on.

privoxy-experimental 3.0.33.20210317 with OpenSSL 1.0.2t-ElectroBSD from base

[fk@benchmark-vm ~]$ siege --concurrent=1000 --reps=10 --benchmark "https://www.electrobsd.org/"
** SIEGE 4.0.7
** Preparing 1000 concurrent users for battle.
The server is now under siege...
Transactions:                  10000 hits
Availability:                 100.00 %
Elapsed time:                  97.60 secs
Data transferred:               3.24 MB
Response time:                  0.12 secs
Transaction rate:             102.46 trans/sec
Throughput:                     0.03 MB/sec
Concurrency:                   12.32
Successful transactions:       10000
Failed transactions:               0
Longest transaction:            5.29
Shortest transaction:           0.01

The Privoxy log confirms that 1000 connections where opened to send 10000 requests but as mentioned before the concurrency is quite a bit lower than requested.

[fk@privoxy-vm ~]$ privoxy-log-parser.pl --statistics /var/log/privoxy/privoxy.log
Client requests total: 10000
No crunches detected. Is 'debug 1024' enabled?
Server keep-alive offers: 10000 (100.00%)
New outgoing connections: 1000 (10.00%)
Reused server connections: 9000 (90.00%; server offers accepted: 90.00%)
Empty responses: 0 (0.00%)
Empty responses on new connections: 0 (0.00%)
Empty responses on reused connections: 0 (0.00%)
Client connections: 1000
Bytes of content transfered to the client: 3520000
Client requests per connection distribution:
    1000: 10
       0: 1
Enable --show-complete-request-distribution to get less common numbers as well.
Improperly accounted requests: ~0
Method distribution:
   10000 : GET
Client HTTP versions:
   10000 : HTTP/1.1
HTTP status codes:
   10000 : 200
URL statistics are disabled. Increase --url-statistics-threshold to enable them.
Passed request statistics are disabled. Increase --passed-request-statistics-threshold to enable them.
Host statistics are disabled. Increase --host-statistics-threshold to enable them.

privoxy-experimental-libressl 3.0.33.20210317 with LibreSSL 3.2.3_2

[fk@benchmark-vm ~]$ siege --concurrent=1000 --reps=10 --benchmark "https://www.electrobsd.org/"
** SIEGE 4.0.7
** Preparing 1000 concurrent users for battle.
The server is now under siege...
Transactions:                  10000 hits
Availability:                 100.00 %
Elapsed time:                  99.43 secs
Data transferred:               3.24 MB
Response time:                  0.16 secs
Transaction rate:             100.57 trans/sec
Throughput:                     0.03 MB/sec
Concurrency:                   16.35
Successful transactions:       10000
Failed transactions:               0
Longest transaction:            6.20
Shortest transaction:           0.01

privoxy-experimental-libressl-devel 3.0.33.20210317 with LibreSSL devel 3.3.1

[fk@benchmark-vm ~]$ siege --concurrent=1000 --reps=10 --benchmark "https://www.electrobsd.org/"
** SIEGE 4.0.7
** Preparing 1000 concurrent users for battle.
The server is now under siege...
Transactions:                  10000 hits
Availability:                 100.00 %
Elapsed time:                  90.48 secs
Data transferred:               3.24 MB
Response time:                  0.11 secs
Transaction rate:             110.53 trans/sec
Throughput:                     0.04 MB/sec
Concurrency:                   11.70
Successful transactions:       10000
Failed transactions:               0
Longest transaction:            5.48
Shortest transaction:           0.01

privoxy-experimental-ports-openssl 3.0.33.20210317 with OpenSSL 1.1.1j,1

[fk@benchmark-vm ~]$ siege --concurrent=1000 --reps=10 --benchmark "https://www.electrobsd.org/"
** SIEGE 4.0.7
** Preparing 1000 concurrent users for battle.
The server is now under siege...
Transactions:                   9999 hits
Availability:                  99.99 %
Elapsed time:                 105.88 secs
Data transferred:               3.24 MB
Response time:                  0.87 secs
Transaction rate:              94.44 trans/sec
Throughput:                     0.03 MB/sec
Concurrency:                   82.33
Successful transactions:        9999
Failed transactions:               1
Longest transaction:           19.94
Shortest transaction:           0.01

It's interesting tha the Concurrency went up compared to the previous tests.

privoxy-experimental-ports-openssl-devel 3.0.33.20210317 with OpenSSL devel 3.0.0.a13

[fk@benchmark-vm ~]$ siege --concurrent=1000 --reps=10 --benchmark "https://www.electrobsd.org/"
** SIEGE 4.0.7
** Preparing 1000 concurrent users for battle.
The server is now under siege...
Transactions:                  10000 hits
Availability:                 100.00 %
Elapsed time:                  92.08 secs
Data transferred:               3.24 MB
Response time:                  0.16 secs
Transaction rate:             108.60 trans/sec
Throughput:                     0.04 MB/sec
Concurrency:                   17.70
Successful transactions:       10000
Failed transactions:               0
Longest transaction:            5.52
Shortest transaction:           0.01

privoxy-experimental-wolfssl 3.0.33.20210317_1 with wolfSSL 4.7.0_6

[fk@benchmark-vm ~]$ siege --concurrent=1000 --reps=10 --benchmark "https://www.electrobsd.org/"
** SIEGE 4.0.7
** Preparing 1000 concurrent users for battle.
The server is now under siege...
Transactions:                  10000 hits
Availability:                 100.00 %
Elapsed time:                  90.33 secs
Data transferred:               3.24 MB
Response time:                  0.36 secs
Transaction rate:             110.71 trans/sec
Throughput:                     0.04 MB/sec
Concurrency:                   39.46
Successful transactions:       10000
Failed transactions:               0
Longest transaction:           11.46
Shortest transaction:           0.01

privoxy-experimental-mbedtls 3.0.33.20210317 with mbedtls 2.16.10

[fk@benchmark-vm ~]$ siege --concurrent=1000 --reps=10 --benchmark "https://www.electrobsd.org/"
** SIEGE 4.0.7
** Preparing 1000 concurrent users for battle.
The server is now under siege...
Transactions:                   9956 hits
Availability:                  99.56 %
Elapsed time:                 158.23 secs
Data transferred:               3.23 MB
Response time:                  6.84 secs
Transaction rate:              62.92 trans/sec
Throughput:                     0.02 MB/sec
Concurrency:                  430.35
Successful transactions:        9956
Failed transactions:              44
Longest transaction:          118.43
Shortest transaction:           0.01

The concurrency went up to 430!

TLS libraryCPU timeMemory size
OpenSSL 1.0.2t0:5999M
LibreSSL 3.2.3_21:1566284K
LibreSSL devel 3.3.11:1244536K
OpenSSL 1.1.1j,11:14209M
OpenSSL devel 3.0.0.a131:1491876K
wolfSSL 4.7.0_60:54108M
mbedtls 2.16.102:24181M

2021-03-21: Privoxy 3.0.33 experimental 20210321 49e1e8ff2aa tested with siege 4.0.7_3

The LibreSSL port has been updated to 3.2.5.

The Privoxy ports haven been updated as well. The Privoxy's wolfSSL-specific code now contains an experimental patch for testing purposes that lets Privoxy use a single SSL context for the server connections instead of setting up a unique context for each connection.

Connection: close, concurrency level 1000

For this test siege is configured with "connection = close" again.

privoxy-experimental 3.0.33.20210321 with OpenSSL 1.0.2t-ElectroBSD from base

[fk@benchmark-vm ~]$ siege --concurrent=1000 --reps=10 --benchmark "https://www.electrobsd.org/"
** SIEGE 4.0.7
** Preparing 1000 concurrent users for battle.
The server is now under siege...
Transactions:                   9949 hits
Availability:                  99.49 %
Elapsed time:                 366.41 secs
Data transferred:               3.57 MB
Response time:                 28.11 secs
Transaction rate:              27.15 trans/sec
Throughput:                     0.01 MB/sec
Concurrency:                  763.33
Successful transactions:        9949
Failed transactions:              51
Longest transaction:           66.06
Shortest transaction:           0.09

privoxy-experimental-libressl 3.0.33.20210321 with LibreSSL 3.2.5

[fk@benchmark-vm ~]$ siege --concurrent=1000 --reps=10 --benchmark "https://www.electrobsd.org/"
** SIEGE 4.0.7
** Preparing 1000 concurrent users for battle.
The server is now under siege...
Transactions:                   9902 hits
Availability:                  99.02 %
Elapsed time:                 803.27 secs
Data transferred:               3.87 MB
Response time:                 40.86 secs
Transaction rate:              12.33 trans/sec
Throughput:                     0.00 MB/sec
Concurrency:                  503.71
Successful transactions:        9902
Failed transactions:              98
Longest transaction:          405.18
Shortest transaction:           0.14

privoxy-experimental-libressl-devel 3.0.33.20210321 with LibreSSL devel 3.3.1

[fk@benchmark-vm ~]$ siege --concurrent=1000 --reps=10 --benchmark "https://www.electrobsd.org/"
** SIEGE 4.0.7
** Preparing 1000 concurrent users for battle.
The server is now under siege...
Transactions:                   9936 hits
Availability:                  99.36 %
Elapsed time:                 496.41 secs
Data transferred:               3.65 MB
Response time:                 40.42 secs
Transaction rate:              20.02 trans/sec
Throughput:                     0.01 MB/sec
Concurrency:                  808.93
Successful transactions:        9936
Failed transactions:              64
Longest transaction:          103.87
Shortest transaction:           0.09

privoxy-experimental-ports-openssl 3.0.33.20210321 with OpenSSL 1.1.1j,1

[fk@benchmark-vm ~]$ siege --concurrent=1000 --reps=10 --benchmark "https://www.electrobsd.org/"
** SIEGE 4.0.7
** Preparing 1000 concurrent users for battle.
The server is now under siege...
Transactions:                   9825 hits
Availability:                  98.25 %
Elapsed time:                 414.31 secs
Data transferred:               4.36 MB
Response time:                 33.77 secs
Transaction rate:              23.71 trans/sec
Throughput:                     0.01 MB/sec
Concurrency:                  800.78
Successful transactions:        9825
Failed transactions:             175
Longest transaction:           86.27
Shortest transaction:           0.09

privoxy-experimental-ports-openssl-devel 3.0.33.20210321 with OpenSSL devel 3.0.0.a13

[fk@benchmark-vm ~]$ siege --concurrent=1000 --reps=10 --benchmark "https://www.electrobsd.org/"
** SIEGE 4.0.7
** Preparing 1000 concurrent users for battle.
The server is now under siege...[alert] socket: select and discovered it's not ready sock.c:384: Operation timed out
[alert] socket: read check timed out(30) sock.c:273: Operation timed out

Transactions:                   9918 hits
Availability:                  99.18 %
Elapsed time:                 556.85 secs
Data transferred:               3.76 MB
Response time:                 47.52 secs
Transaction rate:              17.81 trans/sec
Throughput:                     0.01 MB/sec
Concurrency:                  846.32
Successful transactions:        9918
Failed transactions:              82
Longest transaction:          101.36
Shortest transaction:           0.28

privoxy-experimental-wolfssl 3.0.33.20210321 with wolfSSL 4.7.0_6

[fk@benchmark-vm ~]$ siege --concurrent=1000 --reps=10 --benchmark "https://www.electrobsd.org/"
** SIEGE 4.0.7
** Preparing 1000 concurrent users for battle.
The server is now under siege...
Transactions:                   9992 hits
Availability:                  99.92 %
Elapsed time:                 873.68 secs
Data transferred:               3.28 MB
Response time:                 29.50 secs
Transaction rate:              11.44 trans/sec
Throughput:                     0.00 MB/sec
Concurrency:                  337.39
Successful transactions:        9583
Failed transactions:               8
Longest transaction:          414.19
Shortest transaction:           0.06

It looks like my experimental wolfSSL-specific patch introduces a problem. Both siege and Privoxy where idle for longer amounts of time while siege was running.

Apparently Privoxy did not do log-worthy things for nearly five minutes:

2021-03-21 11:12:46.383 8118ab900 Connect: Closing client socket 5. Keep-alive: 0. Socket alive: 1. Data available: 0. Configuration file change detected: 0. Requests received: 1.
2021-03-21 11:12:46.383 8118ab900 Connect: Failed to shutdown client connection on socket 5. Attempts so far: 1, ret: 2
2021-03-21 11:17:38.971 822340500 Connect: Failed to shutdown server connection on socket 70. Attempts so far: 2, ret: 2
2021-03-21 11:17:38.972 822340500 Connect: Not shutting down server connection on socket 70. The socket is no longer alive.
2021-03-21 11:17:38.972 822340500 Connect: Closing client socket 69. Keep-alive: 0. Socket alive: 1. Data available: 1. Configuration file change detected: 0. Requests received: 1.

Update: The problem was reproducible:

[fk@benchmark-vm ~]$ siege --concurrent=1000 --reps=10 --benchmark "https://www.electrobsd.org/"
** SIEGE 4.0.7
** Preparing 1000 concurrent users for battle.
The server is now under siege...
Transactions:                   9973 hits
Availability:                  99.73 %
Elapsed time:                 891.48 secs
Data transferred:               3.41 MB
Response time:                 22.36 secs
Transaction rate:              11.19 trans/sec
Throughput:                     0.00 MB/sec
Concurrency:                  250.14
Successful transactions:        9669
Failed transactions:              27
Longest transaction:          368.88
Shortest transaction:           0.06

Again it looks like Privoxy spent about five minutes trying to shut down a connection:

2021-03-21 16:23:08.203 810bfaf00 Connect: Not shutting down server connection on socket 638. The socket is no longer alive.
2021-03-21 16:23:08.204 810bfaf00 Connect: Closing client socket 57. Keep-alive: 0. Socket alive: 1. Data available: 1. Configuration file change detected: 0. Requests received: 1.
2021-03-21 16:23:08.204 810bfaf00 Connect: Failed to shutdown client connection on socket 57. Attempts so far: 1, ret: 2
2021-03-21 16:27:24.871 803a69e00 Connect: Failed to shutdown server connection on socket 1780. Attempts so far: 2, ret: 2
2021-03-21 16:27:24.872 803a69e00 Connect: Not shutting down server connection on socket 1780. The socket is no longer alive.
2021-03-21 16:27:24.872 803a69e00 Connect: Closing client socket 628. Keep-alive: 0. Socket alive: 1. Data available: 1. Configuration file change detected: 0. Requests received: 1.
2021-03-21 16:27:24.872 803a69e00 Connect: Failed to shutdown client connection on socket 628. Attempts so far: 1, ret: 2

privoxy-experimental-mbedtls 3.0.33.20210321 with mbedtls 2.16.10

[fk@benchmark-vm ~]$ siege --concurrent=1000 --reps=10 --benchmark "https://www.electrobsd.org/"
** SIEGE 4.0.7
** Preparing 1000 concurrent users for battle.
The server is now under siege...
Transactions:                   9989 hits
Availability:                  99.89 %
Elapsed time:                1244.04 secs
Data transferred:               3.31 MB
Response time:                113.31 secs
Transaction rate:               8.03 trans/sec
Throughput:                     0.00 MB/sec
Concurrency:                  909.86
Successful transactions:        9989
Failed transactions:              11
Longest transaction:          168.69
Shortest transaction:           1.75

Privoxy CPU time and resident memory use according to top after the test

TLS libraryCPU timeMemory size
OpenSSL 1.0.2t4:56400M
LibreSSL 3.2.57:08443M
LibreSSL devel 3.3.17:03335M
OpenSSL 1.1.1j,15:42345M
OpenSSL devel 3.0.0.a137:55324M
wolfSSL 4.7.0_6 test 13:26541M
wolfSSL 4.7.0_6 test 23:31535M
mbedtls 2.16.1018:53131M

Connection: close, concurrency level 10

The siege configuration remains the same but the requested concurrency is reduced to 10 while the reps are set to 1000 to get a similar amount of requests as before.

privoxy-experimental 3.0.33.20210321 with OpenSSL 1.0.2t-ElectroBSD from base

[fk@benchmark-vm ~]$ siege --concurrent=10 --reps=1000 --benchmark "https://www.electrobsd.org/"
** SIEGE 4.0.7
** Preparing 10 concurrent users for battle.
The server is now under siege...
Transactions:                   9956 hits
Availability:                  99.56 %
Elapsed time:                 580.95 secs
Data transferred:               3.52 MB
Response time:                  0.46 secs
Transaction rate:              17.14 trans/sec
Throughput:                     0.01 MB/sec
Concurrency:                    7.93
Successful transactions:        9956
Failed transactions:              44
Longest transaction:           30.17
Shortest transaction:           0.07

privoxy-experimental-libressl 3.0.33.20210321 with LibreSSL 3.2.5

[fk@benchmark-vm ~]$ siege --concurrent=10 --reps=1000 --benchmark "https://www.electrobsd.org/"
** SIEGE 4.0.7
** Preparing 10 concurrent users for battle.
The server is now under siege...[alert] socket: select and discovered it's not ready sock.c:384: Operation timed out
[alert] socket: read check timed out(30) sock.c:273: Operation timed out

Transactions:                   9958 hits
Availability:                  99.58 %
Elapsed time:                 514.22 secs
Data transferred:               3.50 MB
Response time:                  0.42 secs
Transaction rate:              19.37 trans/sec
Throughput:                     0.01 MB/sec
Concurrency:                    8.07
Successful transactions:        9958
Failed transactions:              42
Longest transaction:           30.30
Shortest transaction:           0.07

privoxy-experimental-libressl-devel 3.0.33.20210321 with LibreSSL devel 3.3.1

[fk@benchmark-vm ~]$ siege --concurrent=10 --reps=1000 --benchmark "https://www.electrobsd.org/"
** SIEGE 4.0.7
** Preparing 10 concurrent users for battle.
The server is now under siege...
Transactions:                   9953 hits
Availability:                  99.53 %
Elapsed time:                 562.77 secs
Data transferred:               3.54 MB
Response time:                  0.51 secs
Transaction rate:              17.69 trans/sec
Throughput:                     0.01 MB/sec
Concurrency:                    9.04
Successful transactions:        9953
Failed transactions:              47
Longest transaction:           31.95
Shortest transaction:           0.08

privoxy-experimental-ports-openssl 3.0.33.20210321 with OpenSSL 1.1.1j,1

[fk@benchmark-vm ~]$ siege --concurrent=10 --reps=1000 --benchmark "https://www.electrobsd.org/"
** SIEGE 4.0.7
** Preparing 10 concurrent users for battle.
The server is now under siege...
Transactions:                   9964 hits
Availability:                  99.64 %
Elapsed time:                 518.83 secs
Data transferred:               3.47 MB
Response time:                  0.39 secs
Transaction rate:              19.20 trans/sec
Throughput:                     0.01 MB/sec
Concurrency:                    7.43
Successful transactions:        9964
Failed transactions:              36
Longest transaction:           30.54
Shortest transaction:           0.07

privoxy-experimental-ports-openssl-devel 3.0.33.20210321 with OpenSSL devel 3.0.0.a13

[fk@benchmark-vm ~]$ siege --concurrent=10 --reps=1000 --benchmark "https://www.electrobsd.org/"
** SIEGE 4.0.7
** Preparing 10 concurrent users for battle.
The server is now under siege...[alert] socket: select and discovered it's not ready sock.c:384: Operation timed out
[alert] socket: read check timed out(30) sock.c:273: Operation timed out

Transactions:                   9991 hits
Availability:                  99.91 %
Elapsed time:                 463.96 secs
Data transferred:               3.29 MB
Response time:                  0.40 secs
Transaction rate:              21.53 trans/sec
Throughput:                     0.01 MB/sec
Concurrency:                    8.61
Successful transactions:        9991
Failed transactions:               9
Longest transaction:           30.19
Shortest transaction:           0.08

privoxy-experimental-wolfssl 3.0.33.20210321 with wolfSSL 4.7.0_6

[fk@benchmark-vm ~]$ siege --concurrent=10 --reps=1000 --benchmark "https://www.electrobsd.org/"
** SIEGE 4.0.7
** Preparing 10 concurrent users for battle.
The server is now under siege...
Transactions:                   9973 hits
Availability:                  99.73 %
Elapsed time:                 416.36 secs
Data transferred:               3.41 MB
Response time:                  0.32 secs
Transaction rate:              23.95 trans/sec
Throughput:                     0.01 MB/sec
Concurrency:                    7.58
Successful transactions:        9973
Failed transactions:              27
Longest transaction:           30.10
Shortest transaction:           0.05

privoxy-experimental-mbedtls 3.0.33.20210321 with mbedtls 2.16.10

[fk@benchmark-vm ~]$ siege --concurrent=10 --reps=1000 --benchmark "https://www.electrobsd.org/"
** SIEGE 4.0.7
** Preparing 10 concurrent users for battle.
The server is now under siege...
Transactions:                   9999 hits
Availability:                  99.99 %
Elapsed time:                1154.95 secs
Data transferred:               3.25 MB
Response time:                  1.14 secs
Transaction rate:               8.66 trans/sec
Throughput:                     0.00 MB/sec
Concurrency:                    9.87
Successful transactions:        9999
Failed transactions:               1
Longest transaction:           30.54
Shortest transaction:           0.18

Privoxy CPU time and resident memory use according to top after the test

TLS libraryCPU timeMemory size
OpenSSL 1.0.2t4:2371944K
LibreSSL 3.2.55:2268748K
LibreSSL devel 3.3.15:5562216K
OpenSSL 1.1.1j,14:5285780K
OpenSSL devel 3.0.0.a136:0870112K
wolfSSL 4.7.0_61:5721224K
mbedtls 2.16.1018:0615432K

The significantly reduced CPU time when using wolfSSL indicates that the wolfSSL-specific patch to share the SSL context may not be entirely useless.

2021-03-21: Privoxy 3.0.33 experimental 20210321_1 7c2c69fc43 tested with siege 4.0.7_3

I've updated the Privoxy ports to include a wolfSSL-specific patch that sets MAX_SHUTDOWN_ATTEMPTS to 1. The effect is that wolfSSL_shutdown() is not called more than once, even if it returns WOLFSSL_SHUTDOWN_NOT_DONE in which case one is supposed to call wolfSSL_shutdown again to complete according to wolfssl/ssl.h.

Connection: close, concurrency level 1000

privoxy-experimental-wolfssl 3.0.33.20210321_1 with wolfSSL 4.7.0_6

[fk@benchmark-vm ~]$ siege --concurrent=1000 --reps=10 --benchmark "https://www.electrobsd.org/"
** SIEGE 4.0.7
** Preparing 1000 concurrent users for battle.
The server is now under siege...
Transactions:                   9964 hits
Availability:                  99.64 %
Elapsed time:                 532.43 secs
Data transferred:               3.46 MB
Response time:                 28.14 secs
Transaction rate:              18.71 trans/sec
Throughput:                     0.01 MB/sec
Concurrency:                  526.54
Successful transactions:        9493
Failed transactions:              36
Longest transaction:          352.60
Shortest transaction:           0.06

Not retrying shutdowns doesn't seem to work around the issue:

2021-03-21 17:35:47.531 80abc7500 Connect: Could not connect to [www.electrobsd.org]:443: Operation timed out.
2021-03-21 17:35:47.532 80abc7500 Crunch: Connection failure: https://www.electrobsd.org/
172.16.1.6 - - [21/Mar/2021:17:35:47 +0100] "GET https://www.electrobsd.org/ HTTP/1.1" 503 7035
2021-03-21 17:35:47.533 80abc7500 Connect: Failed to shutdown client connection on socket 59. Attempts so far: 1, ret: 2
2021-03-21 17:35:47.533 80abc7500 Error: Failed to shutdown client connection on socket 59 after 1 attempts. ret: 2, error: 0, unknown error number
2021-03-21 17:38:09.441 812906b00 Error: X509 certificate verification for www.electrobsd.org failed with error -308: error state on socket
2021-03-21 17:38:09.442 812906b00 Crunch: Certificate error: error state on socket: https://www.electrobsd.org/

Privoxy CPU time and resident memory use according to top after the test

TLS libraryCPU timeMemory size
wolfSSL 4.7.0_63:05581M

2021-03-22: Privoxy 3.0.33 experimental 20210322 7e09d037f6 tested with siege 4.0.7_3

The Privoxy ports have been updated. They now contain a Privoxy-Log-Parser patch to detect periods of inactivity.

Connection: close, concurrency level 1000

privoxy-experimental-wolfssl 3.0.33.20210322 with wolfSSL 4.7.0_6

[fk@benchmark-vm ~]$ siege --concurrent=1000 --reps=10 --benchmark "https://www.electrobsd.org/"
** SIEGE 4.0.7
** Preparing 1000 concurrent users for battle.
The server is now under siege...
Transactions:                   9963 hits
Availability:                  99.63 %
Elapsed time:                 672.10 secs
Data transferred:               3.47 MB
Response time:                 28.66 secs
Transaction rate:              14.82 trans/sec
Throughput:                     0.01 MB/sec
Concurrency:                  424.91
Successful transactions:        9719
Failed transactions:              37
Longest transaction:          332.53
Shortest transaction:           0.06

Privoxy-Log-Parser detects inactivity:

[fk@privoxy-vm ~]$ sort /var/log/privoxy/privoxy-3.0.33.20210322-wolfssl-c1000-cc.log | privoxy-log-parser.pl --detect-inactivity --inactivity-threshold 30000 | grep -B 3 -A 2 '^Detected inactivity'
2021-03-22 07:12:20.184 80d962c00 Crunch: Connection failure: https://www.electrobsd.org/
2021-03-22 07:12:20.185 80d962c00 Connect: Failed to shutdown client connection on socket 6. Attempts so far: 1, ret: 2
2021-03-22 07:12:20.185 80d962c00 Error: Failed to shutdown client connection on socket 6 after 1 attempts. ret: 2, error: 0, unknown error number
Detected inactivity: 96305 msecs
2021-03-22 07:13:56.490 80fdc9600 Error: X509 certificate verification for www.electrobsd.org failed with error -308: error state on socket
2021-03-22 07:13:56.491 80fdc9600 Connect: Closing client socket 906. Keep-alive: 0. Socket alive: 1. Data available: 0. Configuration file change detected: 0. Requests received: 1.

privoxy-experimental-wolfssl 3.0.33.20210322 with wolfSSL 4.7.0_6

Privoxy CPU time and resident memory use according to top after the test

TLS libraryCPU timeMemory size
wolfSSL 4.7.0_64:59511M

Privoxy 3.0.33 experimental 20210322_1 1ae5bdd72e tested with siege 4.0.7_3

The Privoxy ports have been updated to remove the wolfSSL-specific ctx-reuse hack which requires additional work before it's ready.

Connection: close, concurrency level 1000

privoxy-experimental 3.0.33.20210321_1 with OpenSSL 1.0.2t-ElectroBSD from base

[fk@benchmark-vm ~]$ siege --concurrent=1000 --reps=10 --benchmark "https://www.electrobsd.org/"
** SIEGE 4.0.7
** Preparing 1000 concurrent users for battle.
The server is now under siege...
Transactions:                   9964 hits
Availability:                  99.64 %
Elapsed time:                 384.18 secs
Data transferred:               3.47 MB
Response time:                 29.86 secs
Transaction rate:              25.94 trans/sec
Throughput:                     0.01 MB/sec
Concurrency:                  774.41
Successful transactions:        9964
Failed transactions:              36
Longest transaction:           60.51
Shortest transaction:           0.09
[fk@privoxy-vm ~]$ privoxy-log-parser.pl --detect-inactivity --inactivity-threshold 30000 /var/log/privoxy/privoxy.log | grep -B 3 -A 2 '^Detected inactivity'

privoxy-experimental-libressl 3.0.33.20210321_1 with LibreSSL 3.2.5

[fk@benchmark-vm ~]$ siege --concurrent=1000 --reps=10 --benchmark "https://www.electrobsd.org/"
** SIEGE 4.0.7
** Preparing 1000 concurrent users for battle.
The server is now under siege...
Transactions:                   9935 hits
Availability:                  99.35 %
Elapsed time:                 464.12 secs
Data transferred:               3.66 MB
Response time:                 37.46 secs
Transaction rate:              21.41 trans/sec
Throughput:                     0.01 MB/sec
Concurrency:                  801.86
Successful transactions:        9935
Failed transactions:              65
Longest transaction:           87.09
Shortest transaction:           0.33
[fk@privoxy-vm ~]$ privoxy-log-parser.pl --detect-inactivity --inactivity-threshold 30000 /var/log/privoxy/privoxy.log | grep -B 3 -A 2 '^Detected inactivity'

privoxy-experimental-libressl-devel 3.0.33.20210321_1 with LibreSSL devel 3.3.1

[fk@benchmark-vm ~]$ siege --concurrent=1000 --reps=10 --benchmark "https://www.electrobsd.org/"
** SIEGE 4.0.7
** Preparing 1000 concurrent users for battle.
The server is now under siege...
Transactions:                   9954 hits
Availability:                  99.54 %
Elapsed time:                 483.00 secs
Data transferred:               3.54 MB
Response time:                 41.74 secs
Transaction rate:              20.61 trans/sec
Throughput:                     0.01 MB/sec
Concurrency:                  860.30
Successful transactions:        9954
Failed transactions:              46
Longest transaction:           89.28
Shortest transaction:           0.36
[fk@privoxy-vm ~]$ privoxy-log-parser.pl --detect-inactivity --inactivity-threshold 30000 /var/log/privoxy/privoxy.log | grep -B 3 -A 2 '^Detected inactivity'

privoxy-experimental-ports-openssl 3.0.33.20210321_1 with OpenSSL 1.1.1j,1

[fk@privoxy-vm ~]$ privoxy-log-parser.pl --detect-inactivity --inactivity-threshold 30000 /var/log/privoxy/privoxy.log | grep -B 3 -A 2 '^Detected inactivity'
[fk@benchmark-vm ~]$ siege --concurrent=1000 --reps=10 --benchmark "https://www.electrobsd.org/"
** SIEGE 4.0.7
** Preparing 1000 concurrent users for battle.
The server is now under siege...
Transactions:                   9944 hits
Availability:                  99.44 %
Elapsed time:                 422.99 secs
Data transferred:               3.60 MB
Response time:                 33.64 secs
Transaction rate:              23.51 trans/sec
Throughput:                     0.01 MB/sec
Concurrency:                  790.90
Successful transactions:        9944
Failed transactions:              56
Longest transaction:           77.73
Shortest transaction:           0.11

privoxy-experimental-ports-openssl-devel 3.0.33.20210321_1 with OpenSSL devel 3.0.0.a13

[fk@benchmark-vm ~]$ siege --concurrent=1000 --reps=10 --benchmark "https://www.electrobsd.org/"
** SIEGE 4.0.7
** Preparing 1000 concurrent users for battle.
The server is now under siege...
Transactions:                   9892 hits
Availability:                  98.92 %
Elapsed time:                 589.88 secs
Data transferred:               3.94 MB
Response time:                 50.67 secs
Transaction rate:              16.77 trans/sec
Throughput:                     0.01 MB/sec
Concurrency:                  849.68
Successful transactions:        9892
Failed transactions:             108
Longest transaction:          166.45
Shortest transaction:           0.20
[fk@privoxy-vm ~]$ sort /var/log/privoxy/privoxy.log | privoxy-log-parser.pl --detect-inactivity --inactivity-threshold 30000 | grep -B 3 -A 2 '^Detected inactivity'

I've added the sort command as there were log messages written out of the chronological order which caused false positives when parsing the unsorted file.

privoxy-experimental-wolfssl 3.0.33.20210321_1 with wolfSSL 4.7.0_6

[fk@benchmark-vm ~]$ siege --concurrent=1000 --reps=10 --benchmark "https://www.electrobsd.org/"
** SIEGE 4.0.7
** Preparing 1000 concurrent users for battle.
The server is now under siege...
Transactions:                   9985 hits
Availability:                  99.85 %
Elapsed time:                 577.09 secs
Data transferred:               3.33 MB
Response time:                 26.49 secs
Transaction rate:              17.30 trans/sec
Throughput:                     0.01 MB/sec
Concurrency:                  458.35
Successful transactions:        9756
Failed transactions:              15
Longest transaction:          443.84
Shortest transaction:           0.06
[fk@privoxy-vm ~]$ sort /var/log/privoxy/privoxy.log | privoxy-log-parser.pl --detect-inactivity --inactivity-threshold 30000 | grep -B 3 -A 2 '^Detected inactivity'
2021-03-22 09:20:19.394 812728f00 Crunch: Connection failure: https://www.electrobsd.org/
2021-03-22 09:20:19.395 812728f00 Connect: Failed to shutdown client connection on socket 129. Attempts so far: 1, ret: 2
2021-03-22 09:20:19.395 812728f00 Error: Failed to shutdown client connection on socket 129 after 1 attempts. ret: 2, error: 0, unknown error number
Detected inactivity: 156418 msecs
2021-03-22 09:22:55.813 8083d4f00 Error: X509 certificate verification for www.electrobsd.org failed with error -308: error state on socket
2021-03-22 09:22:55.814 8083d4f00 Connect: Closing client socket 1232. Keep-alive: 0. Socket alive: 1. Data available: 0. Configuration file change detected: 0. Requests received: 1.

Looks like the issue isn't actually caused by the wolfSSL-specific ctx-reuse hack which is no longer present.

privoxy-experimental-mbedtls 3.0.33.20210321_1 with mbedtls 2.16.10

[fk@benchmark-vm ~]$ siege --concurrent=1000 --reps=10 --benchmark "https://www.electrobsd.org/"
** SIEGE 4.0.7
** Preparing 1000 concurrent users for battle.
The server is now under siege...
Transactions:                   9991 hits
Availability:                  99.91 %
Elapsed time:                1188.16 secs
Data transferred:               3.30 MB
Response time:                108.66 secs
Transaction rate:               8.41 trans/sec
Throughput:                     0.00 MB/sec
Concurrency:                  913.69
Successful transactions:        9991
Failed transactions:               9
Longest transaction:          153.42
Shortest transaction:           0.70
[fk@privoxy-vm ~]$ sort /var/log/privoxy/privoxy.log | privoxy-log-parser.pl --detect-inactivity --inactivity-threshold 30000 | grep -B 3 -A 2 '^Detected inactivity'

Privoxy CPU time and resident memory use according to top after the test

TLS libraryCPU timeMemory size
OpenSSL 1.0.2t5:14344M
LibreSSL 3.2.56:26345M
LibreSSL devel 3.3.17:04366M
OpenSSL 1.1.1j,15:45364M
OpenSSL devel 3.0.0.a138:15453M
wolfSSL 4.7.0_64:00460M
mbedtls 2.16.1018:3195860K

Connection: close, concurrency level 1

In the siege configuration I've disabled the parser mode which may or may not speed up siege. The ElectroBSD homepage doesn't use external images or CSS files so the number of requests shouldn't be affected.

The mbedTLS port has been updated to remove a patch coming from FreeBSD that adds "DTLS-SRTP (RFC 5764)" support. Privoxy isn't using DTLS but it was worth a try. The privoxy-experimental-mbedtls port was recompiled due to the dependency change.

The siege concurrency has been reduced to 1 and the repetitions set to 1000 (so the number of requests is reduced to ~1000). While the tests were running the systems were mostly idle, presumably waiting for the network.

It looks like siege has socket management issues that should be investigated further.

privoxy-experimental 3.0.33.20210321_1 with OpenSSL 1.0.2t-ElectroBSD from base

[fk@benchmark-vm ~]$ siege --concurrent=1 --reps=1000 --benchmark "https://www.electrobsd.org/"
** SIEGE 4.0.7
** Preparing 1 concurrent users for battle.
The server is now under siege...[alert] socket: select and discovered it's not ready sock.c:384: Operation timed out
[alert] socket: read check timed out(30) sock.c:273: Operation timed out

Transactions:                    999 hits
Availability:                  99.90 %
Elapsed time:                 178.88 secs
Data transferred:               0.32 MB
Response time:                  0.15 secs
Transaction rate:               5.58 trans/sec
Throughput:                     0.00 MB/sec
Concurrency:                    0.82
Successful transactions:         999
Failed transactions:               1
Longest transaction:           15.92
Shortest transaction:           0.07

privoxy-experimental-libressl 3.0.33.20210321_1 with LibreSSL 3.2.5

[fk@benchmark-vm ~]$ siege --concurrent=1 --reps=1000 --benchmark "https://www.electrobsd.org/"
** SIEGE 4.0.7
** Preparing 1 concurrent users for battle.
The server is now under siege...[alert] socket: select and discovered it's not ready sock.c:384: Operation timed out
[alert] socket: read check timed out(30) sock.c:273: Operation timed out
[alert] socket: select and discovered it's not ready sock.c:384: Operation timed out
[alert] socket: read check timed out(30) sock.c:273: Operation timed out

Transactions:                    998 hits
Availability:                  99.80 %
Elapsed time:                 220.91 secs
Data transferred:               0.32 MB
Response time:                  0.16 secs
Transaction rate:               4.52 trans/sec
Throughput:                     0.00 MB/sec
Concurrency:                    0.72
Successful transactions:         998
Failed transactions:               2
Longest transaction:            6.31
Shortest transaction:           0.07

privoxy-experimental-libressl-devel 3.0.33.20210321_1 with LibreSSL devel 3.3.1

[fk@benchmark-vm ~]$ siege --concurrent=1 --reps=1000 --benchmark "https://www.electrobsd.org/"
** SIEGE 4.0.7
** Preparing 1 concurrent users for battle.
The server is now under siege...
Transactions:                    999 hits
Availability:                  99.90 %
Elapsed time:                 160.31 secs
Data transferred:               0.33 MB
Response time:                  0.16 secs
Transaction rate:               6.23 trans/sec
Throughput:                     0.00 MB/sec
Concurrency:                    0.98
Successful transactions:         999
Failed transactions:               1
Longest transaction:           30.07
Shortest transaction:           0.08

privoxy-experimental-ports-openssl 3.0.33.20210321_1 with OpenSSL 1.1.1j,1

[fk@benchmark-vm ~]$ siege --concurrent=1 --reps=1000 --benchmark "https://www.electrobsd.org/"
** SIEGE 4.0.7
** Preparing 1 concurrent users for battle.
The server is now under siege...[alert] socket: select and discovered it's not ready sock.c:384: Operation timed out
[alert] socket: read check timed out(30) sock.c:273: Operation timed out
[alert] socket: select and discovered it's not ready sock.c:384: Operation timed out
[alert] socket: read check timed out(30) sock.c:273: Operation timed out

Transactions:                    998 hits
Availability:                  99.80 %
Elapsed time:                 200.48 secs
Data transferred:               0.32 MB
Response time:                  0.14 secs
Transaction rate:               4.98 trans/sec
Throughput:                     0.00 MB/sec
Concurrency:                    0.69
Successful transactions:         998
Failed transactions:               2
Longest transaction:            1.13
Shortest transaction:           0.07

privoxy-experimental-ports-openssl-devel 3.0.33.20210321_1 with OpenSSL devel 3.0.0.a13

[fk@benchmark-vm ~]$ siege --concurrent=1 --reps=1000 --benchmark "https://www.electrobsd.org/"
** SIEGE 4.0.7
** Preparing 1 concurrent users for battle.
The server is now under siege...
Transactions:                   1000 hits
Availability:                 100.00 %
Elapsed time:                 134.23 secs
Data transferred:               0.32 MB
Response time:                  0.13 secs
Transaction rate:               7.45 trans/sec
Throughput:                     0.00 MB/sec
Concurrency:                    0.98
Successful transactions:        1000
Failed transactions:               0
Longest transaction:            0.61
Shortest transaction:           0.08

privoxy-experimental-wolfssl 3.0.33.20210321_1 with wolfSSL 4.7.0_6

[fk@benchmark-vm ~]$ siege --concurrent=1 --reps=1000 --benchmark "https://www.electrobsd.org/"
** SIEGE 4.0.7
** Preparing 1 concurrent users for battle.
The server is now under siege...
Transactions:                   1000 hits
Availability:                 100.00 %
Elapsed time:                 124.91 secs
Data transferred:               0.32 MB
Response time:                  0.12 secs
Transaction rate:               8.01 trans/sec
Throughput:                     0.00 MB/sec
Concurrency:                    1.00
Successful transactions:        1000
Failed transactions:               0
Longest transaction:            5.10
Shortest transaction:           0.06

privoxy-experimental-mbedtls 3.0.33.20210321_1 with mbedtls 2.16.10_1

[fk@benchmark-vm ~]$ siege --concurrent=1 --reps=1000 --benchmark "https://www.electrobsd.org/"
** SIEGE 4.0.7
** Preparing 1 concurrent users for battle.
The server is now under siege...
Transactions:                   1000 hits
Availability:                 100.00 %
Elapsed time:                 248.80 secs
Data transferred:               0.32 MB
Response time:                  0.25 secs
Transaction rate:               4.02 trans/sec
Throughput:                     0.00 MB/sec
Concurrency:                    1.00
Successful transactions:        1000
Failed transactions:               0
Longest transaction:            3.54
Shortest transaction:           0.16

Privoxy CPU time and resident memory use according to top after the test

TLS libraryCPU timeMemory size
OpenSSL 1.0.2t0:2525768K
LibreSSL 3.2.50:3120312K
LibreSSL devel 3.3.10:3421620K
OpenSSL 1.1.1j,10:2726516K
OpenSSL devel 3.0.0.a130:3625756K
wolfSSL 4.7.0_60:168420K
mbedtls 2.16.10_11:477852K

Privoxy 3.0.33 experimental 20210323 e8e838816b tested with siege 4.0.7_3

The Privoxy ports have been updated. The wolfSSL-specific ctx-reuse hack is back but hidden behind a configuration directive so it can be enabled at runtime.

Connection: close, concurrency level 1000

privoxy-experimental 3.0.33.20210323 with OpenSSL 1.0.2t-ElectroBSD from base

[fk@benchmark-vm ~]$ siege --concurrent=1000 --reps=10 --benchmark "https://www.electrobsd.org/"
** SIEGE 4.0.7
** Preparing 1000 concurrent users for battle.
The server is now under siege...
Transactions:                   9962 hits
Availability:                  99.62 %
Elapsed time:                 372.49 secs
Data transferred:               3.49 MB
Response time:                 28.29 secs
Transaction rate:              26.74 trans/sec
Throughput:                     0.01 MB/sec
Concurrency:                  756.72
Successful transactions:        9962
Failed transactions:              38
Longest transaction:           70.66
Shortest transaction:           0.08

privoxy-experimental-libressl 3.0.33.20210323 with LibreSSL 3.2.5

[fk@benchmark-vm ~]$ siege --concurrent=1000 --reps=10 --benchmark "https://www.electrobsd.org/"
** SIEGE 4.0.7
** Preparing 1000 concurrent users for battle.
The server is now under siege...
Transactions:                   9912 hits
Availability:                  99.12 %
Elapsed time:                 448.02 secs
Data transferred:               3.80 MB
Response time:                 36.20 secs
Transaction rate:              22.12 trans/sec
Throughput:                     0.01 MB/sec
Concurrency:                  800.97
Successful transactions:        9912
Failed transactions:              88
Longest transaction:           78.40
Shortest transaction:           0.08

privoxy-experimental-libressl-devel 3.0.33.20210323 with LibreSSL devel 3.3.1

[fk@benchmark-vm ~]$ siege --concurrent=1000 --reps=10 --benchmark "https://www.electrobsd.org/"
** SIEGE 4.0.7
** Preparing 1000 concurrent users for battle.
The server is now under siege...
Transactions:                   9939 hits
Availability:                  99.39 %
Elapsed time:                 492.29 secs
Data transferred:               3.63 MB
Response time:                 40.01 secs
Transaction rate:              20.19 trans/sec
Throughput:                     0.01 MB/sec
Concurrency:                  807.72
Successful transactions:        9939
Failed transactions:              61
Longest transaction:           80.22
Shortest transaction:           0.12

privoxy-experimental-ports-openssl 3.0.33.20210323 with OpenSSL 1.1.1j,1

[fk@benchmark-vm ~]$ siege --concurrent=1000 --reps=10 --benchmark "https://www.electrobsd.org/"
** SIEGE 4.0.7
** Preparing 1000 concurrent users for battle.
The server is now under siege...
Transactions:                   9903 hits
Availability:                  99.03 %
Elapsed time:                 481.68 secs
Data transferred:               3.86 MB
Response time:                 37.87 secs
Transaction rate:              20.56 trans/sec
Throughput:                     0.01 MB/sec
Concurrency:                  778.49
Successful transactions:        9903
Failed transactions:              97
Longest transaction:          115.86
Shortest transaction:           0.12

privoxy-experimental-ports-openssl-devel 3.0.33.20210323 with OpenSSL devel 3.0.0.a13

[fk@benchmark-vm ~]$ siege --concurrent=1000 --reps=10 --benchmark "https://www.electrobsd.org/"
** SIEGE 4.0.7
** Preparing 1000 concurrent users for battle.
The server is now under siege...
Transactions:                   9958 hits
Availability:                  99.58 %
Elapsed time:                 511.93 secs
Data transferred:               3.51 MB
Response time:                 42.92 secs
Transaction rate:              19.45 trans/sec
Throughput:                     0.01 MB/sec
Concurrency:                  834.92
Successful transactions:        9958
Failed transactions:              42
Longest transaction:           92.80
Shortest transaction:           0.09

privoxy-experimental-wolfssl 3.0.33.20210323 with wolfSSL 4.7.0_6 and separate server WOLFSSL_CTXs

[fk@benchmark-vm ~]$ siege --concurrent=1000 --reps=10 --benchmark "https://www.electrobsd.org/"
** SIEGE 4.0.7
** Preparing 1000 concurrent users for battle.
The server is now under siege...
Transactions:                   9967 hits
Availability:                  99.67 %
Elapsed time:                 514.52 secs
Data transferred:               3.45 MB
Response time:                 26.77 secs
Transaction rate:              19.37 trans/sec
Throughput:                     0.01 MB/sec
Concurrency:                  518.50
Successful transactions:        9899
Failed transactions:              33
Longest transaction:          411.27
Shortest transaction:           0.07

Unfortunately I failed to run Privoxy-Log-Parser to investigate the periods of inactivity that occurred. I've repeated the test out of order.

[fk@benchmark-vm ~]$ siege --concurrent=1000 --reps=10 --benchmark "https://www.electrobsd.org/"
** SIEGE 4.0.7
** Preparing 1000 concurrent users for battle.
The server is now under siege...[alert] socket: select and discovered it's not ready sock.c:384: Operation timed out
[alert] socket: read check timed out(30) sock.c:273: Operation timed out

Transactions:                   9928 hits
Availability:                  99.28 %
Elapsed time:                 563.20 secs
Data transferred:               3.69 MB
Response time:                 25.84 secs
Transaction rate:              17.63 trans/sec
Throughput:                     0.01 MB/sec
Concurrency:                  455.42
Successful transactions:        9784
Failed transactions:              72
Longest transaction:          411.16
Shortest transaction:           0.08
[fk@privoxy-vm ~]$ sort /var/log/privoxy/privoxy.log | privoxy-log-parser.pl --detect-inactivity --inactivity-threshold 10000 | grep -B 3 -A 2 '^Detected inactivity'
2021-03-23 09:18:43.600 82bd00f00 Connect: Failed to shutdown client connection on socket 26. Attempts so far: 1, ret: 2
2021-03-23 09:18:43.600 82bd00f00 Crunch: Connection failure: https://www.electrobsd.org/
2021-03-23 09:18:43.600 82bd00f00 Error: Failed to shutdown client connection on socket 26 after 1 attempts. ret: 2, error: 0, unknown error number
Detected inactivity: 12798 msecs
2021-03-23 09:18:56.398 806d7ed00 Connect: Could not connect to [www.electrobsd.org]:443: Operation timed out.
2021-03-23 09:18:56.400 806d7ed00 Connect: Failed to shutdown client connection on socket 121. Attempts so far: 1, ret: 2
--
2021-03-23 09:18:57.750 821f76400 Connect: Failed to shutdown client connection on socket 162. Attempts so far: 1, ret: 2
2021-03-23 09:18:57.750 821f76400 Crunch: Connection failure: https://www.electrobsd.org/
2021-03-23 09:18:57.750 821f76400 Error: Failed to shutdown client connection on socket 162 after 1 attempts. ret: 2, error: 0, unknown error number
Detected inactivity: 157780 msecs
2021-03-23 09:21:35.530 822afd300 Error: X509 certificate verification for www.electrobsd.org failed with error -308: error state on socket
2021-03-23 09:21:35.531 822afd300 Connect: Closing client socket 1336. Keep-alive: 0. Socket alive: 1. Data available: 0. Configuration file change detected: 0. Requests received: 1.
--
2021-03-23 09:21:52.642 8054c5600 Connect: Failed to shutdown server connection on socket 9. Attempts so far: 1, ret: 2
2021-03-23 09:21:52.642 8054c5600 Error: Failed to shutdown client connection on socket 8 after 1 attempts. ret: 2, error: 0, unknown error number
2021-03-23 09:21:52.642 8054c5600 Error: Failed to shutdown server connection on socket 9 after 1 attempts. ret: 2, error: 0, unknown error number
Detected inactivity: 14096 msecs
2021-03-23 09:22:06.738 81a8a2e00 Connect: Could not connect to [www.electrobsd.org]:443: Operation timed out.
2021-03-23 09:22:06.740 81a8a2e00 Crunch: Connection failure: https://www.electrobsd.org/
--
2021-03-23 09:22:53.093 82c91b900 Connect: Failed to shutdown server connection on socket 6. Attempts so far: 1, ret: 2
2021-03-23 09:22:53.093 82c91b900 Error: Failed to shutdown client connection on socket 5 after 1 attempts. ret: 2, error: 0, unknown error number
2021-03-23 09:22:53.093 82c91b900 Error: Failed to shutdown server connection on socket 6 after 1 attempts. ret: 2, error: 0, unknown error number
Detected inactivity: 10075 msecs
2021-03-23 09:23:03.168 82d69c300 Connect: Could not connect to [www.electrobsd.org]:443: Operation timed out.
2021-03-23 09:23:03.170 82d69c300 Connect: Failed to shutdown client connection on socket 25. Attempts so far: 1, ret: 2
  
[fk@privoxy-vm ~]$ grep -1000 '2021-03-23 09:21:35.530 822afd300 Error: X509 certificate' /var/log/privoxy/privoxy.log | privoxy-log-parser.pl --detect-inactivity --inactivity-threshold 10000 | grep -B 3 -A 2 '^Detected inactivity'
2021-03-23 09:18:43.600 82bd00f00 Crunch: Connection failure: https://www.electrobsd.org/
2021-03-23 09:18:43.600 82bd00f00 Connect: Failed to shutdown client connection on socket 26. Attempts so far: 1, ret: 2
2021-03-23 09:18:43.600 82bd00f00 Error: Failed to shutdown client connection on socket 26 after 1 attempts. ret: 2, error: 0, unknown error number
Detected inactivity: 12798 msecs
2021-03-23 09:18:56.398 806d7ed00 Connect: Could not connect to [www.electrobsd.org]:443: Operation timed out.
2021-03-23 09:18:56.400 806d7ed00 Crunch: Connection failure: https://www.electrobsd.org/
--
2021-03-23 09:18:57.750 821f76400 Crunch: Connection failure: https://www.electrobsd.org/
2021-03-23 09:18:57.750 821f76400 Connect: Failed to shutdown client connection on socket 162. Attempts so far: 1, ret: 2
2021-03-23 09:18:57.750 821f76400 Error: Failed to shutdown client connection on socket 162 after 1 attempts. ret: 2, error: 0, unknown error number
Detected inactivity: 157780 msecs
2021-03-23 09:21:35.530 822afd300 Error: X509 certificate verification for www.electrobsd.org failed with error -308: error state on socket
2021-03-23 09:21:35.531 822afd300 Crunch: Certificate error: error state on socket: https://www.electrobsd.org/

privoxy-experimental-wolfssl 3.0.33.20210323 with wolfSSL 4.7.0_6 and shared server WOLFSSL_CTX

[fk@benchmark-vm ~]$ siege --concurrent=1000 --reps=10 --benchmark "https://www.electrobsd.org/"
** SIEGE 4.0.7
** Preparing 1000 concurrent users for battle.
The server is now under siege...
Transactions:                   9951 hits
Availability:                  99.51 %
Elapsed time:                 579.41 secs
Data transferred:               3.54 MB
Response time:                 28.32 secs
Transaction rate:              17.17 trans/sec
Throughput:                     0.01 MB/sec
Concurrency:                  486.46
Successful transactions:        9552
Failed transactions:              49
Longest transaction:          427.17
Shortest transaction:           0.05
[fk@privoxy-vm ~]$ sort /var/log/privoxy/privoxy.log | privoxy-log-parser.pl --detect-inactivity --inactivity-threshold 10000 | grep -B 3 -A 2 '^Detected inactivity'
2021-03-23 08:32:10.885 804735900 Header: scan: Transfer-Encoding: chunked
2021-03-23 08:32:10.886 804735900 Connect: Failed to shutdown client connection on socket 5. Attempts so far: 1, ret: 2
2021-03-23 08:32:10.886 804735900 Error: Failed to shutdown client connection on socket 5 after 1 attempts. ret: 2, error: 0, unknown error number
Detected inactivity: 11460 msecs
2021-03-23 08:32:22.346 81034f300 Connect: Connected to www.electrobsd.org[95.211.138.51]:443.
2021-03-23 08:32:22.346 81034f300 Connect: Created new connection to www.electrobsd.org:443 on socket 731.
--
2021-03-23 08:32:27.400 8253c8300 Crunch: Connection failure: https://www.electrobsd.org/
2021-03-23 08:32:27.401 8253c8300 Connect: Failed to shutdown client connection on socket 230. Attempts so far: 1, ret: 2
2021-03-23 08:32:27.401 8253c8300 Error: Failed to shutdown client connection on socket 230 after 1 attempts. ret: 2, error: 0, unknown error number
Detected inactivity: 147648 msecs
2021-03-23 08:34:55.049 8142e5d00 Connect: Failed to shutdown client connection on socket 477. Attempts so far: 1, ret: 2
2021-03-23 08:34:55.049 8142e5d00 Crunch: Certificate error: error state on socket: https://www.electrobsd.org/
--
2021-03-23 08:36:58.522 816aab500 Header: scan: Last-Modified: Sun, 23 Aug 2020 16:46:10 GMT
2021-03-23 08:36:58.522 816aab500 Header: scan: Server: nginx
2021-03-23 08:36:58.522 816aab500 Header: scan: Transfer-Encoding: chunked
Detected inactivity: 34004 msecs
2021-03-23 08:37:32.526 8094bd700 Connect: Closing client socket 917. Keep-alive: 0. Socket alive: 1. Data available: 0. Configuration file change detected: 0. Requests received: 1.
2021-03-23 08:37:32.526 8094bd700 Connect: Closing server socket 1017 connected to www.electrobsd.org. Keep-alive: 0. Tainted: 1. Socket alive: 0. Timeout: 5.

Due to the sorting the messages with the same time stamp and thread id may appear out of chronological order! Unfortunately the log file is already overwritten.

The unknown error number is the result of the unpolished patch to only call wolfSSL_shutdown() once which the port still contains. I'll remove it in the next update as it doesn't seem to work around the problem.

privoxy-experimental-mbedtls 3.0.33.20210323 with mbedtls 2.16.10_1

[fk@benchmark-vm ~]$ siege --concurrent=1000 --reps=10 --benchmark "https://www.electrobsd.org/"
** SIEGE 4.0.7
** Preparing 1000 concurrent users for battle.
The server is now under siege...
Transactions:                   9998 hits
Availability:                  99.98 %
Elapsed time:                1190.16 secs
Data transferred:               3.26 MB
Response time:                108.83 secs
Transaction rate:               8.40 trans/sec
Throughput:                     0.00 MB/sec
Concurrency:                  914.25
Successful transactions:        9998
Failed transactions:               2
Longest transaction:          167.85
Shortest transaction:           0.18

Privoxy CPU time and resident memory use according to top after the test

TLS libraryCPU timeMemory size
OpenSSL 1.0.2t5:08382M
LibreSSL 3.2.56:09360M
LibreSSL devel 3.3.16:48319M
OpenSSL 1.1.1j,16:38449M
OpenSSL devel 3.0.0.a137:11300M
wolfSSL 4.7.0_6 (separate server WOLFSSL_CTX) 14:25706M
wolfSSL 4.7.0_6 (separate server WOLFSSL_CTX) 24:19840M
wolfSSL 4.7.0_6 (shared server WOLFSSL_CTX)3:26507M
mbedtls 2.16.10_118:2899024K

Privoxy 3.0.33 experimental 20210323_1 e8e838816b tested with siege 4.0.7_4

The Privoxy ports have been updated and no longer contain the wolfSSL-specific patch to only call wolfSSL_shutdown() once.

In the Privoxy VM the previously auto-tuned ZFS ARC size has been limited by setting the vfs.zfs.arc_max sysctl to 256 MB.

The siege port has been updated to include a patch to enable keep-alive mode on the command line but I have not yet investigated why the concurrency level isn't respected when using keep-alive. It looks like the performance of the system under test affects how many concurrent connections siege is using.

Connection: keep-alive, concurrency level 1000 (not actually respected)

privoxy-experimental 3.0.33.20210323_1 with OpenSSL 1.0.2t-ElectroBSD from base

[fk@benchmark-vm ~]$ siege --concurrent=1000 --reps=10 --benchmark --keep-alive "https://www.electrobsd.org/"
** SIEGE 4.0.7
** Preparing 1000 concurrent users for battle.
The server is now under siege...
Transactions:                  10000 hits
Availability:                 100.00 %
Elapsed time:                  91.76 secs
Data transferred:               3.24 MB
Response time:                  0.06 secs
Transaction rate:             108.98 trans/sec
Throughput:                     0.04 MB/sec
Concurrency:                    6.32
Successful transactions:       10000
Failed transactions:               0
Longest transaction:            1.54
Shortest transaction:           0.01

privoxy-experimental-libressl 3.0.33.20210323_1 with LibreSSL 3.2.5

[fk@benchmark-vm ~]$ siege --concurrent=1000 --reps=10 --benchmark --keep-alive "https://www.electrobsd.org/"
** SIEGE 4.0.7
** Preparing 1000 concurrent users for battle.
The server is now under siege...
Transactions:                  10000 hits
Availability:                 100.00 %
Elapsed time:                  96.30 secs
Data transferred:               3.24 MB
Response time:                  0.27 secs
Transaction rate:             103.84 trans/sec
Throughput:                     0.03 MB/sec
Concurrency:                   28.25
Successful transactions:       10000
Failed transactions:               0
Longest transaction:            7.87
Shortest transaction:           0.01

privoxy-experimental-libressl-devel 3.0.33.20210323_1 with LibreSSL devel 3.3.1

[fk@benchmark-vm ~]$ siege --concurrent=1000 --reps=10 --benchmark --keep-alive "https://www.electrobsd.org/"
** SIEGE 4.0.7
** Preparing 1000 concurrent users for battle.
The server is now under siege...
Transactions:                  10000 hits
Availability:                 100.00 %
Elapsed time:                  91.98 secs
Data transferred:               3.24 MB
Response time:                  0.96 secs
Transaction rate:             108.72 trans/sec
Throughput:                     0.04 MB/sec
Concurrency:                  104.72
Successful transactions:       10000
Failed transactions:               0
Longest transaction:            8.39
Shortest transaction:           0.01

privoxy-experimental-ports-openssl 3.0.33.20210323_1 with OpenSSL 1.1.1j,1

[fk@benchmark-vm ~]$ siege --concurrent=1000 --reps=10 --benchmark --keep-alive "https://www.electrobsd.org/"
** SIEGE 4.0.7
** Preparing 1000 concurrent users for battle.
The server is now under siege...
Transactions:                  10000 hits
Availability:                 100.00 %
Elapsed time:                  89.23 secs
Data transferred:               3.24 MB
Response time:                  0.27 secs
Transaction rate:             112.06 trans/sec
Throughput:                     0.04 MB/sec
Concurrency:                   30.07
Successful transactions:       10000
Failed transactions:               0
Longest transaction:            6.01
Shortest transaction:           0.01

privoxy-experimental-ports-openssl-devel 3.0.33.20210323_1 with OpenSSL devel 3.0.0.a13

[fk@benchmark-vm ~]$ siege --concurrent=1000 --reps=10 --benchmark --keep-alive "https://www.electrobsd.org/"
** SIEGE 4.0.7
** Preparing 1000 concurrent users for battle.
The server is now under siege...
Transactions:                   9996 hits
Availability:                  99.96 %
Elapsed time:                 102.48 secs
Data transferred:               3.24 MB
Response time:                  1.47 secs
Transaction rate:              97.54 trans/sec
Throughput:                     0.03 MB/sec
Concurrency:                  143.65
Successful transactions:        9996
Failed transactions:               4
Longest transaction:           26.71
Shortest transaction:           0.01

privoxy-experimental-wolfssl 3.0.33.20210323_1 with wolfSSL 4.7.0_6 (separate server WOLFSSL_CTX)

[fk@benchmark-vm ~]$ siege --concurrent=1000 --reps=10 --benchmark --keep-alive "https://www.electrobsd.org/"
** SIEGE 4.0.7
** Preparing 1000 concurrent users for battle.
The server is now under siege...
Transactions:                  10000 hits
Availability:                 100.00 %
Elapsed time:                  84.31 secs
Data transferred:               3.24 MB
Response time:                  0.06 secs
Transaction rate:             118.61 trans/sec
Throughput:                     0.04 MB/sec
Concurrency:                    7.60
Successful transactions:       10000
Failed transactions:               0
Longest transaction:            5.20
Shortest transaction:           0.01

privoxy-experimental-wolfssl 3.0.33.20210323_1 with wolfSSL 4.7.0_6 (shared server WOLFSSL_CTX)

[fk@benchmark-vm ~]$ siege --concurrent=1000 --reps=10 --benchmark --keep-alive "https://www.electrobsd.org/"
** SIEGE 4.0.7
** Preparing 1000 concurrent users for battle.
The server is now under siege...
Transactions:                  10000 hits
Availability:                 100.00 %
Elapsed time:                  83.74 secs
Data transferred:               3.24 MB
Response time:                  0.04 secs
Transaction rate:             119.41 trans/sec
Throughput:                     0.04 MB/sec
Concurrency:                    4.69
Successful transactions:       10000
Failed transactions:               0
Longest transaction:            1.35
Shortest transaction:           0.01

privoxy-experimental-mbedtls 3.0.33.20210323_1 with mbedtls 2.16.10_1

[fk@benchmark-vm ~]$ siege --concurrent=1000 --reps=10 --benchmark --keep-alive "https://www.electrobsd.org/"
** SIEGE 4.0.7
** Preparing 1000 concurrent users for battle.
The server is now under siege...
Transactions:                   9976 hits
Availability:                  99.76 %
Elapsed time:                 164.62 secs
Data transferred:               3.23 MB
Response time:                  5.65 secs
Transaction rate:              60.60 trans/sec
Throughput:                     0.02 MB/sec
Concurrency:                  342.47
Successful transactions:        9976
Failed transactions:              24
Longest transaction:          104.38
Shortest transaction:           0.01

Privoxy CPU time and resident memory use according to top after the test

TLS libraryCPU timeMemory size
OpenSSL 1.0.2t0:5934616K
LibreSSL 3.2.51:12131M
LibreSSL devel 3.3.11:16188M
OpenSSL 1.1.1j,11:06150M
OpenSSL devel 3.0.0.a131:23248M
wolfSSL 4.7.0_6 (separate server WOLFSSL_CTX)0:4796380K
wolfSSL 4.7.0_6 (shared server WOLFSSL_CTX)0:3829192K
mbedtls 2.16.10_12:29100M

Connection: close, concurrency level 1000

privoxy-experimental-wolfssl 3.0.33.20210323_1 with wolfSSL 4.7.0_6 (separate server WOLFSSL_CTX)

[fk@benchmark-vm ~]$ siege --concurrent=1000 --reps=10 --benchmark "https://www.electrobsd.org/"
** SIEGE 4.0.7
** Preparing 1000 concurrent users for battle.
The server is now under siege...
Transactions:                   9973 hits
Availability:                  99.73 %
Elapsed time:                 460.06 secs
Data transferred:               3.42 MB
Response time:                 31.50 secs
Transaction rate:              21.68 trans/sec
Throughput:                     0.01 MB/sec
Concurrency:                  682.93
Successful transactions:        9938
Failed transactions:              27
Longest transaction:          143.69
Shortest transaction:           0.06
[fk@privoxy-vm ~]$ sort /var/log/privoxy/privoxy.log | privoxy-log-parser.pl --detect-inactivity --inactivity-threshold 10000 | grep -B 3 -A 2 '^Detected inactivity'
2021-03-23 11:17:08.071 80431a900 Header: scan: HTTP/1.1 200 OK
2021-03-23 11:17:08.071 80b4de200 Header: scan: HTTP/1.1 200 OK
2021-03-23 11:17:08.071 81f34df00 Header: scan: HTTP/1.1 200 OK
Detected inactivity: 26176 msecs
2021-03-23 11:17:34.247 820518500 Connect: Could not connect to [www.electrobsd.org]:443: Operation timed out.
Detected inactivity: 13587 msecs
2021-03-23 11:17:47.834 802c5bb00 Connect: Server successfully connected over TLSv1.2 (ECDHE-RSA-AES128-GCM-SHA256).
2021-03-23 11:17:47.834 820674300 Connect: Server successfully connected over TLSv1.2 (ECDHE-RSA-AES128-GCM-SHA256).
--
2021-03-23 11:21:52.464 81d954100 Header: scan: Last-Modified: Sun, 23 Aug 2020 16:46:10 GMT
2021-03-23 11:21:52.464 81d954100 Header: scan: Server: nginx
2021-03-23 11:21:52.464 81d954100 Header: scan: Transfer-Encoding: chunked
Detected inactivity: 31114 msecs
2021-03-23 11:22:23.578 804fe7400 Connect: Failed to shutdown server connection on socket 73. Attempts so far: 2, ret: 2
2021-03-23 11:22:23.578 804fe7400 Connect: Not shutting down server connection on socket 73. The socket is no longer alive.
[fk@privoxy-vm ~]$ grep -1000 '2021-03-23 11:22:23.578 804fe7400 Connect: Failed' /var/log/privoxy/privoxy.log | privoxy-log-parser.pl --detect-inactivity --inactivity-threshold 10000 | grep -B 3 -A 2 '^Detected inactivity'
2021-03-23 11:21:52.464 81d954100 Connect: Failed to shutdown server connection on socket 6. Attempts so far: 1, ret: 2
2021-03-23 11:21:52.464 81d954100 Connect: Closing client socket 5. Keep-alive: 0. Socket alive: 1. Data available: 0. Configuration file change detected: 0. Requests received: 1.
2021-03-23 11:21:52.464 81d954100 Connect: Failed to shutdown client connection on socket 5. Attempts so far: 1, ret: 2
Detected inactivity: 31114 msecs
2021-03-23 11:22:23.578 804fe7400 Connect: Failed to shutdown server connection on socket 73. Attempts so far: 2, ret: 2
2021-03-23 11:22:23.578 804fe7400 Connect: Not shutting down server connection on socket 73. The socket is no longer alive.

privoxy-experimental-wolfssl 3.0.33.20210323_1 with wolfSSL 4.7.0_6 (shared server WOLFSSL_CTX)

[fk@benchmark-vm ~]$ siege --concurrent=1000 --reps=10 --benchmark "https://www.electrobsd.org/"
** SIEGE 4.0.7
** Preparing 1000 concurrent users for battle.
The server is now under siege...
Transactions:                   9983 hits
Availability:                  99.83 %
Elapsed time:                 774.84 secs
Data transferred:               3.34 MB
Response time:                 35.56 secs
Transaction rate:              12.88 trans/sec
Throughput:                     0.00 MB/sec
Concurrency:                  458.17
Successful transactions:        9652
Failed transactions:              17
Longest transaction:          569.32
Shortest transaction:           0.05

Privoxy went idle several times.

[fk@privoxy-vm ~]$ privoxy-log-parser.pl --detect-inactivity --inactivity-threshold 100000 /var/log/privoxy/privoxy.log | grep -10 '^Detected inactivity'
2021-03-23 11:31:14.949 826cbe400 Header: scan: Transfer-Encoding: chunked
2021-03-23 11:31:14.949 826cbe400 Header: scan: Connection: close
2021-03-23 11:31:14.949 826cbe400 Header: scan: ETag: W/"5f429d52-1fe"
2021-03-23 11:31:14.949 826cbe400 Header: scan: Content-Encoding: gzip
2021-03-23 11:31:14.950 826cbe400 Connect: Looks like we got the last chunk together with the server headers. We better stop reading.
2021-03-23 11:31:14.950 826cbe400 Connect: Done reading from server. Content length: 352 as expected. Bytes most recently read: 352.
2021-03-23 11:31:14.950 826cbe400 Connect: Closing server socket 78 connected to www.electrobsd.org. Keep-alive: 0. Tainted: 0. Socket alive: 1. Timeout: 5.
2021-03-23 11:31:14.950 826cbe400 Connect: Failed to shutdown server connection on socket 78. Attempts so far: 1, ret: 2
2021-03-23 11:31:14.950 826cbe400 Connect: Closing client socket 39. Keep-alive: 0. Socket alive: 1. Data available: 0. Configuration file change detected: 0. Requests received: 1.
2021-03-23 11:31:14.950 826cbe400 Connect: Failed to shutdown client connection on socket 39. Attempts so far: 1, ret: 2
Detected inactivity: 115329 msecs
2021-03-23 11:33:10.279 80257c300 Connect: Failed to shutdown server connection on socket 141. Attempts so far: 2, ret: 2
2021-03-23 11:33:10.279 80257c300 Connect: Not shutting down server connection on socket 141. The socket is no longer alive.
2021-03-23 11:33:10.279 80257c300 Connect: Closing client socket 120. Keep-alive: 0. Socket alive: 1. Data available: 1. Configuration file change detected: 0. Requests received: 1.
2021-03-23 11:33:10.279 80257c300 Connect: Failed to shutdown client connection on socket 120. Attempts so far: 1, ret: 2
2021-03-23 11:33:10.531 802416000 Connect: Waiting for the next client connection. Currently active threads: 338
2021-03-23 11:33:10.531 82b25a900 Connect: Accepted connection from 172.16.1.6 on socket 5
2021-03-23 11:33:10.531 82b25a900 Header: scan: CONNECT www.electrobsd.org:443 HTTP/1.0
2021-03-23 11:33:10.531 82b25a900 Header: scan: User-agent: Proxy-User
2021-03-23 11:33:10.531 82b25a900 Header: addh-unique: Host: www.electrobsd.org:443
2021-03-23 11:33:10.532 82b25a900 Connect: Performing the TLS/SSL handshake with client. Hash of host: 6db5da8a16c246d1bd8c0fa7cd160a5b

Privoxy CPU time and resident memory use according to top after the test

TLS libraryCPU timeMemory size
wolfSSL 4.7.0_6 (separate server WOLFSSL_CTX)5:58987M
wolfSSL 4.7.0_6 (shared server WOLFSSL_CTX)4:21558M

Privoxy 3.0.33 experimental 20210323_2 9e35728c68 tested with siege 4.0.7_4

The Privoxy ports have been updated and now contain a wolfSSL-specific patch that prevents wolfSSL_shutdown() from being called at all to see if that works around the hangs under load (it doesn't).

Connection: close, concurrency level 1000

privoxy-experimental-wolfssl 3.0.33.20210323_2 with wolfSSL 4.7.0_6 (shared server WOLFSSL_CTX)

[fk@benchmark-vm ~]$ siege --concurrent=1000 --reps=10 --benchmark "https://www.electrobsd.org/"
** SIEGE 4.0.7
** Preparing 1000 concurrent users for battle.
The server is now under siege...
Transactions:                   9970 hits
Availability:                  99.70 %
Elapsed time:                 596.25 secs
Data transferred:               3.44 MB
Response time:                 36.24 secs
Transaction rate:              16.72 trans/sec
Throughput:                     0.01 MB/sec
Concurrency:                  605.96
Successful transactions:        9487
Failed transactions:              30
Longest transaction:          463.42
Shortest transaction:           0.06
[fk@privoxy-vm ~]$ sort /var/log/privoxy/privoxy.log |  privoxy-log-parser.pl --detect-inactivity --inactivity-threshold 50000 |  grep -10 '^Detected inactivity'
2021-03-23 12:52:54.219 811897100 Header: scan: Content-Type: text/html
2021-03-23 12:52:54.219 811897100 Header: scan: Date: Tue, 23 Mar 2021 12:00:49 GMT
2021-03-23 12:52:54.219 811897100 Header: scan: ETag: W/"5f429d52-1fe"
2021-03-23 12:52:54.219 811897100 Header: scan: HTTP/1.1 200 OK
2021-03-23 12:52:54.219 811897100 Header: scan: Last-Modified: Sun, 23 Aug 2020 16:46:10 GMT
2021-03-23 12:52:54.219 811897100 Header: scan: Server: nginx
2021-03-23 12:52:54.219 811897100 Header: scan: Transfer-Encoding: chunked
2021-03-23 12:52:54.220 811897100 Connect: Closing client socket 73. Keep-alive: 0. Socket alive: 1. Data available: 0. Configuration file change detected: 0. Requests received: 1.
2021-03-23 12:52:54.220 811897100 Connect: Closing server socket 300 connected to www.electrobsd.org. Keep-alive: 0. Tainted: 0. Socket alive: 1. Timeout: 5.
2021-03-23 12:52:54.220 811897100 Connect: Done reading from server. Content length: 352 as expected. Bytes most recently read: 352.
Detected inactivity: 148298 msecs
2021-03-23 12:55:22.518 81abdd700 Error: X509 certificate verification for www.electrobsd.org failed with error -308: error state on socket
2021-03-23 12:55:22.519 817d88500 Error: X509 certificate verification for www.electrobsd.org failed with error -308: error state on socket
2021-03-23 12:55:22.519 818d88b00 Error: X509 certificate verification for www.electrobsd.org failed with error -308: error state on socket
2021-03-23 12:55:22.519 81abdd700 Connect: Closing server socket 1168 connected to www.electrobsd.org. Keep-alive: 0. Tainted: 1. Socket alive: 0. Timeout: 5.
2021-03-23 12:55:22.519 81abdd700 Crunch: Certificate error: error state on socket: https://www.electrobsd.org/
2021-03-23 12:55:22.520 817d88500 Crunch: Certificate error: error state on socket: https://www.electrobsd.org/
2021-03-23 12:55:22.520 818d88b00 Connect: Closing client socket 1055. Keep-alive: 0. Socket alive: 1. Data available: 0. Configuration file change detected: 0. Requests received: 1.
2021-03-23 12:55:22.520 818d88b00 Connect: Closing server socket 1148 connected to www.electrobsd.org. Keep-alive: 0. Tainted: 1. Socket alive: 0. Timeout: 5.
2021-03-23 12:55:22.520 818d88b00 Crunch: Certificate error: error state on socket: https://www.electrobsd.org/
2021-03-23 12:55:22.520 81abdd700 Connect: Closing client socket 1118. Keep-alive: 0. Socket alive: 1. Data available: 0. Configuration file change detected: 0. Requests received: 1.

Apparently it took nearly six minutes for the TLS handshake to fail but it also took more than 30 seconds for the Privoxy thread to call wolfSSL_connect() after the TCP connection was established:

[fk@privoxy-vm ~]$ grep 81abdd700 /var/log/privoxy/privoxy.log |  privoxy-log-parser.pl --detect-inactivity --inactivity-threshold 5000 |  grep -10 '^Detected inactivity'
2021-03-23 12:48:57.373 81abdd700 Header: scan: Accept-Encoding: gzip, deflate
2021-03-23 12:48:57.373 81abdd700 Header: scan: User-Agent: Mozilla/5.0 (portbld-amd64-freebsd11.4) Siege/4.0.7
2021-03-23 12:48:57.373 81abdd700 Header: scan: Connection: close
2021-03-23 12:48:57.373 81abdd700 Header: Keeping the client header 'Connection: close' around. The connection will not be kept alive.
2021-03-23 12:48:57.373 81abdd700 Header: Encrypted request processed
2021-03-23 12:48:57.373 81abdd700 Request: https://www.electrobsd.org/
2021-03-23 12:48:57.373 81abdd700 Header: New HTTP Request-Line: GET / HTTP/1.1
2021-03-23 12:48:57.373 81abdd700 Connect: to www.electrobsd.org
2021-03-23 12:48:57.403 81abdd700 Connect: Connected to www.electrobsd.org[95.211.138.51]:443.
2021-03-23 12:48:57.461 81abdd700 Connect: Created new connection to www.electrobsd.org:443 on socket 1168.
Detected inactivity: 34221 msecs
2021-03-23 12:49:31.682 81abdd700 Connect: Performing the TLS/SSL handshake with the server
Detected inactivity: 350836 msecs
2021-03-23 12:55:22.518 81abdd700 Error: X509 certificate verification for www.electrobsd.org failed with error -308: error state on socket
2021-03-23 12:55:22.519 81abdd700 Crunch: Certificate error: error state on socket: https://www.electrobsd.org/
2021-03-23 12:55:22.519 81abdd700 Connect: Closing server socket 1168 connected to www.electrobsd.org. Keep-alive: 0. Tainted: 1. Socket alive: 0. Timeout: 5.
2021-03-23 12:55:22.520 81abdd700 Connect: Closing client socket 1118. Keep-alive: 0. Socket alive: 1. Data available: 0. Configuration file change detected: 0. Requests received: 1.

Privoxy CPU time and resident memory use according to top after the test

TLS libraryCPU timeMemory size
wolfSSL 4.7.0_6 (shared server WOLFSSL_CTX)4:37573M

2021-03-24: Privoxy 3.0.33 experimental 20210324 75f66c835 tested with siege 4.0.7_4

The Privoxy ports have been updated. Worth mentioning is that privoxy-experimental-wolfssl calls wolfSSL_shutdown() again but now checks to see if the server socket is still alive before attempting to perform a TLS handshake.

Connection: close, concurrency level 1000

privoxy-experimental-wolfssl 3.0.33.20210324 with wolfSSL 4.7.0_6 (shared server WOLFSSL_CTX) 1

[fk@benchmark-vm ~]$ siege --concurrent=1000 --reps=10 --benchmark "https://www.electrobsd.org/"
** SIEGE 4.0.7
** Preparing 1000 concurrent users for battle.
The server is now under siege...
Transactions:                   9957 hits
Availability:                  99.57 %
Elapsed time:                 869.38 secs
Data transferred:               3.50 MB
Response time:                 52.94 secs
Transaction rate:              11.45 trans/sec
Throughput:                     0.00 MB/sec
Concurrency:                  606.30
Successful transactions:        9405
Failed transactions:              43
Longest transaction:          386.99
Shortest transaction:           0.06
[fk@privoxy-vm ~]$ sort /var/log/privoxy/privoxy-3.0.33.20210324-wolfssl-c1000.log |  privoxy-log-parser.pl --detect-inactivity --inactivity-threshold 50000 |  grep -10 '^Detected inactivity'
2021-03-24 09:23:50.350 81cc24300 Header: scan: ETag: W/"5f429d52-1fe"
2021-03-24 09:23:50.350 81cc24300 Header: scan: HTTP/1.1 200 OK
2021-03-24 09:23:50.350 81cc24300 Header: scan: Last-Modified: Sun, 23 Aug 2020 16:46:10 GMT
2021-03-24 09:23:50.350 81cc24300 Header: scan: Server: nginx
2021-03-24 09:23:50.350 81cc24300 Header: scan: Transfer-Encoding: chunked
2021-03-24 09:23:50.351 81cc24300 Connect: Closing client socket 5. Keep-alive: 0. Socket alive: 1. Data available: 0. Configuration file change detected: 0. Requests received: 1.
2021-03-24 09:23:50.351 81cc24300 Connect: Closing server socket 6 connected to www.electrobsd.org. Keep-alive: 0. Tainted: 0. Socket alive: 1. Timeout: 5.
2021-03-24 09:23:50.351 81cc24300 Connect: Done reading from server. Content length: 352 as expected. Bytes most recently read: 352.
2021-03-24 09:23:50.351 81cc24300 Connect: Failed to shutdown client connection on socket 5. Attempts so far: 1, ret: 2
2021-03-24 09:23:50.351 81cc24300 Connect: Failed to shutdown server connection on socket 6. Attempts so far: 1, ret: 2
Detected inactivity: 124595 msecs
2021-03-24 09:25:54.946 811d37d00 Connect: Failed to shutdown client connection on socket 1507. Attempts so far: 1, ret: 2
2021-03-24 09:25:54.946 811d37d00 Crunch: Certificate error: error state on socket: https://www.electrobsd.org/
2021-03-24 09:25:54.946 811d37d00 Error: X509 certificate verification for www.electrobsd.org failed with error -308: error state on socket
2021-03-24 09:25:54.946 811d3a500 Connect: Failed to shutdown client connection on socket 1511. Attempts so far: 1, ret: 2
2021-03-24 09:25:54.946 811d3a500 Crunch: Certificate error: error state on socket: https://www.electrobsd.org/
2021-03-24 09:25:54.946 811d3a500 Error: X509 certificate verification for www.electrobsd.org failed with error -308: error state on socket
2021-03-24 09:25:54.946 811d3c300 Error: X509 certificate verification for www.electrobsd.org failed with error -308: error state on socket
2021-03-24 09:25:54.947 811d37d00 Connect: Closing client socket 1507. Keep-alive: 0. Socket alive: 1. Data available: 0. Configuration file change detected: 0. Requests received: 1.
2021-03-24 09:25:54.947 811d37d00 Connect: Closing server socket 1476 connected to www.electrobsd.org. Keep-alive: 0. Tainted: 1. Socket alive: 0. Timeout: 5.
2021-03-24 09:25:54.947 811d3c300 Connect: Failed to shutdown client connection on socket 1514. Attempts so far: 1, ret: 2
--
2021-03-24 09:28:18.622 80fafef00 Header: scan: HTTP/1.1 200 OK
2021-03-24 09:28:18.622 80fafef00 Header: scan: Last-Modified: Sun, 23 Aug 2020 16:46:10 GMT
2021-03-24 09:28:18.622 80fafef00 Header: scan: Server: nginx
2021-03-24 09:28:18.622 80fafef00 Header: scan: Transfer-Encoding: chunked
2021-03-24 09:28:18.623 80fafef00 Connect: Closing client socket 5. Keep-alive: 0. Socket alive: 1. Data available: 0. Configuration file change detected: 0. Requests received: 1.
2021-03-24 09:28:18.623 80fafef00 Connect: Closing server socket 6 connected to www.electrobsd.org. Keep-alive: 0. Tainted: 0. Socket alive: 1. Timeout: 5.
2021-03-24 09:28:18.623 80fafef00 Connect: Done reading from server. Content length: 352 as expected. Bytes most recently read: 352.
2021-03-24 09:28:18.623 80fafef00 Connect: Failed to shutdown client connection on socket 5. Attempts so far: 1, ret: 2
2021-03-24 09:28:18.623 80fafef00 Connect: Failed to shutdown server connection on socket 6. Attempts so far: 1, ret: 2
2021-03-24 09:28:18.623 80fafef00 Connect: Looks like we got the last chunk together with the server headers. We better stop reading.
Detected inactivity: 52244 msecs
2021-03-24 09:29:10.867 817106900 Connect: Closing client socket 373. Keep-alive: 0. Socket alive: 1. Data available: 1. Configuration file change detected: 0. Requests received: 1.
2021-03-24 09:29:10.867 817106900 Connect: Failed to shutdown client connection on socket 373. Attempts so far: 1, ret: 2
2021-03-24 09:29:10.867 817106900 Connect: Failed to shutdown server connection on socket 441. Attempts so far: 2, ret: 2
2021-03-24 09:29:10.867 817106900 Connect: Not shutting down server connection on socket 441. The socket is no longer alive.
[fk@privoxy-vm ~]$ grep 'Skipping TLS handshake attempt' /var/log/privoxy/privoxy-3.0.33.20210324-wolfssl-c1000.log

privoxy-experimental-wolfssl 3.0.33.20210324 with wolfSSL 4.7.0_6 (shared server WOLFSSL_CTX) 2

[fk@benchmark-vm ~]$ siege --concurrent=1000 --reps=10 --benchmark "https://www.electrobsd.org/"
** SIEGE 4.0.7
** Preparing 1000 concurrent users for battle.
The server is now under siege...
Transactions:                   9958 hits
Availability:                  99.58 %
Elapsed time:                 885.62 secs
Data transferred:               4.08 MB
Response time:                 37.20 secs
Transaction rate:              11.24 trans/sec
Throughput:                     0.00 MB/sec
Concurrency:                  418.28
Successful transactions:        9368
Failed transactions:              42
Longest transaction:          445.58
Shortest transaction:           0.06
[fk@privoxy-vm ~]$ sort /var/log/privoxy/privoxy-3.0.33.20210324-wolfssl-c1000-test-2.log |  privoxy-log-parser.pl --detect-inactivity --inactivity-threshold 50000 |  grep '^Detected inactivity'
Detected inactivity: 67384 msecs
Detected inactivity: 78784 msecs
Detected inactivity: 262694 msecs
[fk@privoxy-vm ~]$ sort /var/log/privoxy/privoxy-3.0.33.20210324-wolfssl-c1000-test-2.log |  privoxy-log-parser.pl --detect-inactivity --inactivity-threshold 150000 |  grep -10 '^Detected inactivity'
2021-03-24 09:55:07.892 808b62f00 Header: scan: Content-Encoding: gzip
2021-03-24 09:55:07.892 808b62f00 Header: scan: Content-Type: text/html
2021-03-24 09:55:07.892 808b62f00 Header: scan: Date: Wed, 24 Mar 2021 09:03:06 GMT
2021-03-24 09:55:07.892 808b62f00 Header: scan: ETag: W/"5f429d52-1fe"
2021-03-24 09:55:07.892 808b62f00 Header: scan: Last-Modified: Sun, 23 Aug 2020 16:46:10 GMT
2021-03-24 09:55:07.892 808b62f00 Header: scan: Server: nginx
2021-03-24 09:55:07.892 808b62f00 Header: scan: Transfer-Encoding: chunked
2021-03-24 09:55:07.893 808b62f00 Connect: Closing client socket 7. Keep-alive: 0. Socket alive: 1. Data available: 0. Configuration file change detected: 0. Requests received: 1.
2021-03-24 09:55:07.893 808b62f00 Connect: Failed to shutdown client connection on socket 7. Attempts so far: 1, ret: 2
2021-03-24 09:55:07.893 808b62f00 Connect: Failed to shutdown server connection on socket 5. Attempts so far: 1, ret: 2
Detected inactivity: 262694 msecs
2021-03-24 09:59:30.587 816d74d00 Connect: Failed to shutdown server connection on socket 21. Attempts so far: 2, ret: 2
2021-03-24 09:59:30.587 816d74d00 Connect: Not shutting down server connection on socket 21. The socket is no longer alive.
2021-03-24 09:59:30.588 816d74d00 Connect: Closing client socket 24. Keep-alive: 0. Socket alive: 1. Data available: 1. Configuration file change detected: 0. Requests received: 1.
2021-03-24 09:59:30.588 816d74d00 Connect: Failed to shutdown client connection on socket 24. Attempts so far: 1, ret: 2
2021-03-24 09:59:30.591 802416000 Connect: Waiting for the next client connection. Currently active threads: 1
2021-03-24 09:59:30.591 828080700 Connect: Accepted connection from 172.16.1.6 on socket 5
2021-03-24 09:59:30.592 828080700 Header: addh-unique: Host: www.electrobsd.org:443
2021-03-24 09:59:30.592 828080700 Header: scan: CONNECT www.electrobsd.org:443 HTTP/1.0
2021-03-24 09:59:30.592 828080700 Header: scan: User-agent: Proxy-User
2021-03-24 09:59:30.593 828080700 Connect: Performing the TLS/SSL handshake with client. Hash of host: 6db5da8a16c246d1bd8c0fa7cd160a5b
[fk@privoxy-vm ~]$ grep 'Skipping TLS handshake attempt' /var/log/privoxy/privoxy-3.0.33.20210324-wolfssl-c1000-test-2.log

Privoxy CPU time and resident memory use according to top after the test

TLS libraryCPU timeMemory size
wolfSSL 4.7.0_6 (shared server WOLFSSL_CTX) 17:20614M
wolfSSL 4.7.0_6 (shared server WOLFSSL_CTX) 25:08577M

Privoxy 3.0.33 experimental 20210324_1 a25e682b268 tested with siege 4.0.7_4

The Privoxy ports have been updated, the wolfssl-specific code now has an option to control whether or not wolfSSL_shutdown() is called.

The wolfssl port has been recompiled with an updated patch to disable 3DES ciphers independently from 3DES support. The change is not expected to affect performance.

Connection: close, concurrency level 1000

privoxy-experimental-wolfssl 3.0.33.20210324_1 with wolfSSL 4.7.0_7 (shared server WOLFSSL_CTX, shutdown-tls-connections 1)

[fk@benchmark-vm ~]$ siege --concurrent=1000 --reps=10 --benchmark "https://www.electrobsd.org/"
** SIEGE 4.0.7
** Preparing 1000 concurrent users for battle.
The server is now under siege...
Transactions:                   9967 hits
Availability:                  99.67 %
Elapsed time:                 737.46 secs
Data transferred:               3.44 MB
Response time:                 18.42 secs
Transaction rate:              13.52 trans/sec
Throughput:                     0.00 MB/sec
Concurrency:                  248.92
Successful transactions:        9528
Failed transactions:              33
Longest transaction:          262.83
Shortest transaction:           0.06
[fk@privoxy-vm ~]$ sort  /var/log/privoxy/privoxy-3.0.33.20210324_1-wolfssl-tls-shutdown-enabled-c1000.log |  privoxy-log-parser.pl --detect-inactivity --inactivity-threshold 50000 |  grep '^Detected inactivity'
Detected inactivity: 55803 msecs
Detected inactivity: 54447 msecs
Detected inactivity: 88680 msecs
[fk@privoxy-vm ~]$ sort  /var/log/privoxy/privoxy-3.0.33.20210324_1-wolfssl-tls-shutdown-enabled-c1000.log |  privoxy-log-parser.pl --detect-inactivity --inactivity-threshold 80000 |  grep -10 '^Detected inactivity'
2021-03-24 11:21:38.626 82e354b00 Connect: Done reading from server. Content length: 352 as expected. Bytes most recently read: 352.
2021-03-24 11:21:38.626 82e354b00 Connect: Failed to shutdown client connection on socket 5. Attempts so far: 1, ret: 2
2021-03-24 11:21:38.626 82e354b00 Connect: Failed to shutdown server connection on socket 6. Attempts so far: 1, ret: 2
2021-03-24 11:21:38.626 82e354b00 Connect: Looks like we got the last chunk together with the server headers. We better stop reading.
2021-03-24 11:21:38.626 82e354b00 Header: scan: Connection: close
2021-03-24 11:21:38.626 82e354b00 Header: scan: Content-Encoding: gzip
2021-03-24 11:21:38.626 82e354b00 Header: scan: Content-Type: text/html
2021-03-24 11:21:38.626 82e354b00 Header: scan: ETag: W/"5f429d52-1fe"
2021-03-24 11:21:38.626 82e354b00 Header: scan: Last-Modified: Sun, 23 Aug 2020 16:46:10 GMT
2021-03-24 11:21:38.626 82e354b00 Header: scan: Transfer-Encoding: chunked
Detected inactivity: 88680 msecs
2021-03-24 11:23:07.306 8196a0800 Connect: Failed to shutdown server connection on socket 18. Attempts so far: 2, ret: 2
2021-03-24 11:23:07.307 8196a0800 Connect: Closing client socket 16. Keep-alive: 0. Socket alive: 1. Data available: 1. Configuration file change detected: 0. Requests received: 1.
2021-03-24 11:23:07.307 8196a0800 Connect: Failed to shutdown client connection on socket 16. Attempts so far: 1, ret: 2
2021-03-24 11:23:07.307 8196a0800 Connect: Not shutting down server connection on socket 18. The socket is no longer alive.
2021-03-24 11:23:07.309 802416000 Connect: Waiting for the next client connection. Currently active threads: 1
2021-03-24 11:23:07.309 82455a100 Connect: Accepted connection from 172.16.1.6 on socket 5
2021-03-24 11:23:07.309 82455a100 Header: addh-unique: Host: www.electrobsd.org:443
2021-03-24 11:23:07.309 82455a100 Header: scan: CONNECT www.electrobsd.org:443 HTTP/1.0
2021-03-24 11:23:07.309 82455a100 Header: scan: User-agent: Proxy-User
2021-03-24 11:23:07.311 82455a100 Connect: Performing the TLS/SSL handshake with client. Hash of host: 6db5da8a16c246d1bd8c0fa7cd160a5b

privoxy-experimental-wolfssl 3.0.33.20210324_1 with wolfSSL 4.7.0_7 (shared server WOLFSSL_CTX, shutdown-tls-connections 0)

[fk@benchmark-vm ~]$ siege --concurrent=1000 --reps=10 --benchmark "https://www.electrobsd.org/"
** SIEGE 4.0.7
** Preparing 1000 concurrent users for battle.
The server is now under siege...
Transactions:                  10000 hits
Availability:                 100.00 %
Elapsed time:                 590.26 secs
Data transferred:               3.23 MB
Response time:                 28.22 secs
Transaction rate:              16.94 trans/sec
Throughput:                     0.01 MB/sec
Concurrency:                  478.16
Successful transactions:        9601
Failed transactions:               0
Longest transaction:          400.39
Shortest transaction:           0.07
[fk@privoxy-vm ~]$ sort  /var/log/privoxy/privoxy-3.0.33.20210324_1-wolfssl-tls-shutdown-disabled-c1000.log |  privoxy-log-parser.pl --detect-inactivity --inactivity-threshold 10000 | grep '^Detected inactivity'
Detected inactivity: 12664 msecs
Detected inactivity: 205247 msecs
Detected inactivity: 65135 msecs
[fk@privoxy-vm ~]$ sort  /var/log/privoxy/privoxy-3.0.33.20210324_1-wolfssl-tls-shutdown-disabled-c1000.log |  privoxy-log-parser.pl --detect-inactivity --inactivity-threshold 100000 | grep -10 '^Detected inactivity'
2021-03-24 11:33:10.470 82a4cde00 Header: scan: Date: Wed, 24 Mar 2021 10:41:09 GMT
2021-03-24 11:33:10.470 82a4cde00 Header: scan: ETag: W/"5f429d52-1fe"
2021-03-24 11:33:10.470 82a4cde00 Header: scan: HTTP/1.1 200 OK
2021-03-24 11:33:10.470 82a4cde00 Header: scan: Last-Modified: Sun, 23 Aug 2020 16:46:10 GMT
2021-03-24 11:33:10.470 82a4cde00 Header: scan: Server: nginx
2021-03-24 11:33:10.470 82a4cde00 Header: scan: Transfer-Encoding: chunked
2021-03-24 11:33:10.471 82a4cde00 Connect: Closing client socket 7. Keep-alive: 0. Socket alive: 1. Data available: 0. Configuration file change detected: 0. Requests received: 1.
2021-03-24 11:33:10.471 82a4cde00 Connect: Closing server socket 8 connected to www.electrobsd.org. Keep-alive: 0. Tainted: 0. Socket alive: 1. Timeout: 5.
2021-03-24 11:33:10.471 82a4cde00 Connect: Done reading from server. Content length: 352 as expected. Bytes most recently read: 352.
2021-03-24 11:33:10.473 82a4cde00 Connect: Drained 31 bytes before closing socket 7
Detected inactivity: 205247 msecs
2021-03-24 11:36:35.720 817198400 Error: X509 certificate verification for www.electrobsd.org failed with error -308: error state on socket
2021-03-24 11:36:35.721 817198400 Connect: Closing client socket 966. Keep-alive: 0. Socket alive: 1. Data available: 0. Configuration file change detected: 0. Requests received: 1.
2021-03-24 11:36:35.721 817198400 Connect: Closing server socket 1039 connected to www.electrobsd.org. Keep-alive: 0. Tainted: 1. Socket alive: 0. Timeout: 5.
2021-03-24 11:36:35.721 817198400 Crunch: Certificate error: error state on socket: https://www.electrobsd.org/
2021-03-24 11:36:35.726 802416000 Connect: Waiting for the next client connection. Currently active threads: 399
2021-03-24 11:36:35.726 82552cd00 Connect: Accepted connection from 172.16.1.6 on socket 5
2021-03-24 11:36:35.726 82552cd00 Header: scan: CONNECT www.electrobsd.org:443 HTTP/1.0
2021-03-24 11:36:35.726 82552cd00 Header: scan: User-agent: Proxy-User
2021-03-24 11:36:35.727 82552cd00 Header: addh-unique: Host: www.electrobsd.org:443
2021-03-24 11:36:35.728 82552cd00 Connect: Performing the TLS/SSL handshake with client. Hash of host: 6db5da8a16c246d1bd8c0fa7cd160a5b

Privoxy CPU time and resident memory use according to top after the test

TLS libraryCPU timeMemory size
wolfSSL 4.7.0_7 (shared server WOLFSSL_CTX, shutdown-tls-connections 1)2:52569M
wolfSSL 4.7.0_7 (shared server WOLFSSL_CTX, shutdown-tls-connections 0)3:34524M

Apparently not explicitly calling wolfSSL_shutdown() actually increased the cpu time ...

Privoxy 3.0.33 experimental 20210324_2 a25e682b268 tested with siege 4.0.7_4

The privoxy-experimental-wolfssl and wolfssl ports have been recompiled with option DEBUG enabled. This is obviously expected to affect performance but may help to diagnose the hangs.

Connection: close, concurrency level 1000

privoxy-experimental-wolfssl 3.0.33.20210324_2 with wolfSSL 4.7.0_8 (shared server WOLFSSL_CTX, shutdown-tls-connections 1)

[fk@benchmark-vm ~]$ siege --concurrent=1000 --reps=10 --benchmark "https://www.electrobsd.org/"
** SIEGE 4.0.7
** Preparing 1000 concurrent users for battle.
The server is now under siege...
Transactions:                   9971 hits
Availability:                  99.71 %
Elapsed time:                 791.51 secs
Data transferred:               3.41 MB
Response time:                 16.97 secs
Transaction rate:              12.60 trans/sec
Throughput:                     0.00 MB/sec
Concurrency:                  213.72
Successful transactions:        9489
Failed transactions:              29
Longest transaction:          633.22
Shortest transaction:           0.06
[fk@privoxy-vm ~]$ sort  /var/log/privoxy/privoxy-3.0.33.20210324_2-wolfssl-tls-shutdown-enabled-c1000.log |  privoxy-log-parser.pl --detect-inactivity --inactivity-threshold 10000 |  grep '^Detected inactivity'
Detected inactivity: 17208 msecs
Detected inactivity: 12763 msecs
Detected inactivity: 28067 msecs
Detected inactivity: 45745 msecs
Detected inactivity: 20359 msecs
Detected inactivity: 10323 msecs
Detected inactivity: 14201 msecs
Detected inactivity: 15173 msecs
Detected inactivity: 15198 msecs
Detected inactivity: 26831 msecs
Detected inactivity: 25923 msecs
Detected inactivity: 62608 msecs
[fk@privoxy-vm ~]$ sort  /var/log/privoxy/privoxy-3.0.33.20210324_2-wolfssl-tls-shutdown-enabled-c1000.log |  privoxy-log-parser.pl --detect-inactivity --inactivity-threshold 50000 |  grep -10 '^Detected inactivity'
2021-03-24 12:35:23.209 819524b00 Header: scan: ETag: W/"5f429d52-1fe"
2021-03-24 12:35:23.209 819524b00 Header: scan: HTTP/1.1 200 OK
2021-03-24 12:35:23.209 819524b00 Header: scan: Last-Modified: Sun, 23 Aug 2020 16:46:10 GMT
2021-03-24 12:35:23.209 819524b00 Header: scan: Server: nginx
2021-03-24 12:35:23.209 819524b00 Header: scan: Transfer-Encoding: chunked
2021-03-24 12:35:23.210 819524b00 Connect: Closing client socket 7. Keep-alive: 0. Socket alive: 1. Data available: 0. Configuration file change detected: 0. Requests received: 1.
2021-03-24 12:35:23.210 819524b00 Connect: Closing server socket 8 connected to www.electrobsd.org. Keep-alive: 0. Tainted: 0. Socket alive: 1. Timeout: 5.
2021-03-24 12:35:23.210 819524b00 Connect: Done reading from server. Content length: 352 as expected. Bytes most recently read: 352.
2021-03-24 12:35:23.210 819524b00 Connect: Failed to shutdown client connection on socket 7. Attempts so far: 1, ret: 2
2021-03-24 12:35:23.210 819524b00 Connect: Failed to shutdown server connection on socket 8. Attempts so far: 1, ret: 2
Detected inactivity: 62608 msecs
2021-03-24 12:36:25.818 80cc31b00 Connect: Closing client socket 51. Keep-alive: 0. Socket alive: 1. Data available: 1. Configuration file change detected: 0. Requests received: 1.
2021-03-24 12:36:25.818 80cc31b00 Connect: Failed to shutdown client connection on socket 51. Attempts so far: 1, ret: 2
2021-03-24 12:36:25.818 80cc31b00 Connect: Failed to shutdown server connection on socket 166. Attempts so far: 2, ret: 2
2021-03-24 12:36:25.818 80cc31b00 Connect: Not shutting down server connection on socket 166. The socket is no longer alive.
2021-03-24 12:36:25.821 802416000 Connect: Waiting for the next client connection. Currently active threads: 1
2021-03-24 12:36:25.821 8235c3500 Connect: Accepted connection from 172.16.1.6 on socket 5
2021-03-24 12:36:25.822 8235c3500 Header: addh-unique: Host: www.electrobsd.org:443
2021-03-24 12:36:25.822 8235c3500 Header: scan: CONNECT www.electrobsd.org:443 HTTP/1.0
2021-03-24 12:36:25.822 8235c3500 Header: scan: User-agent: Proxy-User
2021-03-24 12:36:25.824 8235c3500 Connect: Performing the TLS/SSL handshake with client. Hash of host: 6db5da8a16c246d1bd8c0fa7cd160a5b

Privoxy CPU time and resident memory use according to top after the test

TLS libraryCPU timeMemory size
wolfSSL 4.7.0_7 (shared server WOLFSSL_CTX, shutdown-tls-connections 1)3:20586M

According to gdb the idling Privoxy had lots of threads blocking like this:

Thread 2 (LWP 100598 of process 83579):
#0  0x000000080197297a in _recvfrom () from /lib/libc.so.7
#1  0x000000080165ecbf in __thr_recvfrom (s=812, b=0x8249b6470, l=5, f=0, from=0x0, fl=0x0) at /usr/src/lib/libthr/thread/thr_syscalls.c:456
#2  0x000000080122c698 in wolfIO_Recv (sd=812, buf=0x8249b6470 "", sz=5, rdFlags=0) at src/wolfio.c:700
#3  0x000000080122c569 in EmbedReceive (ssl=0x8249b6200, buf=0x8249b6470 "", sz=5, ctx=0x8249bf7f4) at src/wolfio.c:233
#4  0x00000008012252d5 in wolfSSLReceive (ssl=0x8249b6200, buf=0x8249b6470 "", sz=5) at src/internal.c:8098
#5  0x0000000801212646 in GetInputData (ssl=0x8249b6200, size=5) at src/internal.c:15090
#6  0x0000000801210d9e in ProcessReply (ssl=0x8249b6200) at src/internal.c:15286
#7  0x0000000801242b93 in wolfSSL_connect (ssl=0x8249b6200) at src/ssl.c:12504
#8  0x0000000000436b5f in ?? ()
#9  0x0000000000423c42 in ?? ()
#10 0x000000080165c08c in thread_start (curthread=0x80a1a9300) at /usr/src/lib/libthr/thread/thr_create.c:290
#11 0x0000000000000000 in ?? ()
Backtrace stopped: Cannot access memory at address 0x7fff8d369000

It also turned out that building the port Privoxy with the DEBUG option enabled did not disable stripping ...

Privoxy 3.0.33 experimental 20210324_4 556c687b tested with siege 4.0.7_4

The privoxy-experimental-wolfssl port has been updated and now contains a patch to optionally use a non-default receive callback with a timeout. The DEBUG option is still being used and stripping has been disabled properly (I hope).

Connection: close, concurrency level 1000

privoxy-experimental-wolfssl 3.0.33.20210324_2 with wolfSSL 4.7.0_8 (shared server WOLFSSL_CTX)

Privoxy was configured to shutdown TLS connections and to use a receive callback timeout of 10 seconds.

[fk@benchmark-vm ~]$ siege --concurrent=1000 --reps=10 --benchmark "https://www.electrobsd.org/"
** SIEGE 4.0.7
** Preparing 1000 concurrent users for battle.
The server is now under siege...
Transactions:                   9934 hits
Availability:                  99.34 %
Elapsed time:                 468.43 secs
Data transferred:               3.63 MB
Response time:                 31.75 secs
Transaction rate:              21.21 trans/sec
Throughput:                     0.01 MB/sec
Concurrency:                  673.42
Successful transactions:        8958
Failed transactions:              66
Longest transaction:           89.88
Shortest transaction:           0.07

The Longest transaction and the Elapsed time went both down. I love it when a plan comes together.

[fk@privoxy-vm ~]$ sort  /var/log/privoxy/privoxy-3.0.33.20210324_4-wolfssl-tls-callback-timeout-10-c1000.log |  privoxy-log-parser.pl --detect-inactivity --inactivity-threshold 10000 |  grep -2 '^Detected inactivity'
2021-03-24 15:48:51.240 814505300 Connect: Server successfully connected over TLSv1.2 (ECDHE-RSA-AES128-GCM-SHA256).
2021-03-24 15:48:52.467 817210c00 Connect: Connected to www.electrobsd.org[95.211.138.51]:443.
Detected inactivity: 12120 msecs
2021-03-24 15:49:04.587 80c6d0300 Connect: Server successfully connected over TLSv1.2 (ECDHE-RSA-AES128-GCM-SHA256).
2021-03-24 15:49:04.591 80b656500 Connect: Server successfully connected over TLSv1.2 (ECDHE-RSA-AES128-GCM-SHA256).
--
2021-03-24 15:51:20.017 80c40ee00 Connect: Connected to www.electrobsd.org[95.211.138.51]:443.
2021-03-24 15:51:20.076 814344e00 Connect: Connected to www.electrobsd.org[95.211.138.51]:443.
Detected inactivity: 17795 msecs
2021-03-24 15:51:37.871 8150c6b00 Error: Gave up waiting for TLS data. Timeout: 10 seconds.
2021-03-24 15:51:37.871 81cee5000 Error: Gave up waiting for TLS data. Timeout: 10 seconds.
--
2021-03-24 15:53:07.036 813f5d200 Connect: Failed to shutdown server connection on socket 194. Attempts so far: 6, ret: 2
2021-03-24 15:53:07.036 813f5d200 Error: Gave up waiting for TLS data. Timeout: 10 seconds.
Detected inactivity: 10066 msecs
2021-03-24 15:53:17.102 813f5d200 Connect: Failed to shutdown server connection on socket 194. Attempts so far: 7, ret: 2
2021-03-24 15:53:17.102 813f5d200 Error: Gave up waiting for TLS data. Timeout: 10 seconds.
Detected inactivity: 10034 msecs
2021-03-24 15:53:27.136 813f5d200 Connect: Failed to shutdown server connection on socket 194. Attempts so far: 8, ret: 2
2021-03-24 15:53:27.136 813f5d200 Error: Gave up waiting for TLS data. Timeout: 10 seconds.
Detected inactivity: 10030 msecs
2021-03-24 15:53:37.166 813f5d200 Connect: Failed to shutdown server connection on socket 194. Attempts so far: 9, ret: 2
2021-03-24 15:53:37.166 813f5d200 Error: Gave up waiting for TLS data. Timeout: 10 seconds.
Detected inactivity: 10012 msecs
2021-03-24 15:53:47.178 813f5d200 Connect: Closing client socket 23. Keep-alive: 0. Socket alive: 1. Data available: 1. Configuration file change detected: 0. Requests received: 1.
2021-03-24 15:53:47.178 813f5d200 Connect: Failed to shutdown client connection on socket 23. Attempts so far: 1, ret: 2
[fk@privoxy-vm ~]$ grep -c 'Gave up waiting for TLS data' /var/log/privoxy/privoxy-3.0.33.20210324_4-wolfssl-tls-callback-timeout-10-c1000.log
994

Privoxy CPU time and resident memory use according to top after the test

TLS libraryCPU timeMemory size
wolfSSL 4.7.0_7 (shared server WOLFSSL_CTX, shutdown-tls-connections 1, tls-receive-callback-timeout 10)5:58556M

Privoxy 3.0.33 experimental 20210324_5 82ffe8fb7601 tested with siege 4.0.7_4

The Privoxy ports have been updated. The privoxy-experimental-wolfssl port is no longer build with the DEBUG option enabled and the receive callback patch has been slightly refined.

The wolfssl port has been recompiled with the DEBUG option disabled as well.

Connection: close, concurrency level 1000

privoxy-experimental-wolfssl 3.0.33.20210324_5 with wolfSSL 4.7.0_8 (shared server WOLFSSL_CTX)

Privoxy has been configured with:

reuse-server-tls-context 1
shutdown-tls-connections 1
tls-receive-callback-timeout 20
[fk@benchmark-vm ~]$ siege --concurrent=1000 --reps=10 --benchmark "https://www.electrobsd.org/"
** SIEGE 4.0.7
** Preparing 1000 concurrent users for battle.
The server is now under siege...
Transactions:                   9700 hits
Availability:                  97.00 %
Elapsed time:                 675.19 secs
Data transferred:               7.23 MB
Response time:                 40.55 secs
Transaction rate:              14.37 trans/sec
Throughput:                     0.01 MB/sec
Concurrency:                  582.57
Successful transactions:        8224
Failed transactions:             300
Longest transaction:          149.05
Shortest transaction:           0.06
[fk@privoxy-vm ~]$ sort  /var/log/privoxy/privoxy-3.0.33.20210324_5_tls-timeout-10-1.log |  privoxy-log-parser.pl --detect-inactivity --inactivity-threshold 15000 |  grep -10 '^Detected inactivity'
2021-03-24 17:02:47.049 819cbba00 Header: scan: HTTP/1.1 200 OK
2021-03-24 17:02:47.049 819cbba00 Header: scan: Last-Modified: Sun, 23 Aug 2020 16:46:10 GMT
2021-03-24 17:02:47.049 819cbba00 Header: scan: Server: nginx
2021-03-24 17:02:47.049 819cbba00 Header: scan: Transfer-Encoding: chunked
2021-03-24 17:02:47.050 819cbba00 Connect: Closing client socket 5. Keep-alive: 0. Socket alive: 1. Data available: 0. Configuration file change detected: 0. Requests received: 1.
2021-03-24 17:02:47.050 819cbba00 Connect: Failed to shutdown client connection on socket 5. Attempts so far: 1, ret: 2
2021-03-24 17:02:51.656 81562cc00 Connect: Failed to shutdown server connection on socket 758. Attempts so far: 5, ret: 2
2021-03-24 17:02:51.656 81562cc00 Error: Gave up waiting for TLS data on socket 758. Timeout: 20 seconds.
2021-03-24 17:02:53.656 8061aeb00 Connect: Failed to shutdown server connection on socket 529. Attempts so far: 7, ret: 2
2021-03-24 17:02:53.656 8061aeb00 Error: Gave up waiting for TLS data on socket 529. Timeout: 20 seconds.
Detected inactivity: 18012 msecs
2021-03-24 17:03:11.668 81562cc00 Error: Gave up waiting for TLS data on socket 758. Timeout: 20 seconds.
2021-03-24 17:03:11.669 81562cc00 Connect: Failed to shutdown server connection on socket 758. Attempts so far: 6, ret: 2
2021-03-24 17:03:13.758 8061aeb00 Connect: Failed to shutdown server connection on socket 529. Attempts so far: 8, ret: 2
2021-03-24 17:03:13.758 8061aeb00 Error: Gave up waiting for TLS data on socket 529. Timeout: 20 seconds.
Detected inactivity: 17947 msecs
2021-03-24 17:03:31.705 81562cc00 Error: Gave up waiting for TLS data on socket 758. Timeout: 20 seconds.
2021-03-24 17:03:31.706 81562cc00 Connect: Failed to shutdown server connection on socket 758. Attempts so far: 7, ret: 2
2021-03-24 17:03:33.855 8061aeb00 Connect: Failed to shutdown server connection on socket 529. Attempts so far: 9, ret: 2
2021-03-24 17:03:33.855 8061aeb00 Error: Gave up waiting for TLS data on socket 529. Timeout: 20 seconds.
Detected inactivity: 17871 msecs
2021-03-24 17:03:51.726 81562cc00 Connect: Failed to shutdown server connection on socket 758. Attempts so far: 8, ret: 2
2021-03-24 17:03:51.726 81562cc00 Error: Gave up waiting for TLS data on socket 758. Timeout: 20 seconds.
2021-03-24 17:03:53.877 8061aeb00 Connect: Closing client socket 355. Keep-alive: 0. Socket alive: 1. Data available: 1. Configuration file change detected: 0. Requests received: 1.
2021-03-24 17:03:53.877 8061aeb00 Connect: Failed to shutdown server connection on socket 529. Attempts so far: 10, ret: 2
2021-03-24 17:03:53.877 8061aeb00 Error: Failed to shutdown server connection on socket 529 after 10 attempts. ret: 2, error: 0, unknown error number
2021-03-24 17:03:53.877 8061aeb00 Error: Gave up waiting for TLS data on socket 529. Timeout: 20 seconds.
2021-03-24 17:03:53.878 8061aeb00 Connect: Failed to shutdown client connection on socket 355. Attempts so far: 1, ret: 2
2021-03-24 17:03:53.880 802416000 Connect: Waiting for the next client connection. Currently active threads: 2
2021-03-24 17:03:53.880 80a6a2600 Connect: Accepted connection from 172.16.1.6 on socket 5
2021-03-24 17:03:53.880 80a6a2600 Header: addh-unique: Host: www.electrobsd.org:443
--
2021-03-24 17:03:53.974 80a6a2600 Connect: Looks like we got the last chunk together with the server headers. We better stop reading.
2021-03-24 17:03:53.974 80a6a2600 Header: scan: Connection: close
2021-03-24 17:03:53.974 80a6a2600 Header: scan: Content-Encoding: gzip
2021-03-24 17:03:53.974 80a6a2600 Header: scan: Content-Type: text/html
2021-03-24 17:03:53.974 80a6a2600 Header: scan: Date: Wed, 24 Mar 2021 16:11:53 GMT
2021-03-24 17:03:53.974 80a6a2600 Header: scan: ETag: W/"5f429d52-1fe"
2021-03-24 17:03:53.974 80a6a2600 Header: scan: HTTP/1.1 200 OK
2021-03-24 17:03:53.974 80a6a2600 Header: scan: Last-Modified: Sun, 23 Aug 2020 16:46:10 GMT
2021-03-24 17:03:53.974 80a6a2600 Header: scan: Server: nginx
2021-03-24 17:03:53.974 80a6a2600 Header: scan: Transfer-Encoding: chunked
Detected inactivity: 17814 msecs
2021-03-24 17:04:11.788 81562cc00 Connect: Failed to shutdown server connection on socket 758. Attempts so far: 9, ret: 2
2021-03-24 17:04:11.788 81562cc00 Error: Gave up waiting for TLS data on socket 758. Timeout: 20 seconds.
Detected inactivity: 20033 msecs
2021-03-24 17:04:31.821 81562cc00 Error: Gave up waiting for TLS data on socket 758. Timeout: 20 seconds.
2021-03-24 17:04:31.822 81562cc00 Connect: Closing client socket 293. Keep-alive: 0. Socket alive: 1. Data available: 1. Configuration file change detected: 0. Requests received: 1.
2021-03-24 17:04:31.822 81562cc00 Connect: Failed to shutdown client connection on socket 293. Attempts so far: 1, ret: 2
2021-03-24 17:04:31.822 81562cc00 Connect: Failed to shutdown server connection on socket 758. Attempts so far: 10, ret: 2
2021-03-24 17:04:31.822 81562cc00 Error: Failed to shutdown server connection on socket 758 after 10 attempts. ret: 2, error: 0, unknown error number
[fk@privoxy-vm ~]$ grep 81562cc00  /var/log/privoxy/privoxy-3.0.33.20210324_5_tls-timeout-10-1.log |  privoxy-log-parser.pl --detect-inactivity --inactivity-threshold 15000 | grep -10 '^Detected'
2021-03-24 17:01:31.519 81562cc00 Header: scan: Content-Type: text/html
2021-03-24 17:01:31.519 81562cc00 Header: scan: Last-Modified: Sun, 23 Aug 2020 16:46:10 GMT
2021-03-24 17:01:31.519 81562cc00 Header: scan: Transfer-Encoding: chunked
2021-03-24 17:01:31.519 81562cc00 Header: scan: Connection: close
2021-03-24 17:01:31.519 81562cc00 Header: scan: ETag: W/"5f429d52-1fe"
2021-03-24 17:01:31.519 81562cc00 Header: scan: Content-Encoding: gzip
2021-03-24 17:01:31.519 81562cc00 Connect: Looks like we got the last chunk together with the server headers. We better stop reading.
2021-03-24 17:01:31.519 81562cc00 Connect: Done reading from server. Content length: 352 as expected. Bytes most recently read: 352.
2021-03-24 17:01:31.519 81562cc00 Connect: Closing server socket 758 connected to www.electrobsd.org. Keep-alive: 0. Tainted: 0. Socket alive: 1. Timeout: 5.
2021-03-24 17:01:31.519 81562cc00 Connect: Failed to shutdown server connection on socket 758. Attempts so far: 1, ret: 2
Detected inactivity: 20017 msecs
2021-03-24 17:01:51.536 81562cc00 Error: Gave up waiting for TLS data on socket 758. Timeout: 20 seconds.
2021-03-24 17:01:51.536 81562cc00 Connect: Failed to shutdown server connection on socket 758. Attempts so far: 2, ret: 2
Detected inactivity: 20020 msecs
2021-03-24 17:02:11.556 81562cc00 Error: Gave up waiting for TLS data on socket 758. Timeout: 20 seconds.
2021-03-24 17:02:11.556 81562cc00 Connect: Failed to shutdown server connection on socket 758. Attempts so far: 3, ret: 2
Detected inactivity: 20080 msecs
2021-03-24 17:02:31.636 81562cc00 Error: Gave up waiting for TLS data on socket 758. Timeout: 20 seconds.
2021-03-24 17:02:31.636 81562cc00 Connect: Failed to shutdown server connection on socket 758. Attempts so far: 4, ret: 2
Detected inactivity: 20020 msecs
2021-03-24 17:02:51.656 81562cc00 Error: Gave up waiting for TLS data on socket 758. Timeout: 20 seconds.
2021-03-24 17:02:51.656 81562cc00 Connect: Failed to shutdown server connection on socket 758. Attempts so far: 5, ret: 2
Detected inactivity: 20012 msecs
2021-03-24 17:03:11.668 81562cc00 Error: Gave up waiting for TLS data on socket 758. Timeout: 20 seconds.
2021-03-24 17:03:11.669 81562cc00 Connect: Failed to shutdown server connection on socket 758. Attempts so far: 6, ret: 2
Detected inactivity: 20036 msecs
2021-03-24 17:03:31.705 81562cc00 Error: Gave up waiting for TLS data on socket 758. Timeout: 20 seconds.
2021-03-24 17:03:31.706 81562cc00 Connect: Failed to shutdown server connection on socket 758. Attempts so far: 7, ret: 2
Detected inactivity: 20020 msecs
2021-03-24 17:03:51.726 81562cc00 Error: Gave up waiting for TLS data on socket 758. Timeout: 20 seconds.
2021-03-24 17:03:51.726 81562cc00 Connect: Failed to shutdown server connection on socket 758. Attempts so far: 8, ret: 2
Detected inactivity: 20062 msecs
2021-03-24 17:04:11.788 81562cc00 Error: Gave up waiting for TLS data on socket 758. Timeout: 20 seconds.
2021-03-24 17:04:11.788 81562cc00 Connect: Failed to shutdown server connection on socket 758. Attempts so far: 9, ret: 2
Detected inactivity: 20033 msecs
2021-03-24 17:04:31.821 81562cc00 Error: Gave up waiting for TLS data on socket 758. Timeout: 20 seconds.
2021-03-24 17:04:31.822 81562cc00 Connect: Failed to shutdown server connection on socket 758. Attempts so far: 10, ret: 2
2021-03-24 17:04:31.822 81562cc00 Error: Failed to shutdown server connection on socket 758 after 10 attempts. ret: 2, error: 0, unknown error number
2021-03-24 17:04:31.822 81562cc00 Connect: Closing client socket 293. Keep-alive: 0. Socket alive: 1. Data available: 1. Configuration file change detected: 0. Requests received: 1.
2021-03-24 17:04:31.822 81562cc00 Connect: Failed to shutdown client connection on socket 293. Attempts so far: 1, ret: 2

Looks like the callback is getting called again even if it already returned a timeout for the socket.

I've changed shutdown-tls-connections to 0 and repeated the test.

[fk@benchmark-vm ~]$ siege --concurrent=1000 --reps=10 --benchmark "https://www.electrobsd.org/"
** SIEGE 4.0.7
** Preparing 1000 concurrent users for battle.
The server is now under siege...
Transactions:                   9991 hits
Availability:                  99.91 %
Elapsed time:                 512.08 secs
Data transferred:               3.81 MB
Response time:                 39.53 secs
Transaction rate:              19.51 trans/sec
Throughput:                     0.01 MB/sec
Concurrency:                  771.23
Successful transactions:        8116
Failed transactions:               9
Longest transaction:          126.15
Shortest transaction:           0.06

Looks like the Availability went up quite a bit. More requests were served successfully in less time.

[fk@privoxy-vm ~]$ sort  /var/log/privoxy/privoxy-3.0.33.20210324_5_tls-timeout-10-no-shutdown-2.log |  privoxy-log-parser.pl --detect-inactivity --inactivity-threshold 10000 |  grep -10 '^Detected inactivity'
2021-03-24 17:13:31.059 803037700 Connect: Server successfully connected over TLSv1.2 (ECDHE-RSA-AES128-GCM-SHA256).
2021-03-24 17:13:31.059 803219500 Connect: Server successfully connected over TLSv1.2 (ECDHE-RSA-AES128-GCM-SHA256).
2021-03-24 17:13:31.059 80331b400 Connect: Server successfully connected over TLSv1.2 (ECDHE-RSA-AES128-GCM-SHA256).
2021-03-24 17:13:31.059 8037c4b00 Connect: Server successfully connected over TLSv1.2 (ECDHE-RSA-AES128-GCM-SHA256).
2021-03-24 17:13:31.059 80872f100 Connect: Server successfully connected over TLSv1.2 (ECDHE-RSA-AES128-GCM-SHA256).
2021-03-24 17:13:31.059 80b8f4700 Connect: Server successfully connected over TLSv1.2 (ECDHE-RSA-AES128-GCM-SHA256).
2021-03-24 17:13:31.788 803b51b00 Header: scan: Transfer-Encoding: chunked
2021-03-24 17:13:33.218 803b51b00 Header: scan: Connection: close
2021-03-24 17:13:34.731 803b51b00 Header: scan: ETag: W/"5f429d52-1fe"
2021-03-24 17:13:37.068 803b51b00 Header: scan: Content-Encoding: gzip
Detected inactivity: 10482 msecs
2021-03-24 17:13:47.550 803b51b00 Connect: Looks like we got the last chunk together with the server headers. We better stop reading.
2021-03-24 17:13:56.409 803b51b00 Connect: Done reading from server. Content length: 352 as expected. Bytes most recently read: 352.
2021-03-24 17:13:56.409 804388700 Header: scan: Server: nginx
2021-03-24 17:13:56.826 804388700 Header: scan: Date: Wed, 24 Mar 2021 16:21:25 GMT
2021-03-24 17:13:56.961 804388700 Connect: Closing server socket 113 connected to www.electrobsd.org. Keep-alive: 0. Tainted: 0. Socket alive: 1. Timeout: 5.
2021-03-24 17:13:56.961 804388700 Connect: Done reading from server. Content length: 352 as expected. Bytes most recently read: 352.
2021-03-24 17:13:56.961 804388700 Connect: Looks like we got the last chunk together with the server headers. We better stop reading.
2021-03-24 17:13:56.961 804388700 Header: scan: Connection: close
2021-03-24 17:13:56.961 804388700 Header: scan: Content-Encoding: gzip
2021-03-24 17:13:56.961 804388700 Header: scan: Content-Type: text/html

I've changed shutdown-tls-connections back to 1 and reduced tls-receive-callback-timeout to 5.

[fk@benchmark-vm ~]$ siege --concurrent=1000 --reps=10 --benchmark "https://www.electrobsd.org/"
** SIEGE 4.0.7
** Preparing 1000 concurrent users for battle.
The server is now under siege...[alert] socket: select and discovered it's not ready sock.c:384: Operation timed out
[alert] socket: read check timed out(30) sock.c:273: Operation timed out

Transactions:                   9986 hits
Availability:                  99.86 %
Elapsed time:                 215.26 secs
Data transferred:               3.64 MB
Response time:                 14.02 secs
Transaction rate:              46.39 trans/sec
Throughput:                     0.02 MB/sec
Concurrency:                  650.44
Successful transactions:        1354
Failed transactions:              14
Longest transaction:           65.70
Shortest transaction:           0.06
[fk@privoxy-vm ~]$ grep -c 'Gave up waiting for TLS data' /var/log/privoxy/privoxy-3.0.33.20210324_5_tls-timeout-5-shutdown-1.log
8632
[fk@privoxy-vm ~]$ sort  /var/log/privoxy/privoxy-3.0.33.20210324_5_tls-timeout-5-shutdown-1.log |  privoxy-log-parser.pl --detect-inactivity --inactivity-threshold 3000 |  grep -5 '^Detected inactivity'
2021-03-24 17:57:44.446 806ce9d00 Connect: Server successfully connected over TLSv1.2 (ECDHE-RSA-AES128-GCM-SHA256).
2021-03-24 17:57:44.447 80411f900 Connect: Server successfully connected over TLSv1.2 (ECDHE-RSA-AES128-GCM-SHA256).
2021-03-24 17:57:44.448 804057d00 Connect: Server successfully connected over TLSv1.2 (ECDHE-RSA-AES128-GCM-SHA256).
2021-03-24 17:57:44.521 804058700 Connect: Server successfully connected over TLSv1.2 (ECDHE-RSA-AES128-GCM-SHA256).
2021-03-24 17:57:45.017 8025e5600 Connect: Server successfully connected over TLSv1.2 (ECDHE-RSA-AES128-GCM-SHA256).
Detected inactivity: 4944 msecs
2021-03-24 17:57:49.961 808991b00 Connect: Server successfully connected over TLSv1.2 (ECDHE-RSA-AES128-GCM-SHA256).
2021-03-24 17:57:49.962 802975000 Connect: Server successfully connected over TLSv1.2 (ECDHE-RSA-AES128-GCM-SHA256).
2021-03-24 17:57:49.962 804120300 Connect: Server successfully connected over TLSv1.2 (ECDHE-RSA-AES128-GCM-SHA256).
2021-03-24 17:57:49.962 806899b00 Connect: Server successfully connected over TLSv1.2 (ECDHE-RSA-AES128-GCM-SHA256).
2021-03-24 17:57:49.963 802667700 Connect: Server successfully connected over TLSv1.2 (ECDHE-RSA-AES128-GCM-SHA256).
--
2021-03-24 17:57:58.667 818860f00 Header: scan: Last-Modified: Sun, 23 Aug 2020 16:46:10 GMT
2021-03-24 17:57:58.667 818860f00 Header: scan: Server: nginx
2021-03-24 17:57:58.667 818860f00 Header: scan: Transfer-Encoding: chunked
2021-03-24 17:57:58.668 818860f00 Connect: Failed to shutdown server connection on socket 1062. Attempts so far: 1, ret: 2
2021-03-24 17:57:59.228 807c31400 Error: X509 certificate verification for www.electrobsd.org failed with error -308: error state on socket
Detected inactivity: 3082 msecs
2021-03-24 17:58:02.310 8063d5c00 Error: X509 certificate verification for www.electrobsd.org failed with error -308: error state on socket
2021-03-24 17:58:02.402 806c76900 Error: X509 certificate verification for www.electrobsd.org failed with error -308: error state on socket
2021-03-24 17:58:02.958 806c75f00 Error: X509 certificate verification for www.electrobsd.org failed with error -308: error state on socket
2021-03-24 17:58:04.068 807c33700 Error: X509 certificate verification for www.electrobsd.org failed with error -308: error state on socket
2021-03-24 17:58:06.788 8089a7000 Error: X509 certificate verification for www.electrobsd.org failed with error -308: error state on socket
--
2021-03-24 18:00:19.967 80c0c9c00 Header: scan: Server: nginx
2021-03-24 18:00:19.967 80c0c9c00 Header: scan: Transfer-Encoding: chunked
2021-03-24 18:00:19.968 80c0c9c00 Connect: Closing client socket 5. Keep-alive: 0. Socket alive: 1. Data available: 0. Configuration file change detected: 0. Requests received: 1.
2021-03-24 18:00:19.968 80c0c9c00 Connect: Failed to shutdown client connection on socket 5. Attempts so far: 1, ret: 2
2021-03-24 18:00:19.968 80c0c9c00 Connect: Failed to shutdown server connection on socket 6. Attempts so far: 1, ret: 2
Detected inactivity: 8027 msecs
2021-03-24 18:00:27.995 80c456800 Connect: Could not connect to [www.electrobsd.org]:443: Operation timed out.
2021-03-24 18:00:27.996 80c456800 Connect: Failed to shutdown client connection on socket 672. Attempts so far: 1, ret: 2
2021-03-24 18:00:27.996 80c456800 Crunch: Connection failure: https://www.electrobsd.org/
2021-03-24 18:00:27.999 802416000 Connect: Waiting for the next client connection. Currently active threads: 7
2021-03-24 18:00:27.999 811b8f400 Connect: Accepted connection from 172.16.1.6 on socket 5

Most of the requests failed.

[fk@privoxy-vm ~]$ privoxy-log-parser.pl --statistics  /var/log/privoxy/privoxy-3.0.33.20210324_5_tls-timeout-5-shutdown-1.log    
Client requests total: 9999
Crunches: 8645 (86.46%)
Blocks: 0 (0.00%)
Fast redirections: 0 (0.00%)
Connection timeouts: 0 (0.00%)
Connection failures: 13 (0.13%)
Outgoing requests: 1354 (13.54%)
Server keep-alive offers: 0 (0.00%)
New outgoing connections: 1354 (13.54%)
Reused server connections: 0 (0.00%; server offers accepted: 0.00%)
Empty responses: 0 (0.00%)
Empty responses on new connections: 0 (0.00%)
Empty responses on reused connections: 0 (0.00%)
Client connections: 9986
Bytes of content transferred to the client: 4078522
Client requests per connection distribution:
    9986: 1
Enable --show-complete-request-distribution to get less common numbers as well.
Improperly accounted requests: ~13
Method distribution:
    9999 : GET     
Client HTTP versions:
    9999 : HTTP/1.1
HTTP status codes according to 'debug 512' (status codes sent by the server may differ):
    8632 : 403     
    1354 : 200     
      13 : 503     
URL statistics are disabled. Increase --url-statistics-threshold to enable them.
Passed request statistics are disabled. Increase --passed-request-statistics-threshold to enable them.
Host statistics are disabled. Increase --host-statistics-threshold to enable them.

privoxy-experimental-wolfssl 3.0.33.20210324_5 with wolfSSL 4.7.0_8 (WOLFSSL_CTX not shared)

Privoxy has been configured with:

reuse-server-tls-context 0
shutdown-tls-connections 1
tls-receive-callback-timeout 20
[fk@benchmark-vm ~]$ siege --concurrent=1000 --reps=10 --benchmark "https://www.electrobsd.org/"
** SIEGE 4.0.7
** Preparing 1000 concurrent users for battle.
The server is now under siege...
Transactions:                   9986 hits
Availability:                  99.86 %
Elapsed time:                 446.16 secs
Data transferred:               3.33 MB
Response time:                 33.71 secs
Transaction rate:              22.38 trans/sec
Throughput:                     0.01 MB/sec
Concurrency:                  754.53
Successful transactions:        9889
Failed transactions:              14
Longest transaction:           76.36
Shortest transaction:           0.20
[fk@privoxy-vm ~]$ sort  /var/log/privoxy/privoxy-3.0.33.20210324_5_tls-timeout-20-no-server-ctx-sharing.log |  privoxy-log-parser.pl --detect-inactivity --inactivity-threshold 5000 |  grep -c '^Detected inactivity'
0

Privoxy CPU time and resident memory use according to top after the test

TLS libraryCPU timeMemory size
wolfSSL 4.7.0_8 (shared server WOLFSSL_CTX, shutdown-tls-connections 1, tls-receive-callback-timeout 20)7:14625M
wolfSSL 4.7.0_8 (shared server WOLFSSL_CTX, shutdown-tls-connections 0, tls-receive-callback-timeout 20)7:12???
wolfSSL 4.7.0_8 (shared server WOLFSSL_CTX, shutdown-tls-connections 1, tls-receive-callback-timeout 5)2:35441M
wolfSSL 4.7.0_8 (server WOLFSSL_CTX not shared, shutdown-tls-connections 1, tls-receive-callback-timeout 20)6:42979M

As it turned out there was a memory leak in case of "reuse-server-tls-context 0" ...

Privoxy 3.0.33 experimental 20210325 ecb3f5aa1 tested with siege 4.0.7_4

The Privoxy ports have been updated. The wolfssl-specific code has been changed to fix the memory leak in case of "reuse-server-tls-context 0" and the callback now logs a message after each second spent waiting for data (to rule out a thread scheduling issue).

Connection: close, concurrency level 1000

privoxy-experimental-wolfssl 3.0.33.20210324_5 with wolfSSL 4.7.0_9 (shared server WOLFSSL_CTX)

Privoxy has been configured with:

reuse-server-tls-context 1
shutdown-tls-connections 1
tls-receive-callback-timeout 20
[fk@benchmark-vm ~]$ siege --concurrent=1000 --reps=10 --benchmark "https://www.electrobsd.org/"
** SIEGE 4.0.7
** Preparing 1000 concurrent users for battle.
The server is now under siege...
Transactions:                   9905 hits
Availability:                  99.05 %
Elapsed time:                 489.04 secs
Data transferred:               3.86 MB
Response time:                 33.55 secs
Transaction rate:              20.25 trans/sec
Throughput:                     0.01 MB/sec
Concurrency:                  679.60
Successful transactions:        9069
Failed transactions:              95
Longest transaction:          113.94
Shortest transaction:           0.08
[fk@privoxy-vm ~]$ sort /var/log/privoxy/privoxy-3.0.33.20210325_tls--no-server-ctx-sharing.log | privoxy-log-parser.pl --detect-inactivity --inactivity-threshold 5000 | grep '^Detected'
Detected inactivity: 5499 msecs
Detected inactivity: 5756 msecs
Detected inactivity: 8873 msecs
Detected inactivity: 5110 msecs
Detected inactivity: 13424 msecs
Detected inactivity: 7807 msecs
[fk@privoxy-vm ~]$ grep -c "Gave up" /var/log/privoxy/privoxy-3.0.33.20210325_tls--no-server-ctx-sharing.log
837

Looking at one of those failures:

[fk@privoxy-vm ~]$ grep 825b70600 /var/log/privoxy/privoxy-3.0.33.20210325_tls--no-server-ctx-sharing.log | grep -30 'Error: Gave up waiting for TLS'
2021-03-25 04:37:21.230 825b70600 Header: scan: Accept-Encoding: gzip, deflate
2021-03-25 04:37:21.230 825b70600 Header: scan: User-Agent: Mozilla/5.0 (portbld-amd64-freebsd11.4) Siege/4.0.7
2021-03-25 04:37:21.230 825b70600 Header: scan: Connection: close
2021-03-25 04:37:21.230 825b70600 Header: Keeping the client header 'Connection: close' around. The connection will not be kept alive.
2021-03-25 04:37:21.230 825b70600 Header: Encrypted request processed
2021-03-25 04:37:21.230 825b70600 Request: https://www.electrobsd.org/
2021-03-25 04:37:21.230 825b70600 Header: New HTTP Request-Line: GET / HTTP/1.1
2021-03-25 04:37:21.230 825b70600 Connect: to www.electrobsd.org
2021-03-25 04:37:21.241 825b70600 Connect: Connected to www.electrobsd.org[95.211.138.51]:443.
2021-03-25 04:37:21.241 825b70600 Connect: Created new connection to www.electrobsd.org:443 on socket 1308.
2021-03-25 04:37:22.591 825b70600 Connect: Performing the TLS/SSL handshake with the server
2021-03-25 04:37:23.642 825b70600 Connect: Continuing to wait for TLS data on socket 1308. Time spent waiting so far: 1 seconds.
2021-03-25 04:37:24.695 825b70600 Connect: Continuing to wait for TLS data on socket 1308. Time spent waiting so far: 2 seconds.
2021-03-25 04:37:25.729 825b70600 Connect: Continuing to wait for TLS data on socket 1308. Time spent waiting so far: 3 seconds.
2021-03-25 04:38:30.739 825b70600 Connect: Continuing to wait for TLS data on socket 1308. Time spent waiting so far: 4 seconds.
2021-03-25 04:38:31.759 825b70600 Connect: Continuing to wait for TLS data on socket 1308. Time spent waiting so far: 5 seconds.
2021-03-25 04:38:32.888 825b70600 Connect: Continuing to wait for TLS data on socket 1308. Time spent waiting so far: 6 seconds.
2021-03-25 04:38:33.898 825b70600 Connect: Continuing to wait for TLS data on socket 1308. Time spent waiting so far: 7 seconds.
2021-03-25 04:38:35.344 825b70600 Connect: Continuing to wait for TLS data on socket 1308. Time spent waiting so far: 8 seconds.
2021-03-25 04:38:36.389 825b70600 Connect: Continuing to wait for TLS data on socket 1308. Time spent waiting so far: 9 seconds.
2021-03-25 04:38:37.568 825b70600 Connect: Continuing to wait for TLS data on socket 1308. Time spent waiting so far: 10 seconds.
2021-03-25 04:38:38.585 825b70600 Connect: Continuing to wait for TLS data on socket 1308. Time spent waiting so far: 11 seconds.
2021-03-25 04:38:39.641 825b70600 Connect: Continuing to wait for TLS data on socket 1308. Time spent waiting so far: 12 seconds.
2021-03-25 04:38:40.650 825b70600 Connect: Continuing to wait for TLS data on socket 1308. Time spent waiting so far: 13 seconds.
2021-03-25 04:38:41.668 825b70600 Connect: Continuing to wait for TLS data on socket 1308. Time spent waiting so far: 14 seconds.
2021-03-25 04:38:42.678 825b70600 Connect: Continuing to wait for TLS data on socket 1308. Time spent waiting so far: 15 seconds.
2021-03-25 04:38:43.689 825b70600 Connect: Continuing to wait for TLS data on socket 1308. Time spent waiting so far: 16 seconds.
2021-03-25 04:38:44.698 825b70600 Connect: Continuing to wait for TLS data on socket 1308. Time spent waiting so far: 17 seconds.
2021-03-25 04:38:45.715 825b70600 Connect: Continuing to wait for TLS data on socket 1308. Time spent waiting so far: 18 seconds.
2021-03-25 04:38:46.782 825b70600 Connect: Continuing to wait for TLS data on socket 1308. Time spent waiting so far: 19 seconds.
2021-03-25 04:38:47.908 825b70600 Error: Gave up waiting for TLS data on socket 1308. Timeout: 20 seconds.
2021-03-25 04:38:48.839 825b70600 Error: X509 certificate verification for www.electrobsd.org failed with error -308: error state on socket
2021-03-25 04:38:48.840 825b70600 Crunch: Certificate error: error state on socket: https://www.electrobsd.org/
2021-03-25 04:38:48.840 825b70600 Connect: Failed to shutdown client connection on socket 1195. Attempts so far: 1, ret: 2
2021-03-25 04:38:49.440 825b70600 Connect: Closing server socket 1308 connected to www.electrobsd.org. Keep-alive: 0. Tainted: 1. Socket alive: 1. Timeout: 5.
2021-03-25 04:38:49.441 825b70600 Connect: Closing client socket 1195. Keep-alive: 0. Socket alive: 0. Data available: 0. Configuration file change detected: 0. Requests received: 1.

So the problem doesn't seem to be a scheduling issue.

privoxy-experimental-wolfssl 3.0.33.20210324_5 with wolfSSL 4.7.0_9 (server WOLFSSL_CTX not shared)

Privoxy has been configured with:

reuse-server-tls-context 0
shutdown-tls-connections 1
tls-receive-callback-timeout 20
[fk@benchmark-vm ~]$ siege --concurrent=1000 --reps=10 --benchmark "https://www.electrobsd.org/"
** SIEGE 4.0.7
** Preparing 1000 concurrent users for battle.
The server is now under siege...
Transactions:                   9969 hits
Availability:                  99.69 %
Elapsed time:                 321.38 secs
Data transferred:               3.43 MB
Response time:                 23.46 secs
Transaction rate:              31.02 trans/sec
Throughput:                     0.01 MB/sec
Concurrency:                  727.84
Successful transactions:        9656
Failed transactions:              31
Longest transaction:           73.66
Shortest transaction:           0.08
[fk@privoxy-vm ~]$ grep -c "Gave up" /var/log/privoxy/privoxy-3.0.33.20210325_tls-server-ctx-shared.log     
316

Apparently not sharing the server WOLFSSL_CTX results in better performance and requires less resident memory which seem non-intuitive.

Privoxy CPU time and resident memory use according to top after the test

TLS libraryCPU timeMemory size
wolfSSL 4.7.0_9 (reuse-server-tls-context 1)6:39588M
wolfSSL 4.7.0_9 (reuse-server-tls-context 0)4:33402M

Connection: close, concurrency level 10, requesting 1 MB of data

This test is fetching 1 MB of random data. I reduced the concurrency level to 10 and increased the reps to 100.

Note that the network connections of the server hosting www.electrobsd.org and the system hosting the VMs are fairly busy so Privoxy spends a fair amount of time waiting for data to arrive.

privoxy-experimental 3.0.33.20210325 with OpenSSL 1.0.2t-ElectroBSD from base

[fk@benchmark-vm ~]$ siege --concurrent=10 --reps=100 --benchmark "https://www.electrobsd.org/test/random-data-1M.raw"
** SIEGE 4.0.7
** Preparing 10 concurrent users for battle.
The server is now under siege...[alert] socket: select and discovered it's not ready sock.c:384: Operation timed out
[alert] socket: read check timed out(30) sock.c:273: Operation timed out

Transactions:                    999 hits
Availability:                  99.90 %
Elapsed time:                 217.02 secs
Data transferred:             999.00 MB
Response time:                  2.07 secs
Transaction rate:               4.60 trans/sec
Throughput:                     4.60 MB/sec
Concurrency:                    9.52
Successful transactions:         999
Failed transactions:               1
Longest transaction:           10.62
Shortest transaction:           0.33

privoxy-experimental-libressl 3.0.33.20210325 with LibreSSL 3.2.5

[fk@benchmark-vm ~]$ siege --concurrent=10 --reps=100 --benchmark "https://www.electrobsd.org/test/random-data-1M.raw"
** SIEGE 4.0.7
** Preparing 10 concurrent users for battle.
The server is now under siege...
Transactions:                    999 hits
Availability:                  99.90 %
Elapsed time:                 224.62 secs
Data transferred:             999.01 MB
Response time:                  2.13 secs
Transaction rate:               4.45 trans/sec
Throughput:                     4.45 MB/sec
Concurrency:                    9.49
Successful transactions:         999
Failed transactions:               1
Longest transaction:           30.06
Shortest transaction:           0.34

privoxy-experimental-libressl-devel 3.0.33.20210325 with LibreSSL devel 3.3.1

[fk@benchmark-vm ~]$ siege --concurrent=10 --reps=100 --benchmark "https://www.electrobsd.org/test/random-data-1M.raw"
** SIEGE 4.0.7
** Preparing 10 concurrent users for battle.
The server is now under siege...
Transactions:                   1000 hits
Availability:                 100.00 %
Elapsed time:                 195.53 secs
Data transferred:            1000.00 MB
Response time:                  1.91 secs
Transaction rate:               5.11 trans/sec
Throughput:                     5.11 MB/sec
Concurrency:                    9.77
Successful transactions:        1000
Failed transactions:               0
Longest transaction:            6.92
Shortest transaction:           0.35

privoxy-experimental-ports-openssl 3.0.33.20210325 with OpenSSL 1.1.1j,1

[fk@benchmark-vm ~]$ siege --concurrent=10 --reps=100 --benchmark "https://www.electrobsd.org/test/random-data-1M.raw"
** SIEGE 4.0.7
** Preparing 10 concurrent users for battle.
The server is now under siege...
Transactions:                   1000 hits
Availability:                 100.00 %
Elapsed time:                 215.38 secs
Data transferred:            1000.00 MB
Response time:                  2.11 secs
Transaction rate:               4.64 trans/sec
Throughput:                     4.64 MB/sec
Concurrency:                    9.78
Successful transactions:        1000
Failed transactions:               0
Longest transaction:            7.66
Shortest transaction:           0.40

privoxy-experimental-ports-openssl-devel 3.0.33.20210325 with OpenSSL devel 3.0.0.a13

[fk@benchmark-vm ~]$ siege --concurrent=10 --reps=100 --benchmark "https://www.electrobsd.org/test/random-data-1M.raw"
** SIEGE 4.0.7
** Preparing 10 concurrent users for battle.
The server is now under siege...
Transactions:                   1000 hits
Availability:                 100.00 %
Elapsed time:                 267.44 secs
Data transferred:            1000.00 MB
Response time:                  2.59 secs
Transaction rate:               3.74 trans/sec
Throughput:                     3.74 MB/sec
Concurrency:                    9.69
Successful transactions:        1000
Failed transactions:               0
Longest transaction:            9.52
Shortest transaction:           0.80

privoxy-experimental-wolfssl 3.0.33.20210325 with wolfSSL 4.7.0_9 (shared server WOLFSSL_CTX)

[fk@benchmark-vm ~]$ siege --concurrent=10 --reps=100 --benchmark "https://www.electrobsd.org/test/random-data-1M.raw"
** SIEGE 4.0.7
** Preparing 10 concurrent users for battle.
The server is now under siege...
Transactions:                    998 hits
Availability:                  99.80 %
Elapsed time:                 269.85 secs
Data transferred:             998.01 MB
Response time:                  2.52 secs
Transaction rate:               3.70 trans/sec
Throughput:                     3.70 MB/sec
Concurrency:                    9.31
Successful transactions:         998
Failed transactions:               2
Longest transaction:           30.09
Shortest transaction:           0.77

privoxy-experimental-wolfssl 3.0.33.20210325 with wolfSSL 4.7.0_9 (separate server WOLFSSL_CTX)

[fk@benchmark-vm ~]$ siege --concurrent=10 --reps=100 --benchmark "https://www.electrobsd.org/test/random-data-1M.raw"
** SIEGE 4.0.7
** Preparing 10 concurrent users for battle.
The server is now under siege...
Transactions:                   1000 hits
Availability:                 100.00 %
Elapsed time:                 255.66 secs
Data transferred:            1000.00 MB
Response time:                  2.49 secs
Transaction rate:               3.91 trans/sec
Throughput:                     3.91 MB/sec
Concurrency:                    9.76
Successful transactions:        1000
Failed transactions:               0
Longest transaction:           10.22
Shortest transaction:           0.72

privoxy-experimental-mbedtls 3.0.33.20210325 with mbedtls 2.16.10_1

[fk@benchmark-vm ~]$ siege --concurrent=10 --reps=100 --benchmark "https://www.electrobsd.org/test/random-data-1M.raw"
** SIEGE 4.0.7
** Preparing 10 concurrent users for battle.
The server is now under siege...
Transactions:                   1000 hits
Availability:                 100.00 %
Elapsed time:                 291.95 secs
Data transferred:            1000.00 MB
Response time:                  2.83 secs
Transaction rate:               3.43 trans/sec
Throughput:                     3.43 MB/sec
Concurrency:                    9.68
Successful transactions:        1000
Failed transactions:               0
Longest transaction:            7.88
Shortest transaction:           0.52

Privoxy CPU time and resident memory use according to top after the test

TLS libraryCPU timeMemory size
OpenSSL 1.0.2t1:4425968K
LibreSSL 3.2.51:5023220K
LibreSSL devel 3.3.11:5323796K
OpenSSL 1.1.1j,11:5125668K
OpenSSL devel 3.0.0.a132:0328532K
wolfSSL 4.7.0_9 (shared server WOLFSSL_CTX)1:5918808K
wolfSSL 4.7.0_9 (separate server WOLFSSL_CTX)2:0616724K
mbedtls 2.16.10_13:4515356K

Connection: keep-alive, concurrency level 10, requesting 1 MB of data

For this test I've enabled siege's keep-alive mode.

privoxy-experimental 3.0.33.20210325 with OpenSSL 1.0.2t-ElectroBSD from base

[fk@benchmark-vm ~]$ siege --concurrent=10 --reps=100 --keep-alive --benchmark "https://www.electrobsd.org/test/random-data-1M.raw"
** SIEGE 4.0.7
** Preparing 10 concurrent users for battle.
The server is now under siege...
Transactions:                   1000 hits
Availability:                 100.00 %
Elapsed time:                 233.56 secs
Data transferred:            1000.00 MB
Response time:                  2.27 secs
Transaction rate:               4.28 trans/sec
Throughput:                     4.28 MB/sec
Concurrency:                    9.73
Successful transactions:        1000
Failed transactions:               0
Longest transaction:            6.11
Shortest transaction:           0.23

privoxy-experimental-libressl 3.0.33.20210325 with LibreSSL 3.2.5

[fk@benchmark-vm ~]$ siege --concurrent=10 --reps=100 --keep-alive --benchmark "https://www.electrobsd.org/test/random-data-1M.raw"
** SIEGE 4.0.7
** Preparing 10 concurrent users for battle.
The server is now under siege...
Transactions:                   1000 hits
Availability:                 100.00 %
Elapsed time:                 220.82 secs
Data transferred:            1000.00 MB
Response time:                  2.13 secs
Transaction rate:               4.53 trans/sec
Throughput:                     4.53 MB/sec
Concurrency:                    9.65
Successful transactions:        1000
Failed transactions:               0
Longest transaction:            6.58
Shortest transaction:           0.18

privoxy-experimental-libressl-devel 3.0.33.20210325 with LibreSSL devel 3.3.1

[fk@benchmark-vm ~]$ siege --concurrent=10 --reps=100 --keep-alive --benchmark "https://www.electrobsd.org/test/random-data-1M.raw"
** SIEGE 4.0.7
** Preparing 10 concurrent users for battle.
The server is now under siege...
Transactions:                   1000 hits
Availability:                 100.00 %
Elapsed time:                 208.70 secs
Data transferred:            1000.00 MB
Response time:                  2.02 secs
Transaction rate:               4.79 trans/sec
Throughput:                     4.79 MB/sec
Concurrency:                    9.66
Successful transactions:        1000
Failed transactions:               0
Longest transaction:            6.71
Shortest transaction:           0.26

privoxy-experimental-ports-openssl 3.0.33.20210325 with OpenSSL 1.1.1j,1

[fk@benchmark-vm ~]$ siege --concurrent=10 --reps=100 --keep-alive --benchmark "https://www.electrobsd.org/test/random-data-1M.raw"
** SIEGE 4.0.7
** Preparing 10 concurrent users for battle.
The server is now under siege...
Transactions:                   1000 hits
Availability:                 100.00 %
Elapsed time:                 212.75 secs
Data transferred:            1000.00 MB
Response time:                  2.07 secs
Transaction rate:               4.70 trans/sec
Throughput:                     4.70 MB/sec
Concurrency:                    9.72
Successful transactions:        1000
Failed transactions:               0
Longest transaction:            7.44
Shortest transaction:           0.23

privoxy-experimental-ports-openssl-devel 3.0.33.20210325 with OpenSSL devel 3.0.0.a13

[fk@benchmark-vm ~]$ siege --concurrent=10 --reps=100 --keep-alive --benchmark "https://www.electrobsd.org/test/random-data-1M.raw"
** SIEGE 4.0.7
** Preparing 10 concurrent users for battle.
The server is now under siege...
Transactions:                   1000 hits
Availability:                 100.00 %
Elapsed time:                 220.55 secs
Data transferred:            1000.00 MB
Response time:                  2.14 secs
Transaction rate:               4.53 trans/sec
Throughput:                     4.53 MB/sec
Concurrency:                    9.70
Successful transactions:        1000
Failed transactions:               0
Longest transaction:            6.91
Shortest transaction:           0.34

privoxy-experimental-wolfssl 3.0.33.20210325 with wolfSSL 4.7.0_9 (shared server WOLFSSL_CTX)

[fk@benchmark-vm ~]$ siege --concurrent=10 --reps=100 --keep-alive --benchmark "https://www.electrobsd.org/test/random-data-1M.raw"
** SIEGE 4.0.7
** Preparing 10 concurrent users for battle.
The server is now under siege...
Transactions:                   1000 hits
Availability:                 100.00 %
Elapsed time:                 213.53 secs
Data transferred:            1000.00 MB
Response time:                  2.07 secs
Transaction rate:               4.68 trans/sec
Throughput:                     4.68 MB/sec
Concurrency:                    9.69
Successful transactions:        1000
Failed transactions:               0
Longest transaction:            6.02
Shortest transaction:           0.32

privoxy-experimental-wolfssl 3.0.33.20210325 with wolfSSL 4.7.0_9 (separate server WOLFSSL_CTX)

[fk@benchmark-vm ~]$ siege --concurrent=10 --reps=100 --keep-alive --benchmark "https://www.electrobsd.org/test/random-data-1M.raw"
** SIEGE 4.0.7
** Preparing 10 concurrent users for battle.
The server is now under siege...
Transactions:                   1000 hits
Availability:                 100.00 %
Elapsed time:                 264.38 secs
Data transferred:            1000.00 MB
Response time:                  2.59 secs
Transaction rate:               3.78 trans/sec
Throughput:                     3.78 MB/sec
Concurrency:                    9.80
Successful transactions:        1000
Failed transactions:               0
Longest transaction:            7.64
Shortest transaction:           0.37

privoxy-experimental-wolfssl 3.0.33.20210325 with wolfSSL 4.7.0_9 (separate server WOLFSSL_CTX, default callback)

For this test Privoxy is not overwriting wolfSSL's default receive callback.

[fk@benchmark-vm ~]$ siege --concurrent=10 --reps=100 --keep-alive --benchmark "https://www.electrobsd.org/test/random-data-1M.raw"
** SIEGE 4.0.7
** Preparing 10 concurrent users for battle.
The server is now under siege...
Transactions:                   1000 hits
Availability:                 100.00 %
Elapsed time:                 319.03 secs
Data transferred:            1000.00 MB
Response time:                  3.14 secs
Transaction rate:               3.13 trans/sec
Throughput:                     3.13 MB/sec
Concurrency:                    9.83
Successful transactions:        1000
Failed transactions:               0
Longest transaction:           12.37
Shortest transaction:           0.36

privoxy-experimental-mbedtls 3.0.33.20210325 with mbedtls 2.16.10_1

[fk@benchmark-vm ~]$ siege --concurrent=10 --reps=100 --keep-alive --benchmark "https://www.electrobsd.org/test/random-data-1M.raw"
** SIEGE 4.0.7
** Preparing 10 concurrent users for battle.
The server is now under siege...
Transactions:                   1000 hits
Availability:                 100.00 %
Elapsed time:                 265.41 secs
Data transferred:            1000.00 MB
Response time:                  2.56 secs
Transaction rate:               3.77 trans/sec
Throughput:                     3.77 MB/sec
Concurrency:                    9.66
Successful transactions:        1000
Failed transactions:               0
Longest transaction:            9.44
Shortest transaction:           0.95

Privoxy CPU time and resident memory use according to top after the test

TLS libraryCPU timeMemory size
OpenSSL 1.0.2t1:1820108K
LibreSSL 3.2.51:1820104K
LibreSSL devel 3.3.11:1920432K
OpenSSL 1.1.1j,11:2420464K
OpenSSL devel 3.0.0.a131:2022856K
wolfSSL 4.7.0_9 (shared server WOLFSSL_CTX)2:3823860K
wolfSSL 4.7.0_9 (separate server WOLFSSL_CTX)1:4414328K
wolfSSL 4.7.0_9 (separate server WOLFSSL_CTX, default callback)1:3913144K
mbedtls 2.16.10_11:5214204K

2021-12-21: Privoxy 3.0.34 experimental 2021-12-21 3f1318665e tested with siege

The operating system on the host and in the virtual machines has been changed to ElectroBSD 12.3-STABLE 5c6e955abbf5.

The Privoxy ports have been updated and are close to the 3.0.33 release.

The webserver that serves https://www.electrobsd.org/ has been updated as well and now supports TLS 1.3.

Connection: close, concurrency level 10, requesting 1 MB of data

This test is fetching 1 MB of random data with a concurrency level of 10 and 100 reps.

Keep in mind that the network connections of the server hosting www.electrobsd.org and the system hosting the VMs are fairly busy so Privoxy spends a fair amount of time waiting for data to arrive.

privoxy-experimental 3.0.34.20211221 with OpenSSL 1.1.1m-ElectroBSD from base

[fk@benchmark-vm ~]$ siege --concurrent=10 --reps=100 --benchmark "https://www.electrobsd.org/test/random-data-1M.raw"
** SIEGE 4.1.1
** Preparing 10 concurrent users for battle.
The server is now under siege...
Transactions:                   1000 hits
Availability:                 100.00 %
Elapsed time:                 204.61 secs
Data transferred:            1000.00 MB
Response time:                  1.99 secs
Transaction rate:               4.89 trans/sec
Throughput:                     4.89 MB/sec
Concurrency:                    9.74
Successful transactions:        1000
Failed transactions:               0
Longest transaction:            7.52
Shortest transaction:           0.52

privoxy-experimental 3.0.34.20211221 with LibreSSL 3.3.5

[fk@benchmark-vm ~]$ siege --concurrent=10 --reps=100 --benchmark "https://www.electrobsd.org/test/random-data-1M.raw"
** SIEGE 4.1.1
** Preparing 10 concurrent users for battle.
The server is now under siege...
Transactions:                   1000 hits
Availability:                 100.00 %
Elapsed time:                 204.48 secs
Data transferred:            1000.00 MB
Response time:                  1.99 secs
Transaction rate:               4.89 trans/sec
Throughput:                     4.89 MB/sec
Concurrency:                    9.75
Successful transactions:        1000
Failed transactions:               0
Longest transaction:            7.91
Shortest transaction:           0.31

privoxy-experimental 3.0.34.20211221 with LibreSSL devel 3.4.0

[fk@benchmark-vm ~]$ siege --concurrent=10 --reps=100 --benchmark "https://www.electrobsd.org/test/random-data-1M.raw"
** SIEGE 4.1.1
** Preparing 10 concurrent users for battle.
The server is now under siege...
Transactions:                   1000 hits
Availability:                 100.00 %
Elapsed time:                 259.81 secs
Data transferred:            1000.00 MB
Response time:                  2.54 secs
Transaction rate:               3.85 trans/sec
Throughput:                     3.85 MB/sec
Concurrency:                    9.79
Successful transactions:        1000
Failed transactions:               0
Longest transaction:            9.87
Shortest transaction:           0.62

privoxy-experimental 3.0.34.20211221 with OpenSSL 1.1.1l

[fk@benchmark-vm ~]$ siege --concurrent=10 --reps=100 --benchmark "https://www.electrobsd.org/test/random-data-1M.raw"
** SIEGE 4.1.1
** Preparing 10 concurrent users for battle.
The server is now under siege...
Transactions:                   1000 hits
Availability:                 100.00 %
Elapsed time:                 222.54 secs
Data transferred:            1000.00 MB
Response time:                  2.18 secs
Transaction rate:               4.49 trans/sec
Throughput:                     4.49 MB/sec
Concurrency:                    9.81
Successful transactions:        1000
Failed transactions:               0
Longest transaction:            7.68
Shortest transaction:           0.35

privoxy-experimental 3.0.34.20211221 with OpenSSL devel 3.0.1

[fk@benchmark-vm ~]$ siege --concurrent=10 --reps=100 --benchmark "https://www.electrobsd.org/test/random-data-1M.raw"
** SIEGE 4.1.1
** Preparing 10 concurrent users for battle.
The server is now under siege...
Transactions:                   1000 hits
Availability:                 100.00 %
Elapsed time:                 311.33 secs
Data transferred:            1000.00 MB
Response time:                  3.05 secs
Transaction rate:               3.21 trans/sec
Throughput:                     3.21 MB/sec
Concurrency:                    9.79
Successful transactions:        1000
Failed transactions:               0
Longest transaction:            8.90
Shortest transaction:           0.38

privoxy-experimental 3.0.34.20211221 with wolfSSL 5.0.0

[fk@benchmark-vm ~]$ siege --concurrent=10 --reps=100 --benchmark "https://www.electrobsd.org/test/random-data-1M.raw"
** SIEGE 4.1.1
** Preparing 10 concurrent users for battle.
The server is now under siege...
Transactions:                   1000 hits
Availability:                 100.00 %
Elapsed time:                 179.65 secs
Data transferred:            1000.00 MB
Response time:                  1.75 secs
Transaction rate:               5.57 trans/sec
Throughput:                     5.57 MB/sec
Concurrency:                    9.72
Successful transactions:        1000
Failed transactions:               0
Longest transaction:            6.98
Shortest transaction:           0.43

privoxy-experimental 3.0.34.20211221 with mbedTLS 2.16.11

[fk@benchmark-vm ~]$ siege --concurrent=10 --reps=100 --benchmark "https://www.electrobsd.org/test/random-data-1M.raw"
** SIEGE 4.1.1
** Preparing 10 concurrent users for battle.
The server is now under siege...
Transactions:                   1000 hits
Availability:                 100.00 %
Elapsed time:                 273.14 secs
Data transferred:            1000.00 MB
Response time:                  2.57 secs
Transaction rate:               3.66 trans/sec
Throughput:                     3.66 MB/sec
Concurrency:                    9.42
Successful transactions:        1000
Failed transactions:               0
Longest transaction:           16.87
Shortest transaction:           1.12

Cipher suites and TLS versions used

TLS libraryClient side TLS version and cipher suiteServer side TLS version and cipher suite
OpenSSL 1.1.1mTLSv1.3 (TLS_AES_256_GCM_SHA384)TLSv1.3 (TLS_AES_256_GCM_SHA384)
LibreSSL 3.3.5TLSv1.3 (AEAD-AES256-GCM-SHA384)TLSv1.3 (AEAD-AES256-GCM-SHA384)
LibreSSL devel 3.4.0TLSv1.3 (AEAD-AES256-GCM-SHA384)TLSv1.3 (AEAD-AES256-GCM-SHA384)
OpenSSL 1.1.1lTLSv1.3 (TLS_AES_256_GCM_SHA384)TLSv1.3 (TLS_AES_256_GCM_SHA384)
OpenSSL devel 3.0.1TLSv1.3 (TLS_AES_256_GCM_SHA384)TLSv1.3 (TLS_AES_256_GCM_SHA384)
wolfSSL 5.0.0TLSv1.3 (TLS_AES_256_GCM_SHA384)TLSv1.3 (TLS13-AES256-GCM-SHA384)
mbedtls 2.16.11TLSv1.2 (TLS-ECDHE-RSA-WITH-CHACHA20-POLY1305-SHA256)TLSv1.2 (TLS-ECDHE-ECDSA-WITH-AES-128-GCM-SHA256)

Privoxy CPU time and resident memory use according to top after the test

TLS libraryCPU timeMemory size
OpenSSL 1.1.1m1:4918M
LibreSSL 3.3.51:4217M
LibreSSL devel 3.4.01:4217M
OpenSSL 1.1.1l1:4819M
OpenSSL devel 3.0.14:0218M
wolfSSL 5.0.01:5013M
mbedtls 2.16.113:179508K

2022-01-06: Privoxy 3.0.34 experimental 2022-01-06 af18664c0a3 tested with siege

There's a new year and thus a new quarterly branch (2022Q1) so some of the TLS backends have been updated.

The Privoxy code has been updated as well and now contains an experimental patch that allows to use elliptic-curve-cryptography instead of RSA when generating keys and Privoxy is compiled against OpenSSL 3.0.X. Doing this has been reported to improve performance on macOS in some situations.

Key generation

Privoxy has been configured as described above to force key regeneration for each request. Unless noted otherwise, Privoxy is generating 2048 bit RSA keys.

privoxy-experimental 3.0.34.20220106 with OpenSSL 1.1.1m-ElectroBSD from base

[fk@benchmark-vm ~]$ siege --concurrent=1 --reps=1000 --benchmark 'https://p.p/user-manual/'
** SIEGE 4.1.1
** Preparing 1 concurrent users for battle.
The server is now under siege...
Transactions:                   1000 hits
Availability:                 100.00 %
Elapsed time:                 219.29 secs
Data transferred:              20.70 MB
Response time:                  0.22 secs
Transaction rate:               4.56 trans/sec
Throughput:                     0.09 MB/sec
Concurrency:                    1.00
Successful transactions:        1000
Failed transactions:               0
Longest transaction:            1.98
Shortest transaction:           0.03

privoxy-experimental 3.0.34.20220106 with LibreSSL 3.4.2

The libressl and libressl-devel ports are both at version 3.4.2 so the test with libressl-devel is skipped.

[fk@benchmark-vm ~]$ siege --concurrent=1 --reps=1000 --benchmark 'https://p.p/user-manual/'
** SIEGE 4.1.1
** Preparing 1 concurrent users for battle.
The server is now under siege...
Transactions:                   1000 hits
Availability:                 100.00 %
Elapsed time:                 259.16 secs
Data transferred:              20.70 MB
Response time:                  0.26 secs
Transaction rate:               3.86 trans/sec
Throughput:                     0.08 MB/sec
Concurrency:                    1.00
Successful transactions:        1000
Failed transactions:               0
Longest transaction:            1.65
Shortest transaction:           0.09

privoxy-experimental 3.0.34.20220106 with OpenSSL 1.1.1m_1,1

[fk@benchmark-vm ~]$ siege --concurrent=1 --reps=1000 --benchmark 'https://p.p/user-manual/'
** SIEGE 4.1.1
** Preparing 1 concurrent users for battle.
The server is now under siege...
Transactions:                   1000 hits
Availability:                 100.00 %
Elapsed time:                 213.58 secs
Data transferred:              20.70 MB
Response time:                  0.21 secs
Transaction rate:               4.68 trans/sec
Throughput:                     0.10 MB/sec
Concurrency:                    1.00
Successful transactions:        1000
Failed transactions:               0
Longest transaction:            1.16
Shortest transaction:           0.03

privoxy-experimental 3.0.34.20220106 with OpenSSL devel 3.0.1 (RSA)

[fk@benchmark-vm ~]$ siege --concurrent=1 --reps=1000 --benchmark 'https://p.p/user-manual/'
** SIEGE 4.1.1
** Preparing 1 concurrent users for battle.
The server is now under siege...
Transactions:                   1000 hits
Availability:                 100.00 %
Elapsed time:                 698.57 secs
Data transferred:              20.70 MB
Response time:                  0.70 secs
Transaction rate:               1.43 trans/sec
Throughput:                     0.03 MB/sec
Concurrency:                    1.00
Successful transactions:        1000
Failed transactions:               0
Longest transaction:            3.04
Shortest transaction:           0.19

privoxy-experimental 3.0.34.20220106 with OpenSSL devel 3.0.1 (EC)

[fk@benchmark-vm ~]$ siege --concurrent=1 --reps=1000 --benchmark 'https://p.p/user-manual/'
** SIEGE 4.1.1
** Preparing 1 concurrent users for battle.
The server is now under siege...
Transactions:                   1000 hits
Availability:                 100.00 %
Elapsed time:                  38.38 secs
Data transferred:              20.70 MB
Response time:                  0.04 secs
Transaction rate:              26.06 trans/sec
Throughput:                     0.54 MB/sec
Concurrency:                    0.99
Successful transactions:        1000
Failed transactions:               0
Longest transaction:            0.23
Shortest transaction:           0.03

privoxy-experimental 3.0.34.20220106 with wolfSSL 5.1.0

[fk@benchmark-vm ~]$ siege --concurrent=1 --reps=1000 --benchmark 'https://p.p/user-manual/'
** SIEGE 4.1.1
** Preparing 1 concurrent users for battle.
The server is now under siege...
Transactions:                   1000 hits
Availability:                 100.00 %
Elapsed time:                 158.52 secs
Data transferred:              20.70 MB
Response time:                  0.16 secs
Transaction rate:               6.31 trans/sec
Throughput:                     0.13 MB/sec
Concurrency:                    1.00
Successful transactions:        1000
Failed transactions:               0
Longest transaction:            0.58
Shortest transaction:           0.04

privoxy-experimental 3.0.34.20220106 with mbedTLS 2.16.12

[fk@benchmark-vm ~]$ siege --concurrent=1 --reps=1000 --benchmark 'https://p.p/user-manual/'
** SIEGE 4.1.1
** Preparing 1 concurrent users for battle.
The server is now under siege...
Transactions:                   1000 hits
Availability:                 100.00 %
Elapsed time:                 342.58 secs
Data transferred:              20.70 MB
Response time:                  0.34 secs
Transaction rate:               2.92 trans/sec
Throughput:                     0.06 MB/sec
Concurrency:                    1.00
Successful transactions:        1000
Failed transactions:               0
Longest transaction:            2.83
Shortest transaction:           0.08

Privoxy CPU time and resident memory use according to top after the test

TLS libraryCPU timeMemory size
OpenSSL 1.1.1m2:4810M
LibreSSL 3.4.22:428596K
OpenSSL 1.1.1m_1,12:4410M
OpenSSL devel 3.0.1 (RSA)9:0011M
OpenSSL devel 3.0.1 (EC)0:2611M
wolfSSL 5.1.02:0111M
mbedtls 2.16.124:166672K