knative and language performance test 1

I am trying to understand how containerized php performs, and what are appropriate benchmarks to look at. This first test is the start of a long going research project, that really focuses on how applications perform in container based workloads.

As we follow the trends of today, fine tuned bare metal servers are starting to go away, in favor of more container based orchestration. More specifically highly responsive orchestration of containers, with the focus on short lived containers. Knative is a great example of where the industry is going, which is a great combination of longer lived and lambda like orchestration. the concept of "Lambdas" or serverless have been around for years now, but as the boot times and dynamic resource allocation begin to improve, more business critical applications and larger scale applications begin to make their way into the paradigm.

PHP has dominated the web for 30 years. due to its clever implementation of process management, short lived execution, and heroic set of utilities and helpers. Its process management was a god send on servers 20 years ago where its execution was easily configured to scale to the resources of server in a reliable and consistent way. But as short lived containers are starting the be the mechanism of controlling the resource allocation of servers and now I am worried about how process management will play in containers.

This is not meant to one of those dumb "Language vs Language" debates. But a more realistic look into how to successfully orchestrate and consider a language for an orchestrated environment.

My first goals are to get some basic hello world benchmarks just running on my macs local docker for desktop. Being the ultimate goal of testing in fully orchestrated Knative environment with auto scaling at both the Kubernetes Node level as well as the runtime container level. Following the load testing of the hello worlds I want to start testing building out real world application with dependencies like databases and caches to get an accurate representation of performance. Because it might not matter if your process can handle 10,000 concurrent connections if your database cant. Also how a language and PDO connect to the database might be where real performance is managed.

Lets have some fun together :)

Node hello world local run

hey -z 60s -c 300 http://localhost:8080

Summary:
  Total:    60.3535 secs
  Slowest:  1.8546 secs
  Fastest:  0.1543 secs
  Average:  0.8610 secs
  Requests/sec: 39178.7556
  
  Total data:   121597 bytes
  Size/request: 7 bytes

Response time histogram:
  0.154 [1] |
  0.324 [77]    |
  0.494 [34]    |
  0.664 [672]   |■■■
  0.834 [6719]  |■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■
  1.004 [7997]  |■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■
  1.174 [1633]  |■■■■■■■■
  1.345 [108]   |■
  1.515 [33]    |
  1.685 [67]    |
  1.855 [30]    |


Latency distribution:
  10% in 0.7136 secs
  25% in 0.7809 secs
  50% in 0.8531 secs
  75% in 0.9340 secs
  90% in 1.0107 secs
  95% in 1.0608 secs
  99% in 1.2597 secs

Details (average, fastest, slowest):
  DNS+dialup:   0.0005 secs, 0.1543 secs, 1.8546 secs
  DNS-lookup:   0.0102 secs, 0.0000 secs, 0.1253 secs
  req write:    0.0002 secs, 0.0000 secs, 0.0675 secs
  resp wait:    0.8495 secs, 0.1499 secs, 1.7990 secs
  resp read:    0.0009 secs, 0.0000 secs, 0.0892 secs

Status code distribution:
  [200] 17371 responses

Error distribution:
  [52]  Get "http://localhost:8080": dial tcp [::1]:8080: socket: too many open files
  [2347153] Get "http://localhost:8080": dial tcp: lookup localhost: no such host

PHP Runs on laravel lumen

hey -z 60s -c 300 http://localhost:8080

php-apache container

docker run -d -p 8080:80 php-lite:2

Summary:
  Total:    62.5828 secs
  Slowest:  9.0664 secs
  Fastest:  0.4865 secs
  Average:  5.1358 secs
  Requests/sec: 40628.8539
  

Response time histogram:
  0.486 [1] |
  1.344 [45]    |■
  2.202 [110]   |■■■
  3.060 [102]   |■■■
  3.918 [76]    |■■
  4.776 [133]   |■■■■
  5.634 [1483]  |■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■
  6.492 [936]   |■■■■■■■■■■■■■■■■■■■■■■■■■
  7.350 [42]    |■
  8.208 [12]    |
  9.066 [11]    |


Latency distribution:
  10% in 3.5980 secs
  25% in 5.0447 secs
  50% in 5.4727 secs
  75% in 5.7673 secs
  90% in 5.9218 secs
  95% in 6.0045 secs
  99% in 7.1886 secs

Details (average, fastest, slowest):
  DNS+dialup:   0.0029 secs, 0.4865 secs, 9.0664 secs
  DNS-lookup:   0.0104 secs, 0.0000 secs, 0.0556 secs
  req write:    0.0003 secs, 0.0000 secs, 0.0414 secs
  resp wait:    5.1225 secs, 0.4596 secs, 9.0092 secs
  resp read:    0.0005 secs, 0.0000 secs, 0.0467 secs

Status code distribution:
  [200] 2951 responses

Error distribution:
  [765] Get "http://localhost:8080": dial tcp [::1]:8080: socket: too many open files
  [2538951] Get "http://localhost:8080": dial tcp: lookup localhost: no such host

php-nginx container

docker run -d -p 8080:80 php-lite:1

Summary:
  Total:    62.4371 secs
  Slowest:  8.7891 secs
  Fastest:  0.4058 secs
  Average:  5.9717 secs
  Requests/sec: 35482.5298
  

Response time histogram:
  0.406 [1] |
  1.244 [44]    |■■
  2.082 [49]    |■■■
  2.921 [84]    |■■■■■
  3.759 [92]    |■■■■■
  4.597 [122]   |■■■■■■■
  5.436 [711]   |■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■
  6.274 [168]   |■■■■■■■■■
  7.112 [421]   |■■■■■■■■■■■■■■■■■■■■■■■■
  7.951 [520]   |■■■■■■■■■■■■■■■■■■■■■■■■■■■■■
  8.789 [316]   |■■■■■■■■■■■■■■■■■■


Latency distribution:
  10% in 3.5723 secs
  25% in 4.8940 secs
  50% in 6.2587 secs
  75% in 7.5772 secs
  90% in 8.0336 secs
  95% in 8.2619 secs
  99% in 8.5778 secs

Details (average, fastest, slowest):
  DNS+dialup:   0.0023 secs, 0.4058 secs, 8.7891 secs
  DNS-lookup:   0.0120 secs, 0.0000 secs, 0.1262 secs
  req write:    0.0002 secs, 0.0000 secs, 0.0370 secs
  resp wait:    5.9562 secs, 0.3831 secs, 8.7233 secs
  resp read:    0.0007 secs, 0.0000 secs, 0.0489 secs

Status code distribution:
  [200] 2528 responses

Error distribution:
  [59]  Get "http://localhost:8080": dial tcp [::1]:8080: socket: too many open files
  [2212840] Get "http://localhost:8080": dial tcp: lookup localhost: no such host