Microservices play crucial role to make systems more business agile that allows to change quickly and are flexible in introducing new functionality at rapid pace. However, with microservices architecture, comes a lot of networking and communication complexity. Applications often spend time processing these network functions rather than doing core business logic. So, what is the solution?
With applications deployed on k8s, adding sidecars offload all of this processing from application container. Envoy proxy and Splunk forwarder are such helpers that can be easily integrated with application container.
Introducing Envoy proxy
Envoy is a self-contained process that runs taking very little memory as a sidecar. Quoting Envoy Docs — “All of the Envoys form a transparent communication mesh in which each application sends and receives messages to and from localhost and is unaware of the network topology.”
We introduced Envoy proxy to off load decryption of encrypted payload over SSL. To accomplish this, a small container with 200mb CPU is added. It also requires a config map to configure cluster details and load-balancing, listen to port 8443, specify certificates, filters and route-config. Envoy would listen requests coming in at 8443, decrypt SSL and forward it to localhost at port 8080.
To enable envoy, a new container is added alongside app container. This is what it looks like –
Next comes Splunk forwarder
To further optimise the application, we introduced Splunk sidecar pattern. Similar to envoy, as new container and splunk config is added to process forwarding generated logs to Splunk.
Logs are written to a file and not console to make this work seamlessly. A few things to notice are — while doing so, a good understanding of space available for log files, logs throughput and optimal rotation of files is needed else the disk space could fill up pretty fast making application to go down. It is good to zip the files while rolling them to save disk space.
We used size-based log file rotation with 2gb for each file having 10 max files. Log4j configuration is something like –
After implementing Envoy proxy and Splunk forwarder sidecar pattern, we did a through testing and the results are promising with more than 50% decrease in CPU utilisation and 4x faster response time.
