To debug HTTP requests, it may be useful to capture traffic and look at the packets that are sent back and forth between the client and the server.
This is trivial when HTTP requests are sent over an unencrypted channel. In this case it is easy enough to use a tool like tcpdump to capture the packets and inspect them with a tool like Wireshark.
For the demonstrations below, tshark is used instead of Wireshark. Tshark is the CLI-based version of Wireshark and provides more or less the same capabilities for dissecting network packets. The debugging shown here can of course be done using the GUI-based Wireshark as well.
To start off, let us look at an example, of how we can debug HTTP traffic with tshark. First, we run a simple Apache httpd server that accepts plain-text connections on port 80 and TLS encrypted connections on port 443. We will run Apache httpd inside a Docker container for convenience but it would work just the same for non-containerized Apache httpd installations, whether they are installed from the distro’s software repository or self-compiled.
To run the Apache httpd container, create a file called Dockerfile with the following contents. This sets up an Apache httpd webserver that accepts plain-text connections on port 80 by default. Additionally, we enable TLS encrypted connections on port 443 and for that we use the snake oil certificates provided by the ssl-cert package. These are self-signed certificates that can be used for testing. Since they are self-signed, they will not be recognized as trusted certificates by most user-agents but that is not a problem here.
FROM httpd:2.4
# install snakeoil certificates
RUN apt update && apt install -y ssl-cert
# enable https on port 443 with snakeoil certificates
RUN sed -i \
-e 's/^#\(Include .*httpd-ssl.conf\)/\1/' \
-e 's/^#\(LoadModule .*mod_ssl.so\)/\1/' \
-e 's/^#\(LoadModule .*mod_socache_shmcb.so\)/\1/' \
conf/httpd.conf
RUN sed -i \
-e 's|^\(SSLCertificateFile\).*|\1 /etc/ssl/certs/ssl-cert-snakeoil.pem|' \
-e 's|^\(SSLCertificateKeyFile\).*|\1 /etc/ssl/private/ssl-cert-snakeoil.key|' \
conf/extra/httpd-ssl.conf
# additionally expose port 443
EXPOSE 443
Then build it with this command:
docker build -t simple-apache-httpd .
Now, run the Apache httpd server as a container:
# -i - Keep STDIN open even if not attached.
# -t - Allocate a pseudo-TTY.
# --rm - Automatically remove the container when it exits
# --net="host" - Connect a container to a network
# --name simple-apache-httpd - Assign a name to the container
# simple-apache-httpd - The IMAGE which starts the process
docker run -it --rm --net="host" --name simple-apache-httpd simple-apache-httpd
In a separate terminal, run tcpdump to capture traffic:
# -v - When parsing and printing, produce (slightly more) verbose output.
# -i any - Listen on interface.
# "host localhost" - Filter for all packets related to localhost
# -w dump.pcap - Write the raw packets to file rather than parsing and printing them out.
sudo tcpdump -v -i any "src localhost || dst localhost" -w dump.pcap
In yet another terminal, run the curl command to perform a plaintext request to the container:
$ curl -v http://localhost
* Trying 127.0.0.1:80...
* TCP_NODELAY set
* Connected to localhost (127.0.0.1) port 80 (#0)
> GET / HTTP/1.1
> Host: localhost
> User-Agent: curl/7.68.0
> Accept: */*
>
* Mark bundle as not supporting multiuse
< HTTP/1.1 200 OK
< Date: Thu, 02 Dec 2021 12:55:40 GMT
< Server: Apache/2.4.51 (Unix) OpenSSL/1.1.1k
< Last-Modified: Mon, 11 Jun 2007 18:53:14 GMT
< ETag: "2d-432a5e4a73a80"
< Accept-Ranges: bytes
< Content-Length: 45
< Content-Type: text/html
<
<html><body><h1>It works!</h1></body></html>
* Connection #0 to host localhost left intact
Stop tcpdump and look at the packet capture with tshark:
# -r dump.pcap - Read packet data from infile # -Y http - Cause the specified filter to be applied before printing a decoded form of packets $ tshark -r dump.pcap -Y http 48 11.912607 127.0.0.1 → 127.0.0.1 HTTP 141 GET / HTTP/1.1 50 11.919104 127.0.0.1 → 127.0.0.1 HTTP 353 HTTP/1.1 200 OK (text/html)
We can clearly see the HTTP request we just performed with curl.
With TLS encrypted connections this is not as simple anymore. We would not see any HTTP traffic at all, instead we only see a TLS handshake and encrypted data packets. Note that we now need to provide the -k flag for curl since the snake oil certificates are self-signed and are therefore not trusted:
$ curl -v -k https://localhost * Trying 127.0.0.1:443... * TCP_NODELAY set * Connected to localhost (127.0.0.1) port 443 (#0) * ALPN, offering h2 * ALPN, offering http/1.1 * successfully set certificate verify locations: * CAfile: /etc/ssl/certs/ca-certificates.crt CApath: /etc/ssl/certs * TLSv1.3 (OUT), TLS handshake, Client hello (1): * TLSv1.3 (IN), TLS handshake, Server hello (2): * TLSv1.3 (IN), TLS handshake, Encrypted Extensions (8): * TLSv1.3 (IN), TLS handshake, Certificate (11): * TLSv1.3 (IN), TLS handshake, CERT verify (15): * TLSv1.3 (IN), TLS handshake, Finished (20): * TLSv1.3 (OUT), TLS change cipher, Change cipher spec (1): * TLSv1.3 (OUT), TLS handshake, Finished (20): * SSL connection using TLSv1.3 / TLS_AES_256_GCM_SHA384 * ALPN, server accepted to use http/1.1 * Server certificate: * subject: CN=f31d50e8d088 * start date: Dec 2 10:39:24 2021 GMT * expire date: Nov 30 10:39:24 2031 GMT * issuer: CN=f31d50e8d088 * SSL certificate verify result: self signed certificate (18), continuing anyway. > GET / HTTP/1.1 > Host: localhost > User-Agent: curl/7.68.0 > Accept: */* > * TLSv1.3 (IN), TLS handshake, Newsession Ticket (4): * TLSv1.3 (IN), TLS handshake, Newsession Ticket (4): * old SSL session ID is stale, removing * Mark bundle as not supporting multiuse < HTTP/1.1 200 OK < Date: Thu, 02 Dec 2021 13:00:23 GMT < Server: Apache/2.4.51 (Unix) OpenSSL/1.1.1k < Last-Modified: Mon, 11 Jun 2007 18:53:14 GMT < ETag: "2d-432a5e4a73a80" < Accept-Ranges: bytes < Content-Length: 45 < Content-Type: text/html < <html><body><h1>It works!</h1></body></html> * Connection #0 to host localhost left intact
We cannot see the HTTP request anymore:
$ tshark -r dump.pcap -Y http
We can only see the encrypted TLS packets:
$ tshark -r dump.pcap -Y tls 13 1.492489 127.0.0.1 → 127.0.0.1 TLSv1 585 Client Hello 15 1.496532 127.0.0.1 → 127.0.0.1 TLSv1.3 1381 Server Hello, Change Cipher Spec, ... 17 1.497260 127.0.0.1 → 127.0.0.1 TLSv1.3 148 Change Cipher Spec, Application Data 19 1.497486 127.0.0.1 → 127.0.0.1 TLSv1.3 163 Application Data 21 1.497556 127.0.0.1 → 127.0.0.1 TLSv1.3 355 Application Data 23 1.497709 127.0.0.1 → 127.0.0.1 TLSv1.3 355 Application Data 25 1.498220 127.0.0.1 → 127.0.0.1 TLSv1.3 375 Application Data 27 1.498655 127.0.0.1 → 127.0.0.1 TLSv1.3 92 Application Data 29 1.498830 127.0.0.1 → 127.0.0.1 TLSv1.3 92 Application Data
The traffic is now encrypted and we would need to decrypt the captured packets in order to inspect the HTTP traffic.
For the purpose of inspecting the traffic, TLS connections can be grouped into two groups. Connections that leverage Perfect Forward Secrecy (PFS) and connections that do not.
The main difference between the two types of connections is that connections without PFS can be decrypted after the fact using the long-lived private key of the server. For connections with PFS, this is not possible because short-lived, ephemeral keys are negotiated between the client and server. This is usually achieved by a Diffie-Hellman key exchange or a derivative thereof. The ephemeral keys generated during the key exchange are never persisted or reused for other connections. This prevents the traffic from being decrypted without being a party to the initial handshake of the connection.
Let us first look at connections without PFS. Here, it suffices to pass the private key of the server to Wireshark to decrypt traffic. For this example, we have to ensure, that we use TLS parameters that do not leverage PFS. To achieve that, we explicitly tell curl to use the protocol TLSv1.2 and the cipher suite CAMELLIA128-SHA.
We repeat the example above, capturing the traffic:
$ curl -v -k --tlsv1.2 --tls-max 1.2 --ciphers CAMELLIA128-SHA https://localhost
* Trying 127.0.0.1:443...
* TCP_NODELAY set
* Connected to localhost (127.0.0.1) port 443 (#0)
* ALPN, offering h2
* ALPN, offering http/1.1
* Cipher selection: CAMELLIA128-SHA
* successfully set certificate verify locations:
* CAfile: /etc/ssl/certs/ca-certificates.crt
CApath: /etc/ssl/certs
* TLSv1.2 (OUT), TLS handshake, Client hello (1):
* TLSv1.2 (IN), TLS handshake, Server hello (2):
* TLSv1.2 (IN), TLS handshake, Certificate (11):
* TLSv1.2 (IN), TLS handshake, Server finished (14):
* TLSv1.2 (OUT), TLS handshake, Client key exchange (16):
* TLSv1.2 (OUT), TLS change cipher, Change cipher spec (1):
* TLSv1.2 (OUT), TLS handshake, Finished (20):
* TLSv1.2 (IN), TLS handshake, Finished (20):
* SSL connection using TLSv1.2 / CAMELLIA128-SHA
* ALPN, server accepted to use http/1.1
* Server certificate:
* subject: CN=f31d50e8d088
* start date: Dec 2 10:39:24 2021 GMT
* expire date: Nov 30 10:39:24 2031 GMT
* issuer: CN=f31d50e8d088
* SSL certificate verify result: self signed certificate (18), continuing anyway.
> GET / HTTP/1.1
> Host: localhost
> User-Agent: curl/7.68.0
> Accept: */*
>
* Mark bundle as not supporting multiuse
< HTTP/1.1 200 OK
< Date: Thu, 02 Dec 2021 13:34:01 GMT
< Server: Apache/2.4.51 (Unix) OpenSSL/1.1.1k
< Last-Modified: Mon, 11 Jun 2007 18:53:14 GMT
< ETag: "2d-432a5e4a73a80"
< Accept-Ranges: bytes
< Content-Length: 45
< Content-Type: text/html
<
<html><body><h1>It works!</h1></body></html>
* Connection #0 to host localhost left intact
$ tshark -r dump.pcap -Y tls
6 0.420061 127.0.0.1 → 127.0.0.1 TLSv1 216 Client Hello
8 0.420403 127.0.0.1 → 127.0.0.1 TLSv1.2 943 Server Hello, Certificate, Server Hello Done
10 0.420756 127.0.0.1 → 127.0.0.1 TLSv1.2 414 Client Key Exchange, Change Cipher Spec, ...
12 0.421699 127.0.0.1 → 127.0.0.1 TLSv1.2 147 Change Cipher Spec, Encrypted Handshake Message
14 0.421923 127.0.0.1 → 127.0.0.1 TLSv1.2 189 Application Data
16 0.422142 127.0.0.1 → 127.0.0.1 TLSv1.2 397 Application Data
18 0.422378 127.0.0.1 → 127.0.0.1 TLSv1.2 125 Encrypted Alert
20 0.422497 127.0.0.1 → 127.0.0.1 TLSv1.2 125 Encrypted Alert
Now we extract the TLS private key of the server:
docker exec -it simple-apache-httpd cat /etc/ssl/private/ssl-cert-snakeoil.key > ssl-cert-snakeoil.key
This allow us to use the private key in tshark by specifying it using the -o flag:
# -r dump.pcap - Read packet data from infile
# -Y http - Cause the specified filter to be applied before printing a decoded form of packets
# -o tls.keys_list:,,,ssl-cert-snakeoil.key - override values from the preferences files
$ tshark -o tls.keys_list:,,,ssl-cert-snakeoil.key -r dump.pcap -Y tls
6 0.420061 127.0.0.1 → 127.0.0.1 TLSv1 216 Client Hello
8 0.420403 127.0.0.1 → 127.0.0.1 TLSv1.2 943 Server Hello, Certificate, Server Hello Done
10 0.420756 127.0.0.1 → 127.0.0.1 TLSv1.2 414 Client Key Exchange, Change Cipher Spec, Finished
12 0.421699 127.0.0.1 → 127.0.0.1 TLSv1.2 147 Change Cipher Spec, Finished
14 0.421923 127.0.0.1 → 127.0.0.1 HTTP 189 GET / HTTP/1.1
16 0.422142 127.0.0.1 → 127.0.0.1 HTTP 397 HTTP/1.1 200 OK (text/html)
18 0.422378 127.0.0.1 → 127.0.0.1 TLSv1.2 125 Alert (Level: Warning, Description: Close Notify)
20 0.422497 127.0.0.1 → 127.0.0.1 TLSv1.2 125 Alert (Level: Warning, Description: Close Notify)
As stated above, this approach will not work with PFS. This means that if only TLS parameters with PFS are available, it will not be possible to use the approach described above to decrypt traffic for debugging. For TLSv1.3 this is always the case because TLSv1.3 mandates PFS. We do not have to specify this explicitly as curl will automatically use TLSv1.3.
To illustrate this, we show an example just as before.
$ curl -v -k https://localhost
* Trying 127.0.0.1:443...
* TCP_NODELAY set
* Connected to localhost (127.0.0.1) port 443 (#0)
* ALPN, offering h2
* ALPN, offering http/1.1
* successfully set certificate verify locations:
* CAfile: /etc/ssl/certs/ca-certificates.crt
CApath: /etc/ssl/certs
* TLSv1.3 (OUT), TLS handshake, Client hello (1):
* TLSv1.3 (IN), TLS handshake, Server hello (2):
* TLSv1.3 (IN), TLS handshake, Encrypted Extensions (8):
* TLSv1.3 (IN), TLS handshake, Certificate (11):
* TLSv1.3 (IN), TLS handshake, CERT verify (15):
* TLSv1.3 (IN), TLS handshake, Finished (20):
* TLSv1.3 (OUT), TLS change cipher, Change cipher spec (1):
* TLSv1.3 (OUT), TLS handshake, Finished (20):
* SSL connection using TLSv1.3 / TLS_AES_256_GCM_SHA384
* ALPN, server accepted to use http/1.1
* Server certificate:
* subject: CN=f31d50e8d088
* start date: Dec 2 10:39:24 2021 GMT
* expire date: Nov 30 10:39:24 2031 GMT
* issuer: CN=f31d50e8d088
* SSL certificate verify result: self signed certificate (18), continuing anyway.
> GET / HTTP/1.1
> Host: localhost
> User-Agent: curl/7.68.0
> Accept: */*
>
* TLSv1.3 (IN), TLS handshake, Newsession Ticket (4):
* TLSv1.3 (IN), TLS handshake, Newsession Ticket (4):
* old SSL session ID is stale, removing
* Mark bundle as not supporting multiuse
< HTTP/1.1 200 OK
< Date: Thu, 02 Dec 2021 14:06:14 GMT
< Server: Apache/2.4.51 (Unix) OpenSSL/1.1.1k
< Last-Modified: Mon, 11 Jun 2007 18:53:14 GMT
< ETag: "2d-432a5e4a73a80"
< Accept-Ranges: bytes
< Content-Length: 45
< Content-Type: text/html
<
<html><body><h1>It works!</h1></body></html>
* Connection #0 to host localhost left intact
With PFS, we again see only the TLS handshake. Despite providing the servers private key as before, the packets remain encrypted.
$ tshark -o tls.keys_list:,,,ssl-cert-snakeoil.key -r dump.pcap -Y tls 20 1.660863 127.0.0.1 → 127.0.0.1 TLSv1 585 Client Hello 22 1.676658 127.0.0.1 → 127.0.0.1 TLSv1.3 1381 Server Hello, Change Cipher Spec, ... 24 1.677228 127.0.0.1 → 127.0.0.1 TLSv1.3 148 Change Cipher Spec, Application Data 26 1.677406 127.0.0.1 → 127.0.0.1 TLSv1.3 355 Application Data 27 1.677415 127.0.0.1 → 127.0.0.1 TLSv1.3 163 Application Data 30 1.677504 127.0.0.1 → 127.0.0.1 TLSv1.3 355 Application Data 32 1.680826 127.0.0.1 → 127.0.0.1 TLSv1.3 375 Application Data 34 1.681131 127.0.0.1 → 127.0.0.1 TLSv1.3 92 Application Data 36 1.681262 127.0.0.1 → 127.0.0.1 TLSv1.3 92 Application Data
In order to decrypt this traffic we will have to log the ephemeral keys on either the client or the server side.
More and more technologies adopt the SSLKEYLOGFILE environment variable as a way to specify a log file where the ephemeral keys can be logged. These logs can then be passed to tshark to decrypt the traffic, similarly to how we used the servers private key before. Note that the ephemeral keys are not logged at all if this environment variable is not set.
In order to inspect traffic the SSLKEYLOGFILE environment variable must be set on the server or client side while the traffic is generated. With this log being generated, capturing and decrypting traffic works very similarly to how it worked before.
Let us first run an example where we set the SSLKEYLOGFILE environment variable on the client side with curl.
Make sure you use curl 7.58 or newer for this to work.
$ SSLKEYLOGFILE=sslkeylogfile.log curl -v -k https://localhost
* Trying 127.0.0.1:443...
* TCP_NODELAY set
* Connected to localhost (127.0.0.1) port 443 (#0)
* ALPN, offering h2
* ALPN, offering http/1.1
* successfully set certificate verify locations:
* CAfile: /etc/ssl/certs/ca-certificates.crt
CApath: /etc/ssl/certs
* TLSv1.3 (OUT), TLS handshake, Client hello (1):
* TLSv1.3 (IN), TLS handshake, Server hello (2):
* TLSv1.3 (IN), TLS handshake, Encrypted Extensions (8):
* TLSv1.3 (IN), TLS handshake, Certificate (11):
* TLSv1.3 (IN), TLS handshake, CERT verify (15):
* TLSv1.3 (IN), TLS handshake, Finished (20):
* TLSv1.3 (OUT), TLS change cipher, Change cipher spec (1):
* TLSv1.3 (OUT), TLS handshake, Finished (20):
* SSL connection using TLSv1.3 / TLS_AES_256_GCM_SHA384
* ALPN, server accepted to use http/1.1
* Server certificate:
* subject: CN=f31d50e8d088
* start date: Dec 2 10:39:24 2021 GMT
* expire date: Nov 30 10:39:24 2031 GMT
* issuer: CN=f31d50e8d088
* SSL certificate verify result: self signed certificate (18), continuing anyway.
> GET / HTTP/1.1
> Host: localhost
> User-Agent: curl/7.68.0
> Accept: */*
>
* TLSv1.3 (IN), TLS handshake, Newsession Ticket (4):
* TLSv1.3 (IN), TLS handshake, Newsession Ticket (4):
* old SSL session ID is stale, removing
* Mark bundle as not supporting multiuse
< HTTP/1.1 200 OK
< Date: Thu, 02 Dec 2021 14:17:04 GMT
< Server: Apache/2.4.51 (Unix) OpenSSL/1.1.1k
< Last-Modified: Mon, 11 Jun 2007 18:53:14 GMT
< ETag: "2d-432a5e4a73a80"
< Accept-Ranges: bytes
< Content-Length: 45
< Content-Type: text/html
<
<html><body><h1>It works!</h1></body></html>
* Connection #0 to host localhost left intact
With the key log written by curl, we can decrypt the traffic again. Note that the server’s private key is not required anymore, the key log file is all we need.
# -r dump.pcap - Read packet data from infile
# -Y http - Cause the specified filter to be applied before printing a decoded form of packets
# -o tls.keylog_file:sslkeylogfile.log - override values from the preferences files
$ tshark -o tls.keylog_file:sslkeylogfile.log -r dump.pcap -Y tls
4 0.007706 127.0.0.1 → 127.0.0.1 TLSv1 585 Client Hello
6 0.009024 127.0.0.1 → 127.0.0.1 TLSv1.3 1381 Server Hello, Change Cipher Spec, ...
8 0.009576 127.0.0.1 → 127.0.0.1 TLSv1.3 148 Change Cipher Spec, Finished
10 0.009790 127.0.0.1 → 127.0.0.1 HTTP 163 GET / HTTP/1.1
11 0.009796 127.0.0.1 → 127.0.0.1 TLSv1.3 355 New Session Ticket
14 0.009941 127.0.0.1 → 127.0.0.1 TLSv1.3 355 New Session Ticket
16 0.010132 127.0.0.1 → 127.0.0.1 HTTP 375 HTTP/1.1 200 OK (text/html)
18 0.010355 127.0.0.1 → 127.0.0.1 TLSv1.3 92 Alert (Level: Warning, Description: Close Notify)
20 0.010458 127.0.0.1 → 127.0.0.1 TLSv1.3 92 Alert (Level: Warning, Description: Close Notify)
Similarly, we can set the SSLKEYLOGFILE environment variable on the server side.
For this to work, Apache httpd 2.4.49 or later is required.
We restart the server container and pass the SSLKEYLOGFILE environment variable.
docker run -it --rm --net="host" --name simple-apache-httpd -e SSLKEYLOGFILE="/usr/local/apache2/logs/sslkeylogfile.log" simple-apache-httpd
The following log entry signals that the key log file will be written:
[Tue Dec 21 20:03:20.441733 2021] [ssl:notice] [pid 1:tid 139780428569920] AH10227: Init: Logging SSL private key material to /usr/local/apache2/logs/sslkeylogfile.log
We are ready to repeat the request:
$ curl -v -k https://localhost
* Trying 127.0.0.1:443...
* TCP_NODELAY set
* Connected to localhost (127.0.0.1) port 443 (#0)
* ALPN, offering h2
* ALPN, offering http/1.1
* successfully set certificate verify locations:
* CAfile: /etc/ssl/certs/ca-certificates.crt
CApath: /etc/ssl/certs
* TLSv1.3 (OUT), TLS handshake, Client hello (1):
* TLSv1.3 (IN), TLS handshake, Server hello (2):
* TLSv1.3 (IN), TLS handshake, Encrypted Extensions (8):
* TLSv1.3 (IN), TLS handshake, Certificate (11):
* TLSv1.3 (IN), TLS handshake, CERT verify (15):
* TLSv1.3 (IN), TLS handshake, Finished (20):
* TLSv1.3 (OUT), TLS change cipher, Change cipher spec (1):
* TLSv1.3 (OUT), TLS handshake, Finished (20):
* SSL connection using TLSv1.3 / TLS_AES_256_GCM_SHA384
* ALPN, server accepted to use http/1.1
* Server certificate:
* subject: CN=f31d50e8d088
* start date: Dec 2 10:39:24 2021 GMT
* expire date: Nov 30 10:39:24 2031 GMT
* issuer: CN=f31d50e8d088
* SSL certificate verify result: self signed certificate (18), continuing anyway.
> GET / HTTP/1.1
> Host: localhost
> User-Agent: curl/7.68.0
> Accept: */*
>
* TLSv1.3 (IN), TLS handshake, Newsession Ticket (4):
* TLSv1.3 (IN), TLS handshake, Newsession Ticket (4):
* old SSL session ID is stale, removing
* Mark bundle as not supporting multiuse
< HTTP/1.1 200 OK
< Date: Thu, 02 Dec 2021 14:23:13 GMT
< Server: Apache/2.4.51 (Unix) OpenSSL/1.1.1k
< Last-Modified: Mon, 11 Jun 2007 18:53:14 GMT
< ETag: "2d-432a5e4a73a80"
< Accept-Ranges: bytes
< Content-Length: 45
< Content-Type: text/html
<
<html><body><h1>It works!</h1></body></html>
* Connection #0 to host localhost left intact
After we performed and captured the request, we need to get the log file from inside the container to use it with tshark.
docker exec -it simple-apache-httpd cat /usr/local/apache2/logs/sslkeylogfile.log > sslkeylogfile.log
Now we can decrypt the traffic as before by again passing the key log file to tshark:
$ tshark -o tls.keylog_file:sslkeylogfile.log -r dump.pcap -Y tls
6 0.033156 127.0.0.1 → 127.0.0.1 TLSv1 585 Client Hello
8 0.035385 127.0.0.1 → 127.0.0.1 TLSv1.3 1381 Server Hello, Change Cipher Spec, ...
10 0.035890 127.0.0.1 → 127.0.0.1 TLSv1.3 148 Change Cipher Spec, Finished
12 0.036026 127.0.0.1 → 127.0.0.1 HTTP 163 GET / HTTP/1.1
14 0.036078 127.0.0.1 → 127.0.0.1 TLSv1.3 355 New Session Ticket
16 0.036194 127.0.0.1 → 127.0.0.1 TLSv1.3 355 New Session Ticket
18 0.036511 127.0.0.1 → 127.0.0.1 HTTP 375 HTTP/1.1 200 OK (text/html)
20 0.036814 127.0.0.1 → 127.0.0.1 TLSv1.3 92 Alert (Level: Warning, Description: Close Notify)
22 0.036932 127.0.0.1 → 127.0.0.1 TLSv1.3 92 Alert (Level: Warning, Description: Close Notify)
This approach, using the SSLKEYLOGFILE environment variable works with some browsers as well. However, it might be disabled through a compile time variable because of the risk it poses. If an attacker manages to set this environment variable for either the client or server and has access to the key log, the attacker could easily break TLS encryption as shown above.
For this reason it might be a good idea to monitor the logs for the entry indicating the use of the SSLKEYLOGFILE environment variable as shown above or even add a separate log, where the SSLKEYLOGFILE variable is logged if present. This allows you to determine after the fact which requests were affected, if the environment variable was set.
Further, it might be a good idea to monitor the environment variables of all processes to determine if this variable is set anywhere. An easy way to do this manually is the following command. It shows that the SSLKEYLOGFILE environment variable is in use and where the key log file is written:
$ sudo cat /proc/*/environ | tr '\0' '\n' | grep SSLKEYLOGFILE
SSLKEYLOGFILE=/usr/local/apache2/logs/sslkeylogfile.log
SSLKEYLOGFILE=/usr/local/apache2/logs/sslkeylogfile.log
SSLKEYLOGFILE=/usr/local/apache2/logs/sslkeylogfile.log
SSLKEYLOGFILE=/usr/local/apache2/logs/sslkeylogfile.log
To summarize, we see that the introduction of the SSLKEYLOGFILE environment variable can be of great benefit when debugging TLS encrypted connections. However, it also introduces a new attack vector to break TLS encryption that needs to be addressed by disabling or at least by monitoring its use.