We are capturing the entire HTTP traffic. We will also be decrypting traffic where necessary.<\/p>\n
In daily life, when operating a web or reverse proxy server errors occur that can only be handled with difficultly come up again and again. In numerous cases there is a lack of clarity about what has just passed over the line or there is disagreement about exactly which end of communication was responsible for the error. In cases such as these it is important to be able to capture the entire traffic in order to narrow down the error to this basis.<\/p>\n
In Tutorial 6 we saw how we are able to configure ModSecurity to capture the entire traffic from a single client IP address. However, depending on the settings of the In Tutorial 6 we made the following selection for the individual headers:<\/p>\n We have defined a very comprehensive log. This is the right approach in a lab-like setup. However, in a production environment this is only useful in exceptional cases. A typical variation of this directive in a production environment would thus be:<\/p>\n The request and response bodies are no longer being captured. This saves a lot of storage space, which is important on badly tuned systems. The parts of the body that violate individual rules are nonetheless written to the error log and in Part K. This is sufficient in most cases. However, from case to case, you will still want to capture the entire body. In cases such as these you can use a The first step enables the dynamic modification of audit log parts for a known IP address. But what if we want to permanently enable dynamic logging for selected sessions and, as shown in the example above, expand it to the entire request?<\/p>\n In his ModSecurity Handbook Ivan Risti\u0107 describes an example in which a ModSecurity In the integration of core rules proposed in the preceding tutorials we have already opened a persistent We\u2019ll use this ability to check its In the third rule we check whether this Forcing the audit log, which we have not yet become familiar with, is required, because we want to log requests that don\u2019t actually violate any of the rules. This means that the audit log is not yet enabled for the request. We make up for this with this rule.<\/p>\n Altogether, these three rules enable us to precisely monitor a conspicuous client beyond an individual suspicious request and to capture the entire client traffic in the audit log once suspicion has been aroused.<\/p>\n Traffic between the client and the reverse proxy can normally be well documented using the methods described. In addition, we have the option of documenting traffic on the client. Modern browsers provide options for this and they all seem adequate to me. In practice however there are complications that can make capturing traffic difficult or impossible. Be it a fat client being used outside a browser, the client being used only on a mobile device with an interposed proxy modifying traffic in one direction or the other in such a way that traffic is being modified once more by another module after leaving ModSecurity or that ModSecurity has no access to traffic whatsoever. In individual cases the latter is actually a problem, because an Apache module can abort the processing of a request and suppress access from ModSecurity by doing so.<\/p>\n In these cases one option is to interpose a separate proxy to capture the traffic. A number of tools are available. It is therefore possible for entries in the audit log to not match what was received by the client or no longer match what the client originally sent. In these cases it is preferable to selectively capture the actual traffic and decrypt the encrypted data. This suggestion, however, runs up against the strong encryption we configured in the fourth tutorial to secure it from snooping. The ciphers we prefer rely on In all other cases in which we want to force decryption, but are unable to reconfigure the client, we have to employ a different, weaker type of encryption which is unaware of As a test, Apache can also be configured using conditional In the fourth tutorial we operated the local Laboratory Service using the local The explanations above set the stage for capturing and then decrypting traffic. We\u2019ll be doing it in two steps, first logging the traffic and then decrypting the log. Capturing is also called Alternatively:<\/p>\n Here, the two commands that generate an identical log have been instructed to listen to the local These commands are used to start the log and we can now trigger the traffic in a second window. Let\u2019s give it a try using SecAuditLogParts<\/code> directive, not all parts of the requests are recorded. Let\u2019s have a look at the different options in this directive: The ModSecurity audit engine labels different parts of the audit log using different letter abbreviations. They are as follows:<\/p>\n\n
SecRequestBodyAccess<\/code>)<\/li>\nSecRequestBodyAccess<\/code>)<\/li>\nAction<\/code> taken, timing information, etc. It\u2019s worth taking a look.<\/li>\n<\/a>SecAuditLogParts<\/span> ABEFHIJKZ<\/span><\/code><\/pre><\/div>\n<\/a>SecAuditLogParts<\/span> ABFHKZ<\/span><\/code><\/pre><\/div>\nctl<\/code> directive for the action part of the SecRule<\/code>. Multiple, additional parts can be selected via auditLogParts<\/code>:<\/p>\n<\/a>SecRule<\/span> REMOTE_ADDR "@streq 127.0.0.1"<\/span> \\<\/span><\/span>\n<\/a> "id:10000,phase:1,pass,log,auditlog,msg:'Initializing full traffic log',ctl:auditLogParts=+EIJ"<\/span><\/span><\/code><\/pre><\/div>\nStep 2: Using ModSecurity to write the entire traffic of a single session<\/h3>\n
collection<\/code> is employed to generate a separate session which remains enabled beyond an individual request. We\u2019ll use this idea as the starting point and write a somewhat more complex example:<\/p>\n<\/a>SecRule<\/span> TX:INBOUND_ANOMALY_SCORE "@ge 5"<\/span> \\<\/span><\/span>\n<\/a> "phase:5,pass,id:10001,log,msg:'Logging enabled (High incoming anomaly score)', \\<\/span><\/span>\n<\/a> expirevar:ip.logflag=600"<\/span><\/span>\n<\/a><\/span>\n<\/a>SecRule<\/span> TX:OUTBOUND_ANOMALY_SCORE "@ge 5"<\/span> \\<\/span><\/span>\n<\/a> "phase:5,pass,id:10002,log,msg:'Logging enabled (High outgoing anomaly score)', \\<\/span><\/span>\n<\/a> expirevar:ip.logflag=600"<\/span><\/span>\n<\/a><\/span>\n<\/a>SecRule<\/span> &<\/span>IP<\/span>:LOGFLAG "@eq 1"<\/span> \\<\/span><\/span>\n<\/a> "phase:5,pass,id:10003,log,msg:'Logging is enabled. Enforcing rich auditlog.', \\<\/span><\/span>\n<\/a> ctl:auditEngine=On,ctl:auditLogParts=+EIJ"<\/span><\/span><\/code><\/pre><\/div>\ncollection<\/code> based on the IP address of the client making the request. The collection<\/code> which is stored beyond an individual request is suitable for retaining data between different requests.<\/p>\ncore rules anomaly score<\/code> in the logging phase of the request. If it is 5 or higher (corresponding to an alarm or the critical<\/code> level), we set the variable ip.logflag<\/code> and via expirevar<\/code> give it an expiration of 600 seconds. This means that this variable remains in the IP collection<\/code> for ten minutes and then disappears on its own automatically. This mechanism is repeated for the outgoing anomaly score<\/code> in the subsequent rule.<\/p>\nLogflag<\/code> is set. We earlier saw a wondrous transformation of variable names depending on application in ModSecurity<\/code>. We are seeing it again here, in which ip.logflag<\/code> must be written in a SecRule<\/code> as the variable IP:LOGFLAG<\/code>. We have also become familiar with the &<\/code> at the beginning: It denotes the number of variables of this name (0 or 1). This means we can check for the presence of ip.logflag<\/code>. If the flag is previously set in both rules or at an earlier point in time in the past 10 minutes, then the audit engine is enabled and parts of the log that are not always set in the default configuration are added.<\/p>\nStep 3: Sniffing client traffic with the server\/reverse proxy<\/h3>\n
mitmproxy<\/code>, in particular, appears to have some very interesting features and I use it to great effect. But because the development of this software is still very dynamic, installation of the current version can be quite demanding, which is why I won\u2019t be going into any of the details here. We\u2019ll select a somewhat rougher method.<\/p>\nforward secrecy<\/code> for this. This means that a snooper is foiled in such a way that even possession of the long-term asymmetric SSL key pair no longer permits snooping. But this also means that no capturing of the traffic of any kind is possible between the client and the server unless we position a process in between that terminates the connection and presents a separate certificate to the client.<\/p>\nforward secrecy<\/code>. Something like the AES256-SHA<\/code> cipher which we defined as the only cipher on the client and use to connect to the server. If we are unable to use the cipher on the client side, then we have to weaken encryption on the entire server. It\u2019s immediately obvious that this is not desired and is only useful in a few instances. Be it us binding the client to a separate system or putting a time limit on reconfiguration.<\/p>\n<if><\/code> directives, presenting another cipher to an individual client. However, this will only work via SSL renegotiate<\/code>. This means that an SSL handshake was carried out using forward secrecy<\/code>, but it was then repeated with a weaker cipher. However, in my tests the commonly used decryption tools wireshark<\/code> and ssldump<\/code> were unable to handle this method. This means switching the server over to weaker encryption for the moment. In terms of security, I strong advise against relying on this variation until all other means have been exhausted.<\/p>\nsnake-oil<\/code> key. We are going to use this certificate once more and instruct the server to use the decryptable AES256-SHA<\/code> cipher.<\/p>\n<\/a> ...<\/span><\/span>\n<\/a><\/span>\n<\/a><\/span>\n<\/a> SSLCertificateKeyFile<\/span> \/etc\/ssl\/private\/ssl-cert-snakeoil.key<\/span>\n<\/a> SSLCertificateFile<\/span> \/etc\/ssl\/certs\/ssl-cert-snakeoil.pem<\/span>\n<\/a><\/span>\n<\/a> SSLProtocol<\/span> All -SSLv2 -SSLv3 -TLSv1 -TLSv1.1<\/span>\n<\/a> SSLCipherSuite<\/span> 'AES256-SHA'<\/span><\/span>\n<\/a> SSLHonorCipherOrder<\/span> On<\/span>\n<\/a><\/span>\n<\/a> ...<\/span><\/span><\/code><\/pre><\/div>\nStep 4: Capturing encrypted traffic between the client and the server\/reverse proxy<\/h3>\n
pulling a PCAP<\/code>. This means we are providing a PCAP<\/code> file, or a network traffic log in PCAP<\/code> format. PCAP<\/code> means packet capture<\/code>. For this we\u2019ll either be using the most widespread tool, tcpdump<\/code>, or tshark<\/code> from the Wireshark<\/code> suite. It is also possible to work right away in the Wireshark<\/code> graphical interface.<\/p>\n<\/a>$><\/span> sudo<\/span> tcpdump -i lo -w \/tmp\/localhost-port443.pcap -s0 port 443<\/span>\n<\/a>tcpdump<\/span>: listening on lo, link-type EN10MB (Ethernet), capture<\/span> size 65535 bytes<\/span>\n<\/a>...<\/span><\/span><\/code><\/pre><\/div>\n<\/a>$><\/span> sudo<\/span> tshark -i lo -w \/tmp\/localhost-port443.pcap -s0 port 443<\/span>\n<\/a>tshark<\/span>: Lua: Error during loading:<\/span>\n<\/a> [string<\/span> "\/usr\/share\/wireshark\/init.lua"<\/span>]:46: dofile has been disabled due to running Wireshark as ...<\/span>\n<\/a>Running<\/span> as user "root"<\/span> and group "root"<\/span>. This could be dangerous.<\/span>\n<\/a>Capturing<\/span> on 'Loopback'<\/span><\/span>\n<\/a>...<\/span><\/span><\/code><\/pre><\/div>\nlo<\/code> interface and port 443 and to write to the file localhost-port443.pcap<\/code>. The -s0<\/code> option is important. This is referred to as the snap length<\/code> or capture size<\/code>. This indicates exactly how much data to capture from an IP packet. In our case we want the entire packet. The instruction for this is the value 0, which means everything.<\/p>\ncurl<\/code>:<\/p>\n<\/a>$><\/span> curl<\/span> -v --ciphers AES256-SHA -k https:\/\/127.0.0.1:443\/index.html<\/span>\n<\/a>*<\/span> Rebuilt URL to: https:\/\/localhost:443\/<\/span>\n<\/a>*<\/span> Trying 127.0.0.1...<\/span>\n<\/a>*<\/span> Connected to localhost (127.0.0.1) port<\/span> 443 (#0)<\/span>\n<\/a>*<\/span> found 173 certificates in \/etc\/ssl\/certs\/ca-certificates.crt<\/span>\n<\/a>*<\/span> found 697 certificates in \/etc\/ssl\/certs<\/span>\n<\/a>*<\/span> ALPN, offering http\/1.1<\/span>\n<\/a>*<\/span> SSL connection using TLS1.2 \/ RSA_AES_256_CBC_SHA1<\/span>\n<\/a>*<\/span> server certificate verification SKIPPED<\/span>\n