Writing the 2nd Edition of the ModSecurity Handbook


What I like about Ivan Ristić’s ModSecurity Book is the wide approach it takes. The multipurpose nature of ModSecurity makes it hard to overview all the areas and all the functionality. But Ivan did a very good job providing a gentle introduction to all these areas. Hence, it proved to be the standard book for many years.

The online information about ModSecurity is unfortunately a bit scattered. There is a large number of blog posts written about individual features over the years. The directives and variables are covered in the official reference manual, but – truth be told – said half-titlemanual is lacking a bit. So the ModSecurity Handbook by the original developer has always had a quasi-official status.

But of course, it is showing its age six years after the initial release. Many things dubbed new or innovative when the book came out are not so new anymore. All the references to the latest release (2.5.12) are outdated now. When version 2.7.0 made rule ids mandatory, all examples for the rule language stopped working.

This has resulted in a situation where newcomers have a hard time to start with ModSecurity. Not even the book is providing them an introduction free from hassles. Also, the reference part is no longer complete. In fact, the gap between 2.5.12 and 2.9.1 is almost exactly 1/3 of the directives and variables.

So Ivan Ristić has asked me to update the book: He has asked me to write the 2nd edition of the ModSecurity Handbook. Now writing technical books is no real business anymore. The publisher hopes to cover his costs and the author, well the author will get a decent share of the sales. But this share is nowhere near a reasonable financial compensation for the work he or she is putting into a book. Ivan therefore called it a labour of love. Now, everybody who knows my passion for ModSecurity will understand that he hit the right tune there.

So I spent my Summer mornings closing the gap to the latest ModSecurity release. I completed the reference part and enhanced it with new information. This is best visible with the transformations. Transformations are an important topic, but they are also a bit black magic to many people. So I added before/after transformation examples for all ascii-based transformations. This makes the transformation much easier to understand and apply correctly.

transformation

The escapeSeqDecode transformation serves as an example of what people call ModSec black magic.

I have updated the main part of the book to match the latest ModSecurity release. In the process, I rewrote many passages or I updated them. Then I tested all the examples against the latest version of ModSecurity. In the end, I updated the majority of examples. As with every book, many small bugs had to be fixed (bugs that could make the life of an inexperienced system administrator sour). And I have described PCRE match limit errors and what to do with them in greater detail. I think I settled this once and for all. I am a bit proud of that.

ModSecurity is extremely multipurpose. I felt like I knew maybe a third of the features when I started out. But rewriting the book allowed me to try it all out and to get it working on my own machine. That was rewarding. And outside of a couple of bugs with the engine, I also made surprising discoveries along the way. There is one I want to tell you about. It’s a bit technical, so please bear with me:

When working on the Core Rule Set, I encountered situations where we would do a match operation just to fill the MATCHED_VAR variable. And we know that matching operations have a cost. Thus, executing this on every request on every parameter is excessive. I thought about this problem for a long time. It bugged me. I remember one night dreaming about a way to fill MATCHED_VAR for free. I was ready to file a feature request with Felipe. But before I did, my eyes noted something unusual in the debug log: A reference to an unconditional match. That felt exactly like the operation I was looking for. Checking the source code, I discovered this hidden and hitherto undocumented gem: @unconditionalMatch. Here you see it in action counting occurrences of parameters:

Traditional way using a regular expression:

SecRule ARGS_NAMES "@rx ." \
"phase:2,\
id:2000,\
pass,\
nolog,\
setvar:'TX.paramcounter_%{MATCHED_VAR_NAME}=+1'"

Smart way leveraging @unconditionalMatch, saving many processor cycles:

SecRule ARGS_NAMES "@unconditionalMatch" \
"phase:2,\
id:2000,\
pass,\
nolog,\
setvar:'TX.paramcounter_%{MATCHED_VAR_NAME}=+1'"

Needless to say that @unconditionalMatch and a few undocumented peers are now covered in the reference part. I am drawing a lot of satisfaction from this completeness of the reference part of the book.

Another part that has been redone was the performance chapter. Web applications work differently from what they did six years ago. The performance information in the first edition no longer applies to the latest version. So by stress-testing all sorts of ModSecurity setups, I was able to come up with new numbers. They allow rule writers and system administrators to make the right call when developing rules in the future.

Last week, the publisher Feisty Duck has put my manuscript in his online shop as early access. You can now buy a PDF version of my text. That text will be updated continuously until it will go to print a some moment during the Winter. If you buy it now, you will get continued updates until the final version of the 2nd edition comes out in printed form. We are currently in the 2nd round of technical review. Afterwards, there will be copyediting and a lot more work by the publisher.

It goes without saying that buying the book now gets you the best price for the book. It will be more expensive when the book is finished. So if ModSecurity interests you, you should head over to Feisty Duck and secure your copy immediately.

Follow me on twitter to receive updates about the progress of the book.

 

Christian Folini