Maintaining servers falls into two phases: First, bang head until server works; second, capture effort into some automation tool like Puppet or Chef. Recently, I’ve been playing around with making the first phase closer to the second. For lack of a better word, I’m calling it literate devops.
Update 1: I finally got around to troubleshooting using Tramp references to execute code blocks on remote servers (even those behind firewards).
Update 2: Need another complete example? Check out my notes onsetting up IP Tables (and this org-mode file), where part of the file can be executed in the editor in order to see how my machines are configured, and the other part is a script that can be tangled to a machine and executed to reset to the firewall rules.