sending selenium logs to logstash

I’ve been playing around with the sending various logfiles to logstash and then on to elasticsearch to see whether I can generate any useful metrics from all the bits of testing that I oversee — selenium infrastructure being one of those bits.  Rather than output to a logfile and have a logstash agent ship changes […]

I’ve been playing around with the sending various logfiles to logstash and then on to elasticsearch to see whether I can generate any useful metrics from all the bits of testing that I oversee — selenium infrastructure being one of those bits.  Rather than output to a logfile and have a logstash agent ship changes to a logfile, I wanted to try the log4j socketappender approach.  I saw a few examples out on the web, but nothing that worked end-to-end, so here goes:

First, start up your logstash server with something like this as an input:

input {
    log4j {
      format => "plain"                                                        
      port => 9392                   
      type => "log4j"                
    }
}

Now you want to send selenium logs there. Normally, you’d just start selenium something like this:

java -jar seljars/selenium-server-standalone-2.29.0.jar -role hub

But I wanted to use slf4j so that I could control logging. Plus, I wanted to do so without opening up the selenium jar. To use the -cp option or any other method of setting the classpath I can’t use the -jar option. I end up with this:

java  -Djava.util.logging.config.file=logging.properties -cp \
selenium-server-standalone-2.29.0.jar\
:jul-to-slf4j-1.7.2.jar\
:slf4j-api-1.7.2.jar\
:log4j-1.2.17.jar\
:slf4j-log4j12-1.7.2.jar\
:. org.openqa.grid.selenium.GridLauncher -role hub

It’s not the prettiest looking thing, but you can pretty up the classpath bit if you want. This example pretends all these jars are in the same directory just to try and keep things looking a little less cluttered. Note that I have :. in the classpath so that it finds my log4j.properties file. Which brings me to the final two pieces of configuration:

log4j.properties:

# Root logger option
log4j.rootLogger=INFO, stdout, logstash
 
# Direct log messages to stdout
log4j.appender.stdout=org.apache.log4j.ConsoleAppender
log4j.appender.stdout.Target=System.out
log4j.appender.stdout.layout=org.apache.log4j.PatternLayout
log4j.appender.stdout.layout.ConversionPattern=%d{yyyy-MM-dd HH:mm:ss} %-5p %c{1}:%L - %m%n

# logstash socketappender
log4j.appender.logstash=org.apache.log4j.net.SocketAppender
log4j.appender.logstash.Port=9392
log4j.appender.logstash.RemoteHost=localhost
log4j.appender.logstash.ReconnectionDelay=10000

logging.properties

handlers= org.slf4j.bridge.SLF4JBridgeHandler
.level= ALL

Clearly, this is just all happening on the same host within my example, but just change the hosts and ports to suit your situation. You don’t need to direct messages to stdout in the log4j.properties file, either, I just left that in so that you could see things happening when you start up selenium and then verify that the same stuff was showing up over on the logstash side of things.

To get the jars I used:

selenium
slf4j
log4j

Copyright information