Alfresco Monitoring with JavaMelody

JavaMelodyInspired by Toni de la Fuente (@toniblyxpost about (Alfresco) server monitoring using JavaMelody, I decided to replicate the effort.

There are several ways to monitor your server, with different purposes and related tools. Nagios or Inca could be used to manage infrastructure and applications on a range of parameters that can be measured. Think disk space, memory usage, availability of protocol API’s. These are central monitoring applications probing the environment.

There are specific (SaaS based) application monitoring tools like New-Relic (against Alfresco) and Scout. They integrate with your application server like Tomcat and can determine how the processing time is shared over transport, real calculations, database activity etc. This break-up of the numbers can be of value when looking at a performance problem. however the SaaS concept can be a problem for some of my customers.

Next to that there are tools one can use from the server itself like VisualVM and JConsole. They can display the JMX-like values like the memory usage, disk space, running processes etc. You often need to run the tools at the server, or have enough ports open to access it remotely. Often I just cannot.

My intention of use of JavaMelody is in the play field of New-Relic; I need to check performance and point to a bottle neck. I tried New Relic before, but a commercial SaaS like solution is not always viable. This blog (in Spanish, try the English), and this screenshot in particular made me try JavaMelody. (This seems to be something the psi-probe tool does not do?) If the time spent at  SQL queries can be determined in the context of a single http call, then I am fixed. And using 2 locally added jars instead of outside connections.

I guess, the world has changed a little bit since Toni’s blog. I downloaded the latest javamelody-1.42.0.zip from Google Code. From the inside of this zip, copy javamelody.jar and jrobin.1.5.9.1.jar into tomcat/webapps/alfresco/WEB-INF/lib.

I did not get the plain JDBC route working for Alfresco.  I can get it to work using JNDI only. So, run Alfresco using JNDI to locate your JDBC resources. Follow the next 3 steps and enjoy the insights!

1. Open your tomat/webapps/alfresco/WEB-INF/web.xml. Look for the context-param contextConfigLocation.  Tweak this element to look like:

<context-param>
  <param-name>contextConfigLocation</param-name>
  <param-value>
    classpath:net/bull/javamelody/monitoring-spring-datasource.xml
    /WEB-INF/web-application-context.xml
  </param-value>
  <description>Spring config file location</description>
</context-param>

Add a new section:

<filter>
   <filter-name>monitoring</filter-name>
   <filter-class>net.bull.javamelody.MonitoringFilter</filter-class>
</filter>
<filter-mapping>
   <filter-name>monitoring</filter-name>
   <url-pattern>/*</url-pattern>
</filter-mapping>
<listener>
   <listener-class>net.bull.javamelody.SessionListener</listener-class>
</listener>

2. Edit the two following files (keep the content identical!)

  • tomcat/conf/Catalina/localhost/alfresco.xml and
  • tomcat/webapps/alfresco/META-INF/context.xml

Uncomment the first resource element and adjust the username/password/url accordingly. make sure the name attribute has the value jdbc/dataSource (that is the name Alfresco expects)

3. Update your tomcat/shared/classes/alfresco/extension/alfresco-global.properies. I made the db.username (and/or db.password) invalid. This to prevent Alfresco using the direct JDBC database connection (failing your statistics).

(Re)Start Alfresco, and off you go. Point your webbrowser to http://localhost:8080/alfresco/monitoring and enjoy!

I got it working!

Monitoring-Alfresco-JavaMelody

The SQL time measured is the time from invoking the statement from JDBC until there is a resultset. It does not include the time to fetch the data for a select, nor does it include the time for jdbc ResultSet.next().

If you got this working, consider modifying your alfresco.bat/alfresco.sh in here there is the definitionof JAVA_OPTS. Add to this definition (in the same line, no line breaks):

-Djavamelody.monitoring-path=/faces/jsp/admin/monitoring -Djavamelody.storage-directory=c:/bin/myAlfresco/alf_data/javamelody-repo

What is does?

  • The part ‘monitoring-path=’ moves the URL of the monitoring page to http://localhost:8080/alfresco/faces/jsp/admin/monitoring, and you get Alfresco admin-only protection for free!
  • The second part (‘javamelody.storage=’) moves the location where JavaMelody stores its measured history from Tomcat’s temp folder to a more reasonable location. I collect it in alf_data/javamelody-repo (and alf_data/javamelody-share for my separated Tomcat-Share instance). Makes life easy when making your nightly backup 🙂

Wait a little for a next post on how to use the collect server. This is a little work to configure this securely.

Apart from this you get a lot more! Among others:

  • historical insight in your memory usage,
  • System and http errors (and details)
  • List of threads (connected to SQL statements)  frequency, time of execution, deviation…
  • All sessions
  • Sql statements, frequency, time of execution, deviation…
  • Overview of the garbage collection (even manually invoke if you feel you need that)
  • your full JMX details (MBeans) (Hey, I ve seen this working, it is broken in my install right now)
  • All OS processes
  • Java (jar) dependencies
  • Jobs
  • Caches
  • The full set of JVM JAVA_OPTS and (in my case) a lot of MySQL settings

monitoring goodies

Monitoring goodies

I really appreciate the sortable table columns (click the headers) :0)

(Cool Tool! Anyone got this working on the direct JDBC definition?)

[update 12-12-2012: removed javamelody-objectfactory.jar because not needed (triggered by Emeric Vernat, lead of JavaMelody)]

[update 14-12-2012: added what’s in/out the SQL measured time (triggered by discussion with Emeric Vernat). Also added additional configuration of JAVA_OPTS to move the datafiles and to change the URL]

Advertisements

14 Responses to “Alfresco Monitoring with JavaMelody”


  1. 1 Peter Monks December 12, 2012 at 16:13

    That’s very cool! I wonder if this could be packaged as an AMP, for easier deployment / configuration? I can’t see any reason why not!

    • 2 Tjarda Peelen December 12, 2012 at 21:56

      Hi Peter,

      Thanks for your response! Good trigger. First I thought most of the configuration is enabling Alfresco JNDI (e.g. username/password/database name stuff). After some spinning I can see some meaningful additional configuration (like where to store the measurement results). On my todo-list… Suggestions how to deal with the credentials/db name issue are welcome!

      • 3 Peter Monks December 12, 2012 at 23:51

        hmmm……..yeah JNDI is a pain from a change management perspective. Can JavaMelody be wired up some other way? If so it might be possible to dependency inject the Alfresco JDBC objects directly.

        • 4 Tjarda Peelen December 13, 2012 at 00:28

          I was under the impression that this was one of the ‘features’ of JNDI & appservers; decouple server specific environment (db credentials) from the application 🙂
          Had a quick look. I am too little of a Spring guru I am afraid. Or finally suffering from too little sleep. Not for tonight.

  2. 5 Toni December 13, 2012 at 19:57

    Thanks for mention, great article!

  3. 6 Andreas Amstutz (@javaconsulting) December 29, 2012 at 22:54

    Great article! Got it working on ubuntu. Next step is to install it on a tomcat instance on win2008 so that I hopefully can trace the cause of frequent service shutdowns, which lead to manual restarts of the alfresco service as the customer is not using any form of monitoring.

    • 7 Tjarda Peelen December 29, 2012 at 23:52

      Good to hear it seems to help you out. Let me know if it solved your problem! I guess you are focussing on memory behaviour…

      • 8 Andreas Amstutz (@javaconsulting) January 2, 2013 at 22:20

        Now, after three days, javamelody shows me more than 30k open jdbc-connections. So I guess some of the custom background cleanup jobs are not closing the connections properly. That’s kind of strange, as I actually had focus on that topic. The memory is quite stable – around 400mb – so it seems these connections do not occupy a lot of memory. Going to dig in the code for the obvious now. And btw a happy new year to you 😉

        • 9 Tjarda Peelen January 2, 2013 at 23:28

          Happy New Year to you too!

          Open connections are not bad per definition. Remind the jndi/jdbc settings that Tomcat owns can specify the max number of connections (maxActive=”100″). It helps of course if this matches the DB connection pool of your database, as well as (in alfresco-global.properties) db.pool.max=100 and db.pool.idle=-1. The -1 means that connections are not closed, but reused when needed (resulting in increased performance, yes, with a little penalty, but hey, what’s exactly for free these days…)

  4. 10 Carl Nordenfelt March 6, 2013 at 10:18

    Great article! It was a charm to set it up on a Windows 2008 server and MS SQL db. Now I got a lot of data to analyze 🙂

    One thing I noticed was that the initial results show a very high time consumption for SQL queries compared to Tomcat server cpu (about 85/15). A possible reason could be that the caches in Alfresco is not warm from startup. Do you know if it is possible to reset the JavaMelody data from after the web app is tarted, let’s say after a day when the cache state should be more “normal”?

    • 11 Tjarda Peelen March 7, 2013 at 21:26

      Hi Carl,

      Good to hear you like this post. And nice you got it running against MSSQL.

      Did you consider using the Customized report?
      Customized report

      You can specify a different start/end date. If your system is running for multiple day’s it fits your case. I am not aware how to get rid of the first hour only. (Also because my systems are cleaned because of redeployments w/h JavaMelody).

      Let me know if you succeed!

      Tjarda

      • 12 Carl Nordenfelt March 12, 2013 at 15:17

        Thanks, I also found the custom reports and it works fine. And the results were the same 🙂

  5. 13 Antonio April 4, 2013 at 22:04

    Hi, great post !

    I plugged it (javamelody 1.4.4) into an alfresco community installation (4.2.c) and got summary info and reports working, but the charts …

    =(

    Any idea of what could be the cause to this problem?

    thanks in advance

    • 14 Tjarda Peelen April 5, 2013 at 00:11

      Hi Antonio,

      That’s weird. Does the account running tomcat (I assume) have write permissions where JavaMelody wants to store data? (-Djavamelody.storage= in your JAVA_OPTS). You have put both javamelody.jar and jrobin.1.5.9.1.jar into WEB-INF/lib??!?

      Let me know if you succeed (and how)!


Comments are currently closed.