Monitoring a JVM in Grid Control

As you might know, Oracle Enterprise Manager Grid Control lets you monitor and manage Oracle databases, listeners, instances and things like that as expected, but you can also monitor and manage other systems and software like JVM.

Let's say that you want to monitor a custom service written in Java:

root@myserver # ps ax|grep mycustomserver
 8945 ?        Sl    11:04 /opt/mycustomserver/server -JVMargs /opt/mycustomserver/server.ini -n mycustomserver
14810 pts/2    S+     0:00 grep mycustomserver

You can do it by means of JMX, setting JVM's jmxremote parameters as arguments or in a configuration file. You have to set at least jmxremote.port, there is no default value for it:

root@myserver # cat /opt/mycustomserver/server.ini|grep jmxremote
JVMOPTS3=-Dcom.sun.management.jmxremote.port=2055
JVMOPTS4=-Dcom.sun.management.jmxremote.authenticate=false
JVMOPTS5=-Dcom.sun.management.jmxremote.ssl=false

And if you don't know the application but want to try to monitor it, you can try as root to guess the monitoring port number with netstat:

root@myserver # netstat -tuanp|grep 8945|grep LISTEN
tcp        0      0 0.0.0.0:2055                0.0.0.0:*                   LISTEN      8945/mycustomserver
tcp        0      0 127.0.0.1:1852              0.0.0.0:*                   LISTEN      8945/mycustomserver

With the remote port number, and in case authentication is turned on with the corresponding user and password, you can add your Java application to Grid Control clicking in the agent link in Targets -> All Targets tab, and in the next page select JVM from the Add drop-down menu. In the Monitoring Configuration page you have to set at least the Machine name (would be 127.0.0.1) and the Admin Port number fields, and click the OK button.

After adding your JVM properly to Oracle Enterprise Manager, you will be able to check in the All Metrics link a lot of interesting metrics, at least interesting if you have knowledge of Java.

More information:

Monitoring and Management Using JMX
Monitoring and Management Using JMX Technology

Installing an Enterprise Manager agent for a TimesTen database

This is a small howto about installing an Enterprise Manager 11g agent for a TimesTen 11 database; this howto assumes that you have already downloaded the agent in the client machine and configured the Management Plug-in for Oracle TimesTen In-Memory Database in your Enterprise Manager server.

First, you have to configure a response file to do a silent installation; an installation with OUI should be easy enough to not have to cover it here. In the response directory below your agent staging directory you should find an additional_agent.rsp file, copy it and change at least this fields:

timesten@myserver $ diff additional_agent.rsp my.rsp
41,42c41,42
< SECURITY_UPDATES_VIA_MYORACLESUPPORT=
< DECLINE_SECURITY_UPDATES=
---
> SECURITY_UPDATES_VIA_MYORACLESUPPORT=FALSE
> DECLINE_SECURITY_UPDATES=TRUE
85c85
< ORACLE_AGENT_HOME_LOCATION=
---
> ORACLE_AGENT_HOME_LOCATION=/myagent
169,170c169,170
< OMS_HOST=
< OMS_PORT=
---
> OMS_HOST=myoemserver
> OMS_PORT=4889

Next, launch runInstaller with the full path of your response file:

timesten@myserver $ cd /myagent/linux_x64/agent
timesten@myserver $ ./runInstaller -silent -responseFile /myagent/linux_x64/response/my.rsp

This way you will install an agent in /myagent directory; don't forget to run as root the root.sh configuration script after finishing runInstaller:

root@myserver # /myagent/agent11g/root.sh

If you want to check if the agent is running and communicating with the Grid Control server, run emctl and pay attention to the Last successful upload message:

timesten@myserver $ cd /myagent/agent11g/bin/
timesten@myserver $ ./emctl status agent
Oracle Enterprise Manager 11g Release 1 Grid Control 11.1.0.1.0 
Copyright (c) 1996, 2010 Oracle Corporation.  All rights reserved.
---------------------------------------------------------------
Agent Version     : 11.1.0.1.0
OMS Version       : 11.1.0.1.0
Protocol Version  : 11.1.0.0.0
Agent Home        : /myagent/agent11g
Agent binaries    : /myagent/agent11g
Agent Process ID  : 5493
Parent Process ID : 14892
Agent URL         : http://myserver:3872/emd/main/
Repository URL    : http://myoemserver:4889/em/upload/
Started at        : 2011-11-29 09:48:12
Started by user   : timesten
Last Reload       : 2011-11-29 11:50:40
Last successful upload                       : 2011-11-29 12:53:25
Total Megabytes of XML files uploaded so far :    13.84
Number of XML files pending upload           :        0
Size of XML files pending upload(MB)         :     0.00
Available disk space on upload filesystem    :    66.57%
Last successful heartbeat to OMS             : 2011-11-29 12:53:29
---------------------------------------------------------------
Agent is Running and Ready

If you need to change the configuration of the agent for some reason then stop the agent, edit the emd.properties and targets.xml files, and optionally delete previous files:

timesten@myserver $ cd /myagent/agent11g/bin
timesten@myserver $ ./emctl stop agent
Oracle Enterprise Manager 11g Release 1 Grid Control 11.1.0.1.0 
Copyright (c) 1996, 2010 Oracle Corporation.  All rights reserved.
Stopping agent ... stopped.
timesten@myserver $ cd ..
timesten@myserver $ vi sysman/config/emd.properties
timesten@myserver $ vi sysman/emd/targets.xml
timesten@myserver $ cd sysman/emd
timesten@myserver $ rm -rf recv/* protocol.ini lastupld.xml collection/* state/* cputrack/* agntstmp.txt upload/* ../log/* emagent_storage.config core* sqlnet.log *sql
timesten@myserver $ cd /myagent/agent11g/bin
timesten@myserver $ ./emctl start agent
Oracle Enterprise Manager 11g Release 1 Grid Control 11.1.0.1.0 
Copyright (c) 1996, 2010 Oracle Corporation.  All rights reserved.
Starting agent ..... started.

After these steps you have to be able to see the host and its agent in Grid Control at Targets tab, All Targets option. But in order to add a new TimesTen database to Enterprise Manager you have to first set the agent's password at Preferences -> Preferred Credentials -> Agent's Set Credentials icon; in that page you have to find your new agent and fill in the Host Username and Host Password fields. Next you will go to the Setup -> Management Plug-ins page and click the deploy icon of TimesTen management plug-in, in the next page add the new agent and click the Next button, and finally the Finish button in the last page.

Now you'll be able to add the TimesTen database clicking in the agent link in Targets page; it might be a delay to appear the TimesTen In Memory Database 11g option in the Add drop-down menu so take this into account. In that page you have to select the TimesTen In Memory Database 11g option from the Add menu, and then fill in the TimesTen instance name, Data Source Name, TimesTen user name and TimesTen password fields. Finally, click the Test Connection button and the OK button to complete the operation.

And if you're new to TimesTen and have no idea what the instance name and data source name would be, but the TimesTen user has the TT_HOME environment variable set, there is only one database at the server, and you know the username and password's database, you could guess TimesTen instance name and data source name this way:

timesten@myserver $ basename $TT_HOME
tt1121
timesten@myserver $ ttstatus | grep Data
Data store /TimesTen/DataStore/MyDataStore/MyDataStore

More information:

Meeting Hardware Requirements
Meeting Package, Kernel Parameter, and Library Requirements

About the RHCSA certification

From time to time there are people that ask me how I got the OCP certification and I regret not having posted about it, therefore now that I'm a shiny, brand-new Red Hat Certified System Administrator I would like to write about this.

First of all, this is a practical exam so if you enjoy taking multiple choice quizzes this is not the case. You can check all the information about this test in the Red Hat Certified System Administrator page. Something worth of reading it is the RHCSA Exam Objectives page, mandatory if you choose to take the test without enrolling into Red Hat recommended courses. By the way, if you are unsure if you have the level required for passing the test without enrolling into a course, it would be a good idea to take the pre-assessment quiz; you have to fill in a form with personal information in order to take the quiz, but if you take the certification test you'll have to give it anyway.

I prefer a lot to study a book and practice with my home computers, and for this test I studied the RHCE Red Hat Certified Engineer Linux Study Guide, by Michael Jang. Talking about other certification book, I read some time ago a review that blasted that book because there were some things that weren't covered in the book that were in the test, and unfortunately this is the case for this book too. But taking into account that no book can just put the exact test with its answers and that at this time I don't know a guide for studying for the new Red Hat Enterprise Linux 6 exams, I think that Jang's book is a good one for preparing for the RHCSA exam. I recommend reading the whole book and PRACTICING its exercises, even if this book is intended to be a guide for the (now obsolete) RHCT and RHCE exams; if you study the RHCE part you'll be a lot more confident while taking the test than if you just study the RHCT part.

Finally, as any good certification test, it's aimed to check your skills and knowledge about a concrete field, therefore having experience in this topics will help a lot and would be decisive in passing the test; if you don't have experience in some topics try at lest to practice it a lot before taking the test.

I think that holding a certification is a good way to back your skills and studying for the test is a good opportunity to keep up to date your knowledge, therefore I wish good luck for everybody that decide to take the RHCSA test.

OUI in a Linux Itanium

As you might know, runInstaller is the script you run for launching OUI for applying patches and installing Oracle, and usually you don't have to care about it unless you're running it in an Itanium server.

Lets say that you want to apply some patch:

oracle@myserver$ ./runInstaller 
Starting Oracle Universal Installer...

Checking installer requirements...

Checking operating system version: must be redhat-3, SuSE-9, SuSE-10, SuSE-11, redhat-4, redhat-5, UnitedLinux-1.0, asianux-1, asianux-2 or asianux-3
                                      Passed


All installer requirements met.

Preparing to launch Oracle Universal Installer from /tmp/OraInstall2011-11-18_12-09-55PM. Please wait ...
oracle@myserver$ 

And after launching runInstaller you wait forever for the Oracle Universal Installer and it never shows, and in an eureka moment you decide to check the java distributed with the patch:

oracle@myserver$ java -version
java version "1.4.2"
gcj (GCC) 3.4.6 20060404 (Red Hat 3.4.6-9)
Copyright (C) 2006 Free Software Foundation, Inc.
This is free software; see the source for copying conditions.  There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.

oracle@myserver$ /tmp/OraInstall2011-11-18_12-09-55PM/jre/1.4.2/bin/java -version

Segmentation fault

As documented in Metalink note 391844.1, there is a bug in the JDK version bundled with the installer so you need to install another java distribution. Metalink note 400227.1 lists different methods according to your Oracle version and operating system so for example, in case of installing Oracle patch 10.2.0.4 in a Linux Itanium server, you have to download patch 5390722 and after installing it, you can launch OUI more or less this way:

./runInstaller -jreLoc $ORACLE_HOME/jre

More information:

Oracle 10g R2 Installation Problem