Solaris 11: How to zone Opengrok (GF)

This is an example, how we setup opengrok using Glassfish. To setup the zone we use the zinstall helper library, which lets us focus on the meat and avoid all the boilerplate coding (see Minimize your zones). Here we use src as your zonename, hostname for the zone, as well as name of the glassfish domain. If you prefer another name, just change it as you like and make sure to rename related files and properties as well. For the zone setup script it is required, that the zone's hostname is resolvable via the hosts database (see getent hosts src). If you wondering about the + below, it is expected to by a symbolic link in /usr/bin to ./pfexec if the "Primary Administrator" profile is available, to ./sudo otherwise.

Setup the zone (ca. 5-10 min)

In the global zone:

#0 download the latest glassfish image
	cd /tmp
	wget http://dlc.sun.com.edgesuite.net/glassfish/3.1.2/promoted/latest-glassfish-unix-ml.sh
#1 download the zonehelper archive and extract it where you like. E.g.:
	cd /usr/share; tar vxzf /tmp/zintall.tgz; cd zinstall
#2 optional: rename src-zonesetup.sh to $hostname-zonesetup.sh
#3 Adjust/set values, which might not be [correctly] automatically deduced from
#  the current environment. See ./src-zonesetup.ksh -v
#4 optional: cp files/ipf.conf@src files/ipf.conf@$hostname and adjust as you
#  need. Can be done later as well within the zone. See /var/log/ipmon.log 
#5 finally create, install and reboot the zone
	+ ./src-zonesetup.ksh -i

Everything else in the non-global zone as user admin. You may login using ssh or zlogin -l admin src

Setup a Glassfish Instance aka Domain

# check properties for proper values (see etc/gfadm -h)
	vim gf.properties src/domain.properties
# create the GF instance
	+ etc/gfadm -c domain -d src
# create the SMF service
	+ etc/gfadm -c service -d src
# enable the service
	svcadm enable glassfish/src
# check: svcs -xv ; browse http://src/

# To access the admin console, you might add an ssh tunnel to the host you are
# working on because per default a) remote access is disabled and b) ipfilter
# blocks the related ports. E.g.:
	ssh -f -x -N -L 10000:localhost:4848 src
# Now you should be able to access it via http[s]://localhost:10000/
# and change required parameters (e.g. enable debugging and the like).
 
# NOTE: If you are using binaries, which require libraries outside the default
# locations (see 'crle' and 'crle -64'), you should either add the required
# directories to the default locations. E.g.:
#	crle -c /var/ld/ld.config -l /lib/64:/usr/lib/64:/local/usr/lib/64
# or - much better - inform glassfish about it. E.g.:
#	+ gsed -i -e \
#		'/java-config/ s,">," native-library-path-suffix="/local/usr/lib/64">,' \
#		/data/glassfish/v3.1.2/src/config/domain.xml
#	svcadm restart glassfish/src

Setup opengrok

# enable external javadb
	svcadm enable glassfish/javadb
# save subversion data close to opengrok data - not in the home of the user
	mkdir /data/opengrok/.subversion
# we use a ZFS so that we do not need a full copy of stage1 when doing
# our nightly updates in stage2 (we just clone/update/promote/rename/destroy)
	+ zfs create -p -o compression=on -o atime=off pool1/data/opengrok/stage1/src
	+ zfs create -o compression=off -o atime=off pool1/data/opengrok/stage1/data
	+ chown -R admin:staff /data/opengrok/stage1
	cd /data/opengrok/stage1
# check out all repositories
	cd /data/opengrok/stage1/src/
	# if one need a password, and it is a weak one, store it
	svn --username myname --config-dir /data/opengrok/.subversion \
		checkout svn://src.doma.in/group/project/trunk project
# download opengrok
	curl -o /tmp/opengrok-0.11.tar.gz \
		http://hub.opensolaris.org/bin/download/Project+opengrok/files/opengrok-0.11.tar.gz
# expand it in a directory of your choice - /opt seems to be appropriate
	cd /opt
	+ tar xpzof /tmp/opengrok-0.11.tar.gz
	+ ln -s ./opengrok-0.11 opengrok
# we will use a slightly different cmd script, which allows one to store the
# important properties in a different file --> reduce the noise, no need
# to change the script for every opengrok installation
	+ ln -s ~/src/opengrok opengrok/bin/opengrok
	+ chmod 444 opengrok/bin/OpenGrok
# also we need the derby DB client library
	+ ln -s /opt/glassfish/v3.1.2/javadb/lib/derbyclient.jar \
		opengrok/lib/lib/derbyclient.jar
# since we need to adjust the opengrok library depending on the settings
# we need to make it writable. E.g.:
	+ chgrp staff opengrok/lib/opengrok.jar
	+ chmod g+w opengrok/lib/opengrok.jar
# finally check, whether your need to adjust the OpenGrok environment
# see also: /opt/opengrok/bin/opengrok vars
	vim ~/src/opengrok.rc

# Since the application deployment task also adjusts the java pathes in the
# opengrok.jar, we deploy the web application now:
	/opt/opengrok/bin/opengrok -c ~/src/opengrok.rc deploy

# Now listing all repositories opengrok recognizes should work:
	/opt/opengrok/bin/opengrok -c $HOME/src/opengrok.rc list
# and finally indexing as well
	/opt/opengrok/bin/opengrok -c ~/src/opengrok.rc -v -u update

Update

A very simple, space saving variant to update concurrently is using ZFS clone features. However, if one uses an external JDBC based history cache (i.e. neither file based nor OpenGrok embedded), one needs a second DB instance, so that current requests do not interfere with the "new" stage related data. To do that, first we create the filesystem, where the 2nd DB instance lives:

	+ zfs create -o compression=on -o atime=off -o mountpoint=/data/javadb2 \
		rpool/data/javadb2
	+ chown noaccess /data/javadb2
# verify
	zfs list rpool/data/javadb2

Now we create a new SMF javadb service instance for it, so that it starts automatically on "unattended" [re]boot, when it is enabled:

	svccfg -f - <<EOF
select glassfish/javadb
add opengrok2
select opengrok2
# need to overwrite some settings of the default config - so create the group
addpg config application
# where the 2nd instance lives
setprop config/datadir = "/data/javadb2"
# a different port (default 1527) is needed to run both instance concurrently
setprop config/jvmargs = ( "-Dderby.drda.portNumber=1528" )
# make SMF happy
addpg general framework
setprop general/complete = boolean: true
refresh
EOF

# verify the service
	svcprop glassfish/javadb:opengrok2

# verify that it can be started/stopped on demand, e.g.:
	svcadm enable javadb:opengrok2
	netstat -a -f inet | grep 1528
	svcadm disable javadb:opengrok2
	netstat -a -f inet | grep 1528

# and finally destroy the filesystem (since we clone the running one on demand)
	+ zfs destroy rpool/data/javadb2

Finally create an appropriate config file, which should be used, when the 2nd Opengrok instance/DB needs to be updated:

	gsed -e 's,/stage1/,/stage2/,' ~/src/opengrok.rc > ~/src/opengrok2.rc
	echo 'OG_DERBY_URL="jdbc:derby://localhost:1528/cachedb;create=true"' \
		>> ~/src/opengrok2.rc

Now we can use the following scheme to update the repositories incl. DB on demand, while OpenGrok still serves from the "old" stage area and DB until the update is complete:

# clone the current stage (here stage1)
	+ zfs snapshot -r pool1/data/opengrok/stage1@renew
	+ zfs clone pool1/data/opengrok/stage1@renew pool1/data/opengrok/stage2
	+ zfs clone -o compression=on -o atime=off \
		pool1/data/opengrok/stage1/src@renew pool1/data/opengrok/stage2/src
	+ zfs clone -o compression=off -o atime=off \
		pool1/data/opengrok/stage1/data@renew pool1/data/opengrok/stage2/data
# clone the external DB (no need to stop since used in a read-only manner)
	+ zfs snapshot rpool/data/javadb@renew
	+ zfs clone -o compression=on -o atime=off -o mountpoint=/data/javadb2 \
		rpool/data/javadb@renew rpool/data/javadb2
# update repos in the cloned stageing area
	cd /data/opengrok/stage2/src
	cd project
	svn --config-dir /data/opengrok/.subversion update
	...
# start the alternate DB
	svcadm enable javadb:opengrok2
# re-index (use opengrok2.rc for stage2, opengrok.rc for stage1)
	/opt/opengrok/bin/opengrok -c ~/src/opengrok2.rc -v update
# stop the old DB (here :default, otherwise :opengrok2)
	svcadm disable javadb:default
# make sure, nobody accesses the old stage and delete it (here stage1)
	cd
	+ zfs promote pool1/data/opengrok/stage2
	+ zfs promote pool1/data/opengrok/stage2/src
	+ zfs promote pool1/data/opengrok/stage2/data
	+ zfs destroy -r pool1/data/opengrok/stage1
	+ zfs destroy -r pool1/data/opengrok/stage2@renew
	+ zfs promote rpool/data/javadb2
	+ zfs destroy rpool/data/javadb
	+ zfs destroy rpool/data/javadb@renew

Misc

If you want to replace certain files in the web application archive source.war automatically, you can create a directory named replace in the same directory of your opengrok.rc file and put in all the stuff you wanna add to or overwrite in the archive. NOTE, that this directory must have the same structure as the web application archive. You may check this using the command unzip -t /opt/opengrok/lib/source.war. So if you wanna change the styles used by opengrok, one could do:

mkdir /tmp/orig
cd /tmp/orig
unzip /opt/opengrok/lib/source.war
mkdir ~/src/replace
find . -type f -name "*.css" -print | cpio -puvmd ~/src/replace/
cd ~/src/replace/
# now edit the files you want and finally re-deploy the web application. E.g.:
# find ~/src/replace/ -type f -name "*.css" -exec \
#	gsed -i -e 's/: *monospace/:"Liberation Mono",monospace/g' \
#		-e 's/: *sans-serif/:"Liberation Sans",sans-serif/g' {} +
/opt/opengrok/bin/opengrok -c ~/src/opengrok.rc deploy
# cleanup
rm -rf /tmp/orig

Furthermore it is possible to add an html fragment to the:

• header of each page (header_include)
• body of the index page (body_include)
• footer of each page (footer_include)

Opengrok expects these files in its data directory (e.g. /data/opengrok/stage1/data/). However, you can also put them into the local config directory (~/src/) and the opengrok script takes care of copying it to the configured data directory of Opengrok.

Glassfish tuning

Certain settings may improve the performance of your glassfish instance. One of these are the following JVM settings, which might be set using the admin GUI (https://localhost:4848/) via "Common Tasks" | "Configurations" | "server-config" | "JVM settings":

• -Dcom.sun.grizzly.displayConfiguration=true
• -Dcom.sun.grizzly.autoConfigure=true
• -Dcom.sun.grizzly.useDirectByteBuffer=true
• -Xmx1024m

Also it is possible to enable compression and file caching. For this choose "Common Tasks" | "Configurations" | "server-config" | "Network Config" | "Network Listeners" | "http-listener-1" and select on the right side "HTTP". Here one may set "Compression" to "on" and set "Compressible Mime Types" to "text/html,text/xml,text/plain,application/json,text/javascript,text/css". Last but not least one may enable file caching by selecting the appropriate option within the "File Cache" tab.

Finally, if one needs apache httpd like access logs, it can be enabled via "Common Tasks" | "Configurations" | "server-config" | "HTTP Service" by checking the "Access Logging" option on the right side of the page.

Copyright (C) 2012 Jens Elkner (jel+s11@cs.uni-magdeburg.de)