Click here to forward to a colleague

In this issue:
1. Improving RMI Stability and Reliability
2. Guest Spot: Lawson Web Applications Performance and Stability
3. Worthwhile Reading
4. Lawson Tips & Tricks

The LawsonGuru Letter is a free periodic newsletter providing provocative commentary on issues important to the Lawson Software community.  The LawsonGuru Letter is published by—and is solely the opinion of—John Henley of Decision Analytics.

Visit Decision Analytics at https://www.danalytics.com. For subscription information, see the bottom of this message.  The LawsonGuru Letter is not affiliated with Lawson Software.

When you try getting support if you run into an issue, you’ll find out the hard way that you really can’t do it without Lawson.  They're not making this easy.  Remember how Lawson encouraged you to use their consultants for your 8.0 upgrades?  “Use Lawson consulting or never get your project done.  If someone else installs your software you risk losing your support.”   Turn the clock back four years (!), and read “Will Lawson deny support for non-Lawson installed software?”  (see https://www.danalytics.com/guru/letter/archive/2002-09.htm).  Sound familiar? 

Now, where was I?  Oh yeah--you still have to deal with 8.0.3 RMI issues.  So, this month we'll delve into some of the ways in which you can improve RMI stability. 

Unfortunately, there doesn't seem to be any universal fix that you can make, but there are some ways you can improve RMI reliability and stability.  These tips are culled from various other consultants as well as my personal experience.  Alas, there is no silver bullet, and it takes a lot of iterative tuning and experimentation.  It's extremely painful.  If you're one of those clients who doesn't experience RMI-related issues, consider yourself lucky!  For the rest of you, here goes.  For the most part, these hints are meant for clients running RMI on Unix or Windows and using Tomcat as their servlet container.

	First and foremost, if you can schedule a very short downtime (assuming you're not running a 24x7 and/or global operation), take the pro-active step of a daily re-cycle of RMI, Tomcat, and your web server.  Remember that the shutdown (and more importantly, the startup) order are important: Startup RMI, then Tomcat, then your web server. Then ProcessFlow and BCI. Shutdown in the reverse order.  Having a quick nightly script that does this on a scheduled basis sure beats unscheduled downtime!
	Check, double-check, and then triple-check your .jar files.  Heck, check 'em four times! The jar files in GENDIR/java/jar need to match the ones deployed in WebSphere’s "ear" file, or in CATALINA_BASE/webapps/ROOT/WEB-INF/lib.  Likewise, the versions of the jar files in CATALINA_BASE/webapps/ROOT/WEB-INF/lib and CATALINA_BASE/webapps/ROOT/WEB-INF/classes should match as closely as possible.  You would be amazed how many clients I know who have missed those steps (as well as the step to remove the xerces.jar file) in the post-install instructions, and have 8.0.3 files mixed with 8.0.2 files.  Simply fixing those can make a world of difference.  Also, there are
	If you have a version 8.0.2 lawsonrt.jar on the IOSRE of an 8.0.3 IOS install, remove it. It is not automatically de-installed with the upgrade to 8.0.3.
	Tuning the RMI settings in PoolMgr.properties is an art.  For each connection, 14 MB of RAM is used, so that means for those 50 users, 140MB of RAM is required.  I'm told that this is flawed.  In reality there are 2 PTS processes, 2 streamDME’s and however many are streamIDA’s configured. The streamIDA’s increase with the Number of min and max are configured upwards in the PoolMgr.properties file.  Based on observation it is much higher and also the reason RMI becomes unresponsive many times and requires the restart/reboot.  Tune for the maximum expected load, and make the initial settings equal to ~25% of the maximum, e.g. if maximum is 10, set initial to 2.  The pts processes are declared globally across all product lines, but the streamida and streamdme processes are created specific to each product line. If you expect a peak load of 50 concurrent connections, this equates to 50 / 5, so you need to configure of 10 as the maximum number of concurrent connections.
	"Pooled DME" (aka streamdme) Although the settings are typically not set up when Lawson was installed, you can configure the initial and maximum number of DME connections (initconns_dme and maxconns_dme).
	See Lawson Knowledgebase article 938683 for OS-specific issues, patches and environment variables settings.
	Kept 100% current at all times on your Java releases.  If you're using 1.3.1, then it needs to be at the latest release of 1.3.1; if 1.4.2 then at the latest release of 1.4.2 and depending on the release all the available patches applied.  If you're lucky enough to be on AIX (which bundles Java with the OS patches), after you've applied the latest AIX maintenance release, apply the latest java fixes beyond the fix pack; they are available from the IBM patch site.
	Medium and larger clients need to pay particular attention to the "Xms###m" and "Xmx###m" (upper and lower Java Memory Stack) settings; adjust them higher for both lower and upper ends based on the configured connections in the PoolMgr.properties file.  Change them in the startps.sh in $CGIDIR/rmi on Unix or in startps.bat for Windows Systems. There are also registry updates that are listed in the Windows Registry for Windows installations.  The stable installations with large clients have Xmx and Xms set at 1000m (i.e. 1 GB of RAM min and max in the Java Stack).  Also make the adjustments in the logan.env configuration files.
	For larger implementations where a single Tomcat is handling the load for large numbers of users, there has to be an adjustment to the number of processes available for that one Tomcat in the CATALINA_BASE/conf/server.xml file. The default is 75 and it needs to be moved higher to 100 or 150 (or even higher based on the number of users that are actually accessing the system). This is necessary for large numbers of Lawson Portal and Requisition Self Service users.
	Currently, the most reliable supported RMI / servlet combination seems to be running IBM Websphere 6.0, which is now supported by 8.0.3 ESP7.
	Another way to achieve high reliability is by running IIS 6.0 and Tomcat 5.0.28 in a VMWare Image. (It's not supported but very reliable and makes the best use of available resources.) This way you can have FOUR Tomcats on 1 Server with Four IIS web server instances. It works great and makes complete use of the resources on that server. 4 CPU’s and 4 Images with 4 GB of RAM. All hooked to one Lawson instance. Needs to be load leveled via a network appliance.
	Most robust Tomcat installation on UNIX is a free-standing Apache 2.0.X web server and Tomcat 5.0.28, running on a Unix server by itself running remotely.

(by Alex Tsekhansky, Analysts International)

Test results

• Starting with, I believe, ESP4, where streamdme ("Pooled DME") was introduced, Lawson can sustain large number of concurrent users (I’ve tested up to 500) on Portal and self-service applications without failures of Lawson components. Lawson’s recent formula of non-concurrent-to-concurrent users’ ratio on one of the installations was 20:1. So, that translates to 10,000 non-concurrent users. More scientific approach I’ve developed was acquired during large transaction volume experiments related to SEA and benefit enrollment. There I came up with 30:1 or more formula instead (the actual number is based on the test, and is different every time). So, Lawson’s formula was even more conservative than the one I calculated.



• The failures usually appear first in non-Lawson components – if Lawson is configured correctly.



• The failures usually appear first in Tomcat. The Websphere testing did not include stress component that would find the first point of failure. On the Windows platform, the first failed component—providing configuration of all software is correct—is the ISAPI filter for Tomcat.  I used the ISAPI DLL from Tomcat 3.3 (128Kb in size).  I will do testing with a newer version of ISAPI DLL in the future with Tomcat 5.  Interestingly, Tomcat 4.1 ISAPI did not work with the other versions correctly.



• The servlets can easily sustain 4-times or more concurrent users than CGI-based calls. Hence, any custom modifications designed for large concurrent number of users must use servlets rather than CGI calls for DME, AGS, etc..



• Environment version 8.0.3 on UNIX and Windows handles prolonged servlet usage much better than 8.0.2. Installing ESP6 for 8.0.3 environment is highly recommended for stability reasons.



• Lawson web server gets about 5-10% of the CPU load, while the Lawson application server gets 80-85%. The rest is the database server (for SEA/Portal work without batch jobs and Crystal Reports).



• If majority of the calls use servlets, there is no need to keep databases in a shared access mode (e.g. setting up Oracle in a shared mode for large concurrent user numbers).



• One needs to tweak Lawson settings related to the number of concurrent LATM users for the most popular forms, and monitor form usage, so no more than 10 instances of the same form would be in memory. With the high volume of user switching form cleanup processes seem to get “stuck” sometimes if this tweak is not done.



• If you encounter non-Tomcat-related failures on the web side, most probably it’s related to either the authentication process, especially if you have a custom one, or a CGI DME call to LOBKMARK table during the initial login process. Then IE Windows Refresh will allow a user to continue. Providing configuration is correct, you will not encounter the DME-related issue until you will have more than 200 concurrent users (6000 non-concurrent), hence most people do not see it, even at the peak usage.

For years, one of the Lawson tips I’ve been sharing with users has been how to add a new column to an upload mapping you’d previously saved. This is often needed when you add a new column to a spreadsheet and want to include it in your upload.

Now, Lawson has added it. Download version 2.0.5 of the Lawson Add-Ins for Microsoft Office. Then, it’s just a click away!