Java out of heap space error with invoice report for some customers only.
G'day all,
I have been puzzling over this for some time but have gone around in circles. So its time for the experts!
On both v1.8 and v2.01, when printing an invoice for 1 client only ( I thought there were 1 or 2 others but no one can remember), the system sits with a "Please Wait", increases memory use to heap space limit (1G with system total of 4G), then increases CPU to 100% of both cores, with much swapping, then gives the heap space error. If running for a while since a tomcat restart, this process takes about 10 minutes, but after a fresh restart it took 1hr34min on v1.8, still with the same result.
This same report works fine for all other clients.
Excerpts from openvpms-full.log:
v2.01
23 Nov 2018 16:33:27,331 WARN ShortNamePairArchetypeHandlers,http-apr-8080-exec-4:235 - simons - Invalid archetype for handler=class org.openvpms.web.component.im.edit.identity.IdentityLayoutStrategy, short name=entityIdentity.insurer* from jar:file:/var/lib/tomcat/webapps/openvpms/WEB-INF/lib/openvpms-web-workspaces-2.0.1-SNAPSHOT.jar!/DefaultLayoutStrategyFactory.properties: ignoring
23 Nov 2018 17:11:33,790 ERROR JRFillSubreport,http-apr-8080-exec-5:878 - simons - Fill 1: exception
java.lang.OutOfMemoryError: GC overhead limit exceeded
at Invoice32Reminders32A5_1542951269138_772767.evaluate(Invoice32Reminders32A5_1542951269138_772767:190)
at net.sf.jasperreports.engine.fill.JREvaluator.evaluate(JREvaluator.java:277)
at net.sf.jasperreports.engine.fill.JRCalculator.evaluate(JRCalculator.java:618)
at net.sf.jasperreports.engine.fill.JRCalculator.initializeVariable(JRCalculator.java:490)
at net.sf.jasperreports.engine.fill.JRCalculator.initializeVariables(JRCalculator.java:313)
at net.sf.jasperreports.engine.fill.JRVerticalFiller.fillReportStart(JRVerticalFiller.java:241)
at net.sf.jasperreports.engine.fill.JRVerticalFiller.fillReport(JRVerticalFiller.java:119)
at net.sf.jasperreports.engine.fill.JRBaseFiller.fill(JRBaseFiller.java:558)
at net.sf.jasperreports.engine.fill.BaseReportFiller.fill(BaseReportFiller.java:414)
at net.sf.jasperreports.engine.fill.JRFillSubreport.fillSubreport(JRFillSubreport.java:746)
at net.sf.jasperreports.engine.fill.JRSubreportRunnable.run(JRSubreportRunnable.java:58)
at net.sf.jasperreports.engine.fill.AbstractThreadSubreportRunner.run(AbstractThreadSubreportRunner.java:216)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
23 Nov 2018 17:11:36,378 ERROR ErrorHelper,http-apr-8080-exec-5:175 - simons - Failed to generate report pvA5Invoice.jrxml: java.lang.OutOfMemoryError: GC overhead limit exceeded
org.openvpms.report.ReportException: Failed to generate report pvA5Invoice.jrxml: java.lang.OutOfMemoryError: GC overhead limit exceeded
v1.8
Caused by: java.lang.OutOfMemoryError: Java heap space
at sun.text.UCompactIntArray.initPlane(UCompactIntArray.java:160)
at sun.text.UCompactIntArray.setElementAt(UCompactIntArray.java:73)
at java.text.RBTableBuilder.addOrder(RBTableBuilder.java:342)
at java.text.RBTableBuilder.build(RBTableBuilder.java:154)
at java.text.RBCollationTables.<init>(RBCollationTables.java:83)
at java.text.RuleBasedCollator.<init>(RuleBasedCollator.java:300)
at java.text.Collator.getInstance(Collator.java:268)
at net.sf.jasperreports.engine.fill.DataSourceComparator.<init>(DatasetSortUtil.java:234)
at net.sf.jasperreports.engine.fill.DatasetSortUtil.getSortedDataSource(DatasetSortUtil.java:121)
at net.sf.jasperreports.engine.fill.JRFillDataset.initDatasource(JRFillDataset.java:698)
at net.sf.jasperreports.engine.fill.JRBaseFiller.setParameters(JRBaseFiller.java:1314)
at net.sf.jasperreports.engine.fill.JRBaseFiller.fill(JRBaseFiller.java:931)
at net.sf.jasperreports.engine.fill.JRBaseFiller.fill(JRBaseFiller.java:892)
at net.sf.jasperreports.engine.fill.JRFillSubreport.fillSubreport(JRFillSubreport.java:669)
at net.sf.jasperreports.engine.fill.JRSubreportRunnable.run(JRSubreportRunnable.java:59)
at net.sf.jasperreports.engine.fill.AbstractThreadSubreportRunner.run(AbstractThreadSubreportRunner.java:203)
... 3 more
Increasing heap space (https://openvpms.org/forum/busy-threads#comment-6693) does not seem to affect things using -Xmn=1g. It is v1.8 report that is also used in v2.01.
1/ Is there any other way to permanently change heap space? ie JAVA_OPTS vs CATALINA_OPTS
2/ Is there something wrong in the (sub) report that only comes into play with certain records?
3/ Could there be corrupt data with that client?
Any ideas most welcome.
Re: Java out of heap space error with invoice report for ...
It could be an issue with the invoice JasperReport template. If a report band can't fit on a page, but is not set to split, it can trigger an infinite loop. Are you using the standard templates?
Re: Java out of heap space error with invoice report for ...
Hi Tim,
Yes the column header and footer didn't have splitType set in the notes sub-report, but after setting this to Stretch, the error still occurs with this 1 customer, over all (randomly selected) invoices tested in a wide date range.
To speed the process up, I set heap size to small values; ie export JAVA_OPTS="-Xms256m -Xmx512m" in /usr/share/tomcat/bin/setenv.sh . After re-reading the man page I realized that using -Xmn was probably wrong (as in the original post). However, jmap -heap seems to show that more than that given in the options is being used.
On the side issue of java memory, do you prefer -Xms and -Xmx to be set the same on a production server? Could -XX:MaxGCPauseMillis=<nnn> or -XX:GCTimeRatio=<nnn> be used and let java vary the heap to suit?
Any other thoughts on getting out of the loop?
Simon Slater
Registered Linux User #463789 @ http://linuxcounter.net
Re: Java out of heap space error with invoice report for ...
From the "Invoice32Reminders32A5_1542951269138_772767" the problem is probably in the Invoice Reminders template. This was updated for 2.0 so you could try loading that version.
Regarding setting Xms and Xmx the same, this is covered here:
Re: Java out of heap space error with invoice report for ...
Tim,
Those links clarify what I had found on Oracle, Stackoverflow and others, thanks.
I can see the differences in the v2 invoices and will load the default templates later.
However, the v1.8 Invoice report generates the reminder sub-report fine on all but 1 client. I checked it with a few more customers to be sure. We are still using v1.8 on the production server, hence the problem, because they are a frequent customer.
Simon Slater
Registered Linux User #463789 @ http://linuxcounter.net
Re: Java out of heap space error with invoice report for ...
Hi Tim,
Loaded up the default invoice templates into v2 and all works well there. Still stumped as to the problem in v1.8. Any particular phrase to look for in the logs?
Simon Slater
Registered Linux User #463789 @ http://linuxcounter.net
Re: Java out of heap space error with invoice report for ...
You mean that in 2.0 the customer now prints successfully, but in 1.8 it doesn't?
You could try going through each of of the bands in the invoice and its subreports and change the Split Type to Immediate.
Re: Java out of heap space error with invoice report for ...
Thanks Tim, that's fixed it!
Before reading your last reply, I had reverted to an earlier version of the invoice jrxml to check and it worked fine.
Changing splitType to Immediate for all bands has got the new version working, as you suggested.
I had made template changes back in July when we started using a different paper size with a different printer and it was printing single page invoices back-to-front through the duplexer compared to multi-page invoices. What those changes were must have caused the problem. It took a while to pin it down to the invoice and that customer because I would only be told "the system started going slow". I'll go through the changes made back then to see what caused it, so I don't do it again.
Thanks again.
Simon Slater
Registered Linux User #463789 @ http://linuxcounter.net
Re: Java out of heap space error with invoice report for ...
I think I found the problem: isPrintWhenDetailOverflows="true" for the band. When A4 size, everything fitted, but when moving to A5 size, it caused the infinite loop.
Simon Slater
Registered Linux User #463789 @ http://linuxcounter.net
Re: Java out of heap space error with invoice report for ...
If you have an A4 JasperReport template want in A5, there is a tool to scale these in the bin/ directory:
> scale usage: usage: scale [options] <input> [output] -A4 scale to A4 -A5 scale to A5 -f scale fonts -LETTER scale to US-LETTER -m scale margins -o overwrite existing output -r <arg> replace values from properties file -v verbose output input may be a single .jrxml file, or a directory output may be a single .jrxml file, or a directory, or unspecified. If unspecified, the scaled version will be output to the consoleThis was introduced in the 1.9 release.
It is used to generate the A5 and US-Letter versions of the standard A4 templates. E.g.