Problems when uploading pdf with too many pages(?)

If you think that something doesn't work in Aware IM post your message here
Post Reply
joben
Posts: 221
Joined: Wed Nov 06, 2019 9:49 pm
Location: Sweden
Contact:

Problems when uploading pdf with too many pages(?)

Post by joben »

I have a document attribute without size limitations.
I noticed a problem when uploading PDF files that contains many pages:
Skärmbild 2021-05-25 180341.png
Skärmbild 2021-05-25 180341.png (2.58 KiB) Viewed 22294 times
"Internal error, Java heap space"

You can try it yourself with this document as an example:
https://www.lopp.net/pdf/princeton_bitcoin_book.pdf

308 pages, around 18 MB.

Have had similar errors with these bigger PDF files, like: "unknown server error".

In this scenario, we don't do any magic with the document once it's uploaded. So I'm pretty sure either the file size, page size or some weird meta data in the PDF makes Aware IM go nuts. Any ideas?

Version 8.4 Build 2713.
Regards, Joakim

Image
intra
Posts: 279
Joined: Thu Oct 11, 2012 1:30 pm
Location: Australia

Re: Problems when uploading pdf with too many pages(?)

Post by intra »

How much memory do you have allocated to the AIM server and Tomcat?
Avid Linux user....
joben
Posts: 221
Joined: Wed Nov 06, 2019 9:49 pm
Location: Sweden
Contact:

Re: Problems when uploading pdf with too many pages(?)

Post by joben »

The server itself has 8 GB RAM. We could increase this easily if necessary.

About Tomcat, I don't know.
I can't see anything in the server.xml file.
Regards, Joakim

Image
intra
Posts: 279
Joined: Thu Oct 11, 2012 1:30 pm
Location: Australia

Re: Problems when uploading pdf with too many pages(?)

Post by intra »

try startupoptions.props.
Avid Linux user....
joben
Posts: 221
Joined: Wed Nov 06, 2019 9:49 pm
Location: Sweden
Contact:

Re: Problems when uploading pdf with too many pages(?)

Post by joben »

Thanks!

We havn't touched this file so the values should be default:

Code: Select all

#Thu Jun 27 15:39:42 CEST 2019
CONFIG_TOOL_STARTUP=..\\JDK\\bin\\java -classpath ../JDK/lib/tools.jar;../docs/help.jar;../lib/* com.bas.uiconfiguration.Application
AWAREIM_SERVER_STARTUP=..\\JDK\\bin\\java -Xmx192m -Xms20m -classpath ../lib/*;../CustomJars/* com.bas.newcp.ServerStarterECP
TOMCAT_STARTUP=..\\JDK\\bin\\java -Xmx64m -Djava.endorsed.dirs\=../Tomcat/endorsed -classpath ../lib/tools.jar;../Tomcat/bin/bootstrap.jar;../Tomcat/bin/tomcat-juli.jar -Dcatalina.base\=../Tomcat -Dcatalina.home\=../Tomcat -Djava.io.tmpdir\=../Tomcat/temp org.apache.catalina.startup.Bootstrap start
Will try tampering with the values later today and report back :D
Regards, Joakim

Image
intra
Posts: 279
Joined: Thu Oct 11, 2012 1:30 pm
Location: Australia

Re: Problems when uploading pdf with too many pages(?)

Post by intra »

Gee, performance must be terrible!

I would at least do this.

<snip>
AWAREIM_SERVER_STARTUP=..\\JDK\\bin\\java -Xmx4096m -Xms4096m -classpath ../lib/*;../CustomJars/* com.bas.newcp.ServerStarterECP
TOMCAT_STARTUP=..\\JDK\\bin\\java -Xmx2048m -Xms2048m -Djava.endorsed.dirs\=../Tomcat/endorsed -classpath ../lib/tools.jar;../Tomcat/bin/bootstrap.jar;../Tomcat/bin/tomcat-juli.jar -Dcatalina.base\=../Tomcat -Dcatalina.home\=../Tomcat -Djava.io.tmpdir\=../Tomcat/temp org.apache.catalina.startup.Bootstrap start
....
</snip>

The memory settings can actually be setup from the control panel, so have a look in there if your GUI based.
Avid Linux user....
joben
Posts: 221
Joined: Wed Nov 06, 2019 9:49 pm
Location: Sweden
Contact:

Re: Problems when uploading pdf with too many pages(?)

Post by joben »

AwareIM is installed as a service.

I did changes to startupoptions.props and restarted the AwareIM and Tomcat service.
I don't think this had any effect at all.
The PDF errors persist, and a quick benchmark of some queries shows that the performance is the same.

Is it possible to see the Tomcat running config?
Regards, Joakim

Image
BobK
Posts: 544
Joined: Thu Jan 31, 2008 2:14 pm
Location: Cincinnati, Ohio, USA

Re: Problems when uploading pdf with too many pages(?)

Post by BobK »

when running AwareIM as a service...
aware_support wrote: Wed May 12, 2021 2:15 am In two files, which are identical - only one is used, but I do not remember which one, so make the change in both.

The files are wrapper.conf and awareim.conf in the BIN directory.

This is the property in these files that sets memory:
wrapper.java.maxmemory=256
I believe the file that needs changing is awareim.conf
Bob
kklosson
Posts: 1617
Joined: Sun Nov 23, 2008 3:19 pm
Location: Virginia

Re: Problems when uploading pdf with too many pages(?)

Post by kklosson »

Yes, I allocate as follows in both wrapper.conf files.

# Initial Java Heap Size (in MB)
wrapper.java.initmemory=1024

# Maximum Java Heap Size (in MB)
wrapper.java.maxmemory=4096

Note that these files (or one of them) are only read when running as a service as they provide params for the Tanutki Wrapper, which supports running as a service. But I had a long-running issue of large files uploading and crashing the heap. I eventually set the file size limit to 5MB, which works for my purposes. There must be a way to upload much larger files. If you figure it out, please share.
V8.8
MySQL, AWS EC2, S3
PDFtk Toolkit
Post Reply