Large Documents in File System

If you have questions or if you want to share your opinion about Aware IM post your message on this forum
Post Reply
customaware
Posts: 2413
Joined: Mon Jul 02, 2012 12:24 am
Location: Ulaanbaatar, Mongolia

Large Documents in File System

Post by customaware »

I can understand some limitation for the max size of a document that is stored in the database. But I would have expected there to be less limitation when stored in the File System.
I need to upload documents to a max file size of 256MB.

I have set max_allowed_packets in MySQL to 268,435,456 (256M) successfully but Aware still complains.

Is there anywhere else I need to up the max value to allow the upload?

or alternatively.... if anyone has experience uploading large files like this.... is there a trick that I am missing?
Cheers,
Mark
_________________
AwareIM 6.0, 8.7, 8.8, 9.0 , MariaDB, Windows 10, Ubuntu Linux. Theme: Default, Browser: Arc
Upcloud, Obsidian....
Image
BenHayat
Posts: 2749
Joined: Thu Dec 23, 2010 5:48 am
Location: Fla, USA
Contact:

Re: Large Documents in File System

Post by BenHayat »

customaware
Posts: 2413
Joined: Mon Jul 02, 2012 12:24 am
Location: Ulaanbaatar, Mongolia

Re: Large Documents in File System

Post by customaware »

Thanx Ben but already tried that. No go
Cheers,
Mark
_________________
AwareIM 6.0, 8.7, 8.8, 9.0 , MariaDB, Windows 10, Ubuntu Linux. Theme: Default, Browser: Arc
Upcloud, Obsidian....
Image
Powerm
Posts: 473
Joined: Mon Feb 01, 2010 9:44 pm
Contact:

Re: Large Documents in File System

Post by Powerm »

I prefer to upload attachments to Dropbox or to my local NAS and copy/paste the link to my app...
Independent Developer
www.atwing.com
aware_support
Posts: 7526
Joined: Sun Apr 24, 2005 12:36 am
Contact:

Re: Large Documents in File System

Post by aware_support »

Code: Select all

I have set max_allowed_packets in MySQL to 268,435,456 (256M) successfully but Aware still complains.
What exactly does Aware IM say?
Aware IM Support Team
kklosson
Posts: 1629
Joined: Sun Nov 23, 2008 3:19 pm
Location: Virginia

Re: Large Documents in File System

Post by kklosson »

I think max allowed packets is about 2GB and I've had to set it there. I don't use the file system, though...
V8.8
MySQL, AWS EC2, S3
PDFtk Toolkit
customaware
Posts: 2413
Joined: Mon Jul 02, 2012 12:24 am
Location: Ulaanbaatar, Mongolia

Re: Large Documents in File System

Post by customaware »

aware_support wrote:

Code: Select all

I have set max_allowed_packets in MySQL to 268,435,456 (256M) successfully but Aware still complains.
What exactly does Aware IM say?
Testing uploading a 60mb file.....

I select the file and the uploading progress bar completes.... then I click Create and it never closes. Need to restart the Aware IM service.
bigfileresponse.png
bigfileresponse.png (70.5 KiB) Viewed 47408 times
I can send the wrapper.log if it will help.
Cheers,
Mark
_________________
AwareIM 6.0, 8.7, 8.8, 9.0 , MariaDB, Windows 10, Ubuntu Linux. Theme: Default, Browser: Arc
Upcloud, Obsidian....
Image
idpSteve
Posts: 201
Joined: Thu Jul 27, 2017 6:13 am
Location: Johannesburg, South Africa
Contact:

Re: Large Documents in File System

Post by idpSteve »

I'm having the same issue. Hoping for a solution!
aware_support
Posts: 7526
Joined: Sun Apr 24, 2005 12:36 am
Contact:

Re: Large Documents in File System

Post by aware_support »

As Ben said you MUST do what is described in this thread:
https://www.awareim.com/forum/viewtopic ... 495#p44495

Don't forget to restart Aware IM after you make changes to the BASServer.props file

If you are still having this problem check the output of the server and Tomcat. If you see "Out of memory there" increase memory for the appropriate component.
Aware IM Support Team
RLJB
Posts: 914
Joined: Tue Jan 05, 2010 10:16 am
Location: Sydney, Australia

Re: Large Documents in File System

Post by RLJB »

Hey mark - it definitely works for >256mb. We built a Digital Asset Mgmt module, and tested the heck out of it, big is fine. No effect on other users either which was our initial concern. Will ask Rob to check if he sets anything else.
Rod. Aware 8.6 (latest build), Developer Edition, on OS Linux (Ubuntu) using GUI hosted on AWS EC2, MYSQL on AWS RDS
customaware
Posts: 2413
Joined: Mon Jul 02, 2012 12:24 am
Location: Ulaanbaatar, Mongolia

Re: Large Documents in File System

Post by customaware »

Well... I get mixed results.

I have JMS_BROKER_MEMORY=300000000

When I upload a 254 MB file I see the progress bar working to 100% but when I click ok, system hangs.
When I upload a 55 MB file, all good. Progress Bar to 100% and clikcing ok is fine.

However,

Which I click the Document Control on the Form it views (or downloads) ok.
BUT, if I call it in a Process using DISPLAY Document, I get this error....
Erro.png
Erro.png (29.09 KiB) Viewed 46797 times
Note that the file is saved to the File System. Not the DB.
Cheers,
Mark
_________________
AwareIM 6.0, 8.7, 8.8, 9.0 , MariaDB, Windows 10, Ubuntu Linux. Theme: Default, Browser: Arc
Upcloud, Obsidian....
Image
aware_support
Posts: 7526
Joined: Sun Apr 24, 2005 12:36 am
Contact:

Re: Large Documents in File System

Post by aware_support »

Change the type of the LONGDATA column in the execution_contexts table from MEDIUMBLOB to LARGEBLOB (depending on the database you are using the column types may be called differently)
Aware IM Support Team
customaware
Posts: 2413
Joined: Mon Jul 02, 2012 12:24 am
Location: Ulaanbaatar, Mongolia

Re: Large Documents in File System

Post by customaware »

Still problems

After uploading (progress bar 100%) clicking the create button, the screen locks up with the Progress Mask.
Screen Shot 2017-11-13 at 8.19.21 am.png
Screen Shot 2017-11-13 at 8.19.21 am.png (79.79 KiB) Viewed 46773 times
Cheers,
Mark
_________________
AwareIM 6.0, 8.7, 8.8, 9.0 , MariaDB, Windows 10, Ubuntu Linux. Theme: Default, Browser: Arc
Upcloud, Obsidian....
Image
ACDC
Posts: 1146
Joined: Sat Jun 30, 2007 5:03 pm
Location: California, USA

Re: Large Documents in File System

Post by ACDC »

This is a wild guess, based on the error you posted, I had a similar problem some time back. It had to do with a MySql limitation which I can't put words to right now - try rename the file to a shorter file name, if that doesn't work try placing the document attribute on a reference object and see if u get the same problem
aware_support
Posts: 7526
Joined: Sun Apr 24, 2005 12:36 am
Contact:

Re: Large Documents in File System

Post by aware_support »

Have you checked the output of Tomcat and the Aware IM Server when you get a freeze?
Aware IM Support Team
Post Reply