File upload question

[pbrad]

Hi,

I am wondering whether it might be possible to have the option to choose whether a document upload is stored in the database as a blob(current method) or stored in a file based system with AwareIM tracking the name and path of the files so as to reduce the size of the database. We have several instances where the size of the table behind a BO is several hundred megabytes because of the blob fields.

I suppose that we could use a third party upload utility and access that through an html cell on a form but it would be nice to handle it directly through AwareIM.

thanks,
Pete

[aware_support]

Thanks for this idea - something for our TODO list.

[pbrad]

Sounds great, I notice in the EXT release notes that in version 2.2.0 which I presume you are now using they have added:

"Added FileUploadField extension and sample"

For my puposes, there is no reason that a file stored asset needs to be treated as an AwareIM document, an additional simple file upload field with the ability to allow users to download the large uploaded file is all that I am looking for.

Cheers,
Pete

[pbrad]

Hi again,

I found a working sample on the ext site:

http://extjs.com/deploy/dev/examples/fo ... pload.html

Pete

[tford]

I would definitely use this also.

Tom

[pixerp]

Sorry, if I interfere but I have a question :

where do you want to upload your file to ?

I can't see any difference between the extjs object you point at and what is currently available in AwareIM : the selected file to upload goes into an attribute of type Document.

Isn't it what you want to avoid ?

When I read your 1st post, I'd thought you'd like to find a file in a shared filesystem and store the path + file name in a plaintext attribute.

Thus I would be possible to define an HTML field in a form like :

<a href='file:///<<BO.filename>>'>See the document</a>

As I mentionned in a separate thread :
http://www.awareim.com/forum/viewtopic.php?p=7685#7685

we have developped a java plugin that enables the use of addtional functions in a proces rule :

. GET_UNIQUE_FILE
used to define a unique filename and the shared path of the document stored in a document attribute (enables EXPORT and IMPORT action from a process rule)

. FIND_FILE
used to find an external file in a shared path using a search string (with *) defined in a process rule and store the filename and the path in a plaintext attribute

. CHMOD
to modify the access rights on linux (or samba share) and Mac platform

. DELETE_FILE
to delete a file when its name and path is stored in an attribute

I've had pm's with tford about these and I'm preparing a small document with screenshots. If you're interested...

Thanks

PS - As Tom suggested, I need to mention that it will not be free. But maybe, Aware_support is also interested !

[pbrad]

Hi,

I don't think that this should require the purchase of a third party java plugin to implement. Currently uploaded documents are stored as blobs in the database and this drastically increases database overheads.

I am looking for a simple option of storing the uploaded item in a file system rather than in the database and being able to allow users to download it. Print ready artwork would be one example of a large file that would be applicable.

Unless I am mistaken, your solution requires a third party ftp action to get the files onto the server in the first place and I would like to handle this function within the AwareIM form functionality. (or perhaps this is handled by your java plugin).

It looks to me that EXT may have an existing solution that hopefully can be incorporated into a future version.

Cheers,
Pete

[pixerp]

Hi,

I don't think that this should require the purchase of a third party java plugin to implement. Currently uploaded documents are stored as blobs in the database and this drastically increases database overheads.

I am looking for a simple option of storing the uploaded item in a file system rather than in the database and being able to allow users to download it. Print ready artwork would be one example of a large file that would be applicable.

Unless I am mistaken, your solution requires a third party ftp action to get the files onto the server in the first place and I would like to handle this function within the AwareIM form functionality. (or perhaps this is handled by your java plugin).

It looks to me that EXT may have an existing solution that hopefully can be incorporated into a future version.

Cheers,
Pete

Hi Pete,

I see it's never easy to undersand somebody's requirement...

Our GET_UNIQUE_FILE function has nothing to do with ftp. In fact, it should be named SET_UNIQUE_FILE. It is used to DEFINE a unique filename for the EXPORT DOCUMENT action. This is required if you want to be able to IMPORT DOCUMENT back into the blob attribute after it has been modified. So really, this is not your requirement.

As I understand, you want to be able to REFERENCE, in a BO plaintext attribute, a file stored in an external file system, don't you ?

The point is accessing the filesystem from the application server side AND from the client side. It means you have to define the paths and manage the access rights. This is exactly what we have done.

One of our implementation is with our telephone system. Briefly, all phonecalls are recorded and stored on a Linux filesystem.

With the FIND_FILE function, we can find the relevant phonecall record and store the filename (+ path) into a BO OutgoingCall.Record plaintext attribute. The file stays on the external filesystem.

Now, on the OutgoingCall form, we have an HTML link "Listen Callrecord" that a user can click... Obviously, on the same form, we have "buttons" to a LinkCallrecord process and RemoveLink process.

In conclusion, I'm not trying to sell you anything. I'd rather have our functions added to AwareIM for the sake of eyerybody !?

Cheers,
Ernest

[mraath]

Hi
I have the following error on trying to save a larger document to a mysql 5 database.
Internal error. Error persisting business object MainDocument Packet for query is too large (28847989 > 16776192). You can change this value on the server by setting the max_allowed_packet variable.
My first question is where does this variabe get set, so i can increase the packet size.
Second question is how can i save files to the server file system instead of to the database.

Thanks

Hi,

I don't think that this should require the purchase of a third party java plugin to implement. Currently uploaded documents are stored as blobs in the database and this drastically increases database overheads.

I am looking for a simple option of storing the uploaded item in a file system rather than in the database and being able to allow users to download it. Print ready artwork would be one example of a large file that would be applicable.

Unless I am mistaken, your solution requires a third party ftp action to get the files onto the server in the first place and I would like to handle this function within the AwareIM form functionality. (or perhaps this is handled by your java plugin).

It looks to me that EXT may have an existing solution that hopefully can be incorporated into a future version.

Cheers,
Pete

[aware_support]

Regarding the first question - you need to check your MySQL documentation on how to do this. It is a MySQL problem, not Aware IM.

Regarding the second question - yes, you can save documents in the file system rather than in the database, but you have to do it manually, Aware IM will not do it for you automatically at this stage. You need to define an attribute that would store the file path to the document rather than the document itself.