Large File Uploads [was Re: Large volume file upload design question ]
Large File Uploads [was Re: Large volume file upload design question ]
- Subject: Large File Uploads [was Re: Large volume file upload design question ]
- From: "Jonathan Fleming" <email@hidden>
- Date: Mon, 21 Jul 2003 17:53:55 +0100
Calling All Guru's
How can I... or can I even, link a long upload in with a LongRequestPage
(I've never used this feature, so ecuse me if it does not do what I am
thinking).
While testing on the fly conversions of image sizes using an external
program (ImageMagick), I find no problems with the external process and it's
ability to convert even 100mg files on the fly, but obviously I don't want
some crank to come along and test the apps vulnerabilities by accidently or
on purpose try to upload a 100mg file for conversion on the web.
1) First, if possible, I'd like to get the size of the file from the input
Stream before anything begins to get uploaded. How is it done or can't it be
done?
2) Armed with that info I'm hoping to tell the LongRequestReponse page to
kick in if the data is more than a particular length or simply attach it's
process to the length full stop.
In testing Apple's WebObjects example of the LongRequestReponse page I found
that if you try to cancel by using the browser's stop (x) button it seems to
safguard you against hangs and outOf Memory errors and other related
problems you fall into when a user decides they want out in the middle of a
process. Whereas doing a straight upload, if the user wants out, they just
press stop and that's exactly what happens everything stops. I imagine your
instance hangs as I have not yet tested in deployment, but in development
all functions of the browser hang and the app has to be rebuilt to get it
going again... worrying to say the least. I need to overcome this problem
badly. Who can help.
3) Once finished I want to return to the upload form (how do you get the
stream to inform you that it has finished its processing so you can tell the
LongResponse page this?)
Kind Regards
Jonathan :^)
Below are some interesting point Jonathan Rochkind made in at post called:
Re: Large volume file upload design question
I never saw any replies to this so if someone is willing, I'm waiting.
I think. Using WO 5.2's streaming upload functionality, for instance by
binding the WOFileUpload "outputStream" to a FileOutputStream, should work
fine for any size file. Now, you are still using HTTP to upload the
file---some people have suggested that FTP's error detection/correction
routines are preferable to HTTP for large files, I don't know enough about
the technical protocols to comment really. But as far as HTTP goes, using
5.2's streaming features should work fine for arbitrarily large files.
Of course, if the file upload takes a long time, and your application
instance does not handle requests concurrently (this is the default, and
it's something of a pain to do otherwise), the app instance will block until
the file is uploaded. That is, other users making HTTP requests will have
their browsers waiting until the first users upload is complete (Someone
correct me if I'm wrong here? We could also test this somewhat easily
ourselves). This is kind of a potential problem, but I don't know what to do
about it.
Wouldn't it be possible to somehow measure the amount of data that passes
the stream and use that in a WOLongResponse page?
Yes, it would be theoretically possible. But it's not obvious (to me) how to
make it work. There are a couple things that need to be solved. One is that
ordinarily the _browser_ is not going to display _anything_ until the
request it is submitting is complete---that is, until the file upload is
complete. Solving this would probably require some javascript to somehow
submit the request with the upload to one (invisible?) frame, while
displaying the progress information in the main frame. But then you run up
against a web objects limitation----without concurrent request handling
enabled (again, it usually isn't and it's somewhat of a pain to implement
it), your WO app is not going to respond to requests for 'give me the
progress of the file upload' until the upload is complete!
There was a brief discussion about this before on the list, and some people
thought it shouldn't be _too_ hard to fix this by writing your own
WORequestHandler for the streaming file upload. I haven't looked into this;
it's probably do-able for the experienced WO developer, but probably not for
the beginner.
But if someone DID solve this, it would have the wonderful side effect (if
done properly) of solving the problem with the "app instance blocks until
upload is complete" problem discussed above. Which would result in a really
excellent file upload system, as good as you could ask for if you are using
HTTP as the transfer protocol.
In a post From : Elizabeth Lynch <email@hidden>
Subject : java.lang.OutOfMemory error on file uploads
Elizabeth stated that she had an occasional problem when uploading and
downloading large files via a WebObjects app running under W2K.
The end users upload and download files of anything from 15MB to 85MB in
size. These are occasionally .gif images but more usually zip files (JF --
in my case jpg, gif, png's). Smaller files (2 - 10MB) upload and download
fine. But sometimes - not always - the uploads and downloads fail
immediately or after a couple of minutes with a:
Error: com.webobjects.foundation.NSForwardException
[java.lang.OutOfMemoryError] null
Reason:
Stack trace: File Line# Method Package
NA : Non applicable, JIT activated (JF -- same problem if user hit stop(x),
except no stacktrace was given)
Elizabeth' questions were:
* Is this probably a problem with the Java VM min and max heap size
parameters? If not, what else might it be?
* What would sensible settings for NSJavaMaxHeapSize and NSJavaMinHeapSize
be? NSJavaMinHeapSize is not currently set. NSJavaMaxHeapSize was set
at 100000000 and increasing it to 200000000 made no apparent difference.
What's the maximum we could set it to and what are the downsides of
increasing it?
(JF -- what is the JavaHeapSize? Is it where you use the Java command-line
option -Xmx to change the maximum RAM usage?)
Configuration:
------------------
WebObjects 5.2 updated to latest patch
Windows 2K, 1GB RAM
Java 1.3.1
All suggestions welcomed.
Jonathan
_________________________________________________________________
Stay in touch with absent friends - get MSN Messenger
http://www.msn.co.uk/messenger
_______________________________________________
webobjects-dev mailing list | email@hidden
Help/Unsubscribe/Archives: http://www.lists.apple.com/mailman/listinfo/webobjects-dev
Do not post admin requests to the list. They will be ignored.