Re: 5.4 uploading large files
Re: 5.4 uploading large files
- Subject: Re: 5.4 uploading large files
- From: Ramsey Gurley <email@hidden>
- Date: Tue, 19 Jun 2012 08:24:03 -0700
Rather than modify the old WOAdaptor, an alternative option is to write a WOAdaptor. ERWOAdaptor is already there, and it's based on netty. Personally, I consider that a better way forward, but right now, it is also limited in file size because it uses the chunked decoder.
I believe it could be updated to support large file sizes, but 1) I haven't had the time to try and 2) I haven't had the problem :-) I was more interested in making websockets work when I was working on it last year. I recently updated the websocket implementation to support the latest spec. I didn't have any luck finding a large file upload example while I was doing it, so it's still limited to 100Mb uploads.
Ramsey
On Jun 19, 2012, at 2:30 AM, Johann Werner wrote:
> Apparently no one has tried to modify/override the involved classes yet so I made some additions to Wonder by reimplementing the smallest possible number of classes to support upload streams > 2.1GB. These are
>
> WOHttpIO
> WOMultipartIterator
> WONoCopyPushbackInputStream
>
> leaving NSRange as it is as that class is used in many places and changing the API from int to long would involve many dependent fixes. For those wanting to try out these changes and help in testing/modifying the code get the corresponding wonder branch [1]. I attached an Eclipse project that can be used to test the upload. Just pick a big file or create one with
>
> dd if=/dev/zero of=big_file.bin bs=1g count=3
>
> (this will create a 3GB file 'big_file.bin')
>
> Some observations I made:
>
> - using DirectConnect everything works as expected
> - Firefox cannot correctly upload files > 2.1GB [2] as the variable used for the content-length header is an int (doh!), the header length is added to the file length so the file cannot be exactly (2^32 - 1)
> - passing through Apache the upload stream is interrupted sometimes -> timeout/problem within mod_WebObjects?
>
>
> Please share your thoughts
>
> jw
>
>
>
> [1] https://github.com/darkv/wonder/tree/big_files_upload
> [2] https://bugzilla.mozilla.org/show_bug.cgi?id=215450
>
> <UploadDownloadTest.zip>
>
>
> Am 14.06.2012 um 18:55 schrieb Helmut Tschemernjak:
>
>>
>> Downloading large files works fine, for uploads there are several bugs in the WO code. I reported it via the Apple bug reporter some time ago. Problem ID: 10765546
>>
>> There is no workaround, here is some input in case somebody from Apple is looking into it:
>>
>> The class com.webobjects.appserver._private.WOHttpIO tries to parse the content-length header with the method parseInt of the Integer class. This will fail for all files larger than 2.1GB. Further there are many instances in the code that make use of an integer for the content length of a request:
>> WOInputStreamData: public WOInputStreamData(InputStream, int)
>> WORequest: public int _contentLengthHeader()
>> Also classes not directly involved in the request handling do show such a limitation (e.g. public NSRange(int, int)).
>>
>> PS: looking forward to see many of you at WOWODC 2012 in Montreal
>>
>> Regards
>>
>> Helmut Tschemernjak
>>
>>
>> On 13.06.12 09:45, Johann Werner wrote:
>>> Just hitting the same problem for uploading files > 5GB. Does anyone has already done some work relating to that problem?
>>>
>>>
>>> Am 05.03.2012 um 00:51 schrieb Q:
>>>
>>>> On 05/03/2012, at 9:40 AM, Lachlan Deck wrote:
>>>>
>>>>> Hey Alan,
>>>>>
>>>>> On 05/03/2012, at 9:32 AM, Alan Ward wrote:
>>>>>
>>>>>> Speaking for myself (and definitely not on behalf of my employer) I'm curious where you are coming from here? Did you have a contract that obligated Apple to continue to support you? What grounds to you believe you have for a law suit? I'm just curious.
>>>>> Yes it's a somewhat comical notion :-)
>>>>>
>>>>> Just to make your point/question clearer for people: the said product was supplied for free for the last few releases. So if people were to sue for damages, it would need to be a very strong case indeed! I'm sure Apple would be happy to supply a full refund ;-)
>>>>>
>>>>> More seriously, I think the sentiment is coming from promises (or assumed promises) that were made when, for example, the Xcode tools were dropped. The reasoning given was to concentrate on enhancing the frameworks. Thus the initial impression given to the community by Apple (bolstered by its funding of tools like WOLips) was that a renewed energy was being put into WO by Apple that people could continue to build their business apps with. And, there was some initial promise of better community involvement in the evolution of WO when the Apple maven repo was made available (which showed some future versions in the works). It didn't last long though which was a shame at the time, especially for us maven users. Hmm so, perhaps it was the community's unwillingness to embrace maven that canned it moving forward :-)
>>>> What are you trying to say? Maven killed WebObjects? Why am I not surprised. :)
>>>>
>>>>> cheers,
>>>>> Lachlan Deck
>>>
>
>
> _______________________________________________
> Do not post admin requests to the list. They will be ignored.
> Webobjects-dev mailing list (email@hidden)
> Help/Unsubscribe/Update your Subscription:
>
> This email sent to email@hidden
_______________________________________________
Do not post admin requests to the list. They will be ignored.
Webobjects-dev mailing list (email@hidden)
Help/Unsubscribe/Update your Subscription:
This email sent to email@hidden