Large upload over HTTP

Upload core product.
Post Reply
User avatar
support
Posts: 1503
Joined: Sun Jan 27, 2008 6:19 pm

Large upload over HTTP

Post by support »

Large upload over HTTP could be a pain depending on your web server technology and settings.
Basically browser relies on HTTP POST multipart to upload files to web server. It works fine for hundred MB file size if the server-side software is able to to handle large POST request.
But known problems are:
- PHP server are usually setup to allow 8MB upload max size. It can be updated if admin or hosting service allows it (in php.ini or .htaccess). If not then you will have to live with this limit.
- ASP.NET/IIS is also limited for HTTP POST in Web.Config file.
- Some old web servers are compiled in 32 bits mode, so they cannot handle more than 2GB content-length.


To solve these issues, JFileUpload supports file chunks upload. It means that applet will split file in parts, upload each part and a server-side script will recompose original file from parts.

Another workaround to HTTP POST limitation is to use HTTP PUT upload. Most web servers consider it as streaming and do not limit stream size. They could just setup a limit on the time to handle the HTTP request. For instance, PHP script execution could be limited to 1 minute so it could stop upload before the end. However, such limit can be increased easily by API in PHP script. So, JFileUpload also supports HTTP PUT upload.

Last problem, if there is a problem while uploading, browser cannot resume transfer and you have to start from beginning again. So, JFileUpload provides resume support by sending an HTTP HEAD request before upload. So it can know if the file to upload is already on server and its size to find out resume offset. JFileUpload is also able to retry upload automatically on failure.

Finally, the best configuration for large upload (>2GB) with resume support is to enable:
- HTTP PUT (to get rid of HTTP POST limitation)
- Chunk support (to split file in part, part size to define and could be very high)
- Resume (based on size, CRC32 or MD5)
- Retry on failure

It means usage of:

Code: Select all

<PARAM NAME="param2" VALUE="httpmethod">
<PARAM NAME="value2" VALUE="put">
<PARAM NAME="resume" VALUE="true"> // Resume based on filename and filesize
<PARAM NAME="retry" VALUE="3"> // Retry 3 times on failure
<PARAM NAME="chunksize" VALUE="16777216"> // Here 16MB but can be increased
<PARAM NAME="chunkmode" VALUE="onflyrange"> // Parts in memory not on disk
<PARAM NAME="resources" VALUE="i18n"> // Overall progress bar, not per part.
<PARAM NAME="mode" VALUE="http">
and a server-side script able to handle HTTP PUT upload + resume + parts:
http://www.jfileupload.com/products/tools/index.html
(You will find ready-to-use samples of PHP, ASP.NET and JSP scripts)

Here is the HTML/JavaScript sample below:
largeupload.zip
HTML/JavaScript
(4.11 KiB) Downloaded 921 times

Post Reply