Using Accelerated Proxy Responses for XThreads Attachments

[ZiNgA BuRgA]

Have recently moved my forums onto a new server setup which uses an nginx frontend -> Apache backend, proxy setup.
One feature of such a reverse proxy / "HTTP accelerator" setup that XThreads can take advantage of is the nginx "accelerated proxy response" (lighttpd has something similar, and probably other webservers do too).
[note, "accelerated proxy response" is a term I've coined for this article, since I don't really know the proper name for it]

Benefits of accelerated proxy responses
This feature is most beneficial to sites which serve larger downloads, and probably also to those that have a high number of downloads relative to visitors.
You see, it seems somewhat silly to have a PHP script feed a file to a webserver when the webserver is much more efficient at sending the file itself - after all, that's what they're designed for.  But serving static files directly also means less flexibility since there's no scripting opportunity (such as download counters).  Accelerated proxy responses sit in the middle of these two and allow you to take advantage of the benefits of each method.
This essentially means that "download" scripts (xthreads_attach.php for XThreads) do less work, and won't stay resident in memory for the duration of the file transfer, not to mention the fact that you save server resources on not having to have large PHP instances hang around memory.  Furthermore, clients performing multi-part downloading greatly magnify this problem.
For example, if we take a typical prefork Apache with PHP module setup, it's not uncommon for each instance/process of Apache to consume 10MB (or more) of memory (xthreads_attach.php is smart enough to close the database connection when not needed, so we won't worry about MySQL threading overhead).  On this example setup, if we have a user downloading a large file, using 10 concurrent connections (not an unusual thing for someone to do), that's 100MB of memory being consumed via Apache/PHP/MyBB just to serve this download.  And this 100MB will stay consumed throughout the duration of the download - so if it takes them 2 hours to complete the download, that's 100MB of "unusable" memory for 2 hours.  Even if you have a more efficient CGI/FastCGI setup, there's still going to be a fair number of CGI processes needed to feed this download.
With accelerated proxy responses though, this request could probably be served with a few MB memory being consumed throughout the download, if even that.  Of course, the initial 10 requests may cause a bit of a spike in memory usage as PHP does its thing, but this will be short lived, so unlikely to be a problem for most setups.

Who is for?
Obviously you need to have a reverse proxy setup already in place to take advantage of this (i.e. have a VPS/dedicated server set up specifically like this) so I'm going to assume that you already have such a setup in place.
And of course, to those looking at reducing server load, in particular, memory usage, and have a number of large downloads served by XThreads (or another download system which you are willing to migrate to XThreads for this benefit =P).

XThreads has supported this feature for quite some time (from memory, I added it during the 1.2x versions, so any recent version will have this feature).


I'm just going to give a basic outline of how to enable XThreads to take advantage of it on an nginx frontend here, since I'm going to be setting this up as I write this; the process is probably similar for other webservers, so check the documentation.
Even if you are going to be doing this for nginx, I still recommend reading this page to get an idea of how it all works.

Enabling support in XThreads
You need to have XThreads installed and activated.  If you have done this, you should have a file named cache/xthreads.php on your server.
Edit this file, finding the line:

PHP Code:

define('PROXY_REDIR_HEADER_PREFIX', '');

---

For nginx, change it to something like

PHP Code:

define('PROXY_REDIR_HEADER_PREFIX', 'X-Accel-Redirect: /uploads/xthreads_ul/');

---

if your forum is installed at the root of your domain, or, if your forum is installed at /forums/, use this instead:

PHP Code:

define('PROXY_REDIR_HEADER_PREFIX', 'X-Accel-Redirect: /forums/uploads/xthreads_ul/');

---

Save this edit and confirm that your downloads still work.  If so, then you've successfully set up this simple optimisation!

Getting weird file names?  Ensure that the proxying webserver is forwarding Content-Type and Content-Disposition headers (nginx does this by default).

Note: as this configuration option is edited in the cache file, uninstalling XThreads means that you'll lose this option and have to set it back up (if you want to) when you reinstall.


Issues
Download counters: as delegating the task of sending the file to the webserver means that XThreads is a bit less flexible, the download counting mechanism will revert to a MyBB style method (number of downloads = number of requests).  This is usually not a big issue, especially for sites receiving high number of downloads anyway.

URL format: if you've just migrated to a frontend -> Apache webserver setup, you may realise that XThreads attachments (as well as forums/threads in MyBB's archive mode) return not found errors.  This is somewhat beyond this article, but ensure that you're passing these requests onto Apache, for example, I typically use the configuration option:

Code:

location ~ \.php(/|$) { # note that this handles .php files as well as .php/ URL formats proxy_pass http://localhost:8080; }

---

Alternatively, disable fancy URLs in cache/xthreads.php and MyBB's archive/global.php (ARCHIVE_QUERY_STRINGS).


Other Steps?
The nginx XSendfile page does recommend declaring the redirected area as internal to stop users accessing the file directly.  As XThreads randomises internal filenames, sending appropriate Content-Disposition headers, and doesn't really have any authentication anyway, this is somewhat unnecessary.

[SunDi3yansyah]

does not work on my server
I use pure nginx, and have done editing the /cache/xthread.php

PHP Code:

define('PROXY_REDIR_HEADER_PREFIX', 'X-Accel-Redirect: /uploads/xthreads_ul/');

---

but no change :'( --help

[ZiNgA BuRgA]

How to set it up is very configuration specific.
I don't recommend trying to do this unless you have a good understanding of HTTP, your webserver and its configuration.

However your screenshot suggests a completely different issue - unless you've got split fcgi path info set up, your URL won't even work.  Set XTHREADS_ATTACH_USE_QUERY to 1 in /cache/xthreads.php and remove your edit above.

[SunDi3yansyah]

my php.ini

Code:

cgi.fix_pathinfo=0

---

was worth 1
due to security so I edit 0
What should I give value to it?

[SunDi3yansyah]

Thanks work
Thanks before ZiNgA BuRgA

[dthiago]

Only for nginx?

[ZiNgA BuRgA]

(05-01-2014 07:45 AM)dthiago Wrote:  Only for nginx?

It will work on any server which supports 'webserver accelerated sends'.  Your configuration may vary though.

[SunDi3yansyah]

(05-01-2014 10:05 AM)ZiNgA BuRgA Wrote:  

(05-01-2014 07:45 AM)dthiago Wrote:  Only for nginx?

It will work on any server which supports 'webserver accelerated sends'.  Your configuration may vary though.

whether the user is included in the GZIP

[ZiNgA BuRgA]

wot?

[SunDi3yansyah]

yes
I mean GZIP is 'accelerated webserver sends'