My Feature Ideas

[ZiNgA BuRgA]

I think I'll ditch gzip in XThreads attachments. (and I kinda wanted this)

The stupid HTTP specification fails to mention the ordering which Content-Encoding and Content-Range should be applied in.  And it's hard to find any info on this via Google.
But I did manage to find something after quite a bit of searching.  Apparently, it's compression then range selection, though IDK about the support of clients/download managers for this are:
http://www.mail-archive.com/www-talk@w3....http://www.mail-archive.com/www-talk@w3.org/msg

Which makes things a little tricky:

• If I store the file uncompressed, there is no way to to select the range as I cannot generate compressed data from a specified range.  Compressing the entire file up to the range may be possible (difficult because PHP doesn't provide all the zlib APIs), but absurdly slow
• If I store the file compressed, everything works well, except if a client doesn't support gzip content-encoding, and decides to use ranged requests.  This wouldn't be a problem if gzseek wasn't slow.
• All problems are solved if the file is stored both compressed and uncompressed - except for the fact that this increases the amount of disk space used
• If PHP exposed all of zlib's API, it may be possible to force deflate block splits mid-stream - this would be somewhat of a tradeoff in that you'd get a slight bit of slowness (but this is limited because you only need to start decompressing from the start of the block rather than the start of the file) and less compression efficiency (due to forcing block boundaries).  Nonetheless, would be difficult to implement, and would require managing an index (yuck)
  EDIT: actually, this may be possible with a bit of fiddling around with binary strings.  I still think it's yucky, but doable perhaps.
• I could offer a tradeoff - enable gzip and disable ranged requests; maybe offer such an option to smaller (uncompressed?) files
  

Also, PHP doesn't offer a way to compress in chunks (like zlib) - you can only compress the whole data at once.
None of these work:

PHP Code:

fopen('compress.zlib://php://output', 'wb'); gzopen('php://output', 'wb');

---

However, a workaround may be to use stream_filter_append: http://php.net/manual/en/filters.compression.php
Will only work on PHP 5.1.0 and higher.  Gzip headers can be outputted, the rest through something like:

PHP Code:

$fp = fopen('php://output', 'wb'); stream_filter_append($fp, 'zlib.deflate', STREAM_FILTER_WRITE, 6); fwrite($fp, $data); fclose($fp);

---

The remaining difficulty would be calculating the CRC32 in chunks, but this can probably be reasonably done by porting zlib's crc32_combine function.

Ah well.