Introducing slimRAW: A Fast CinemaDNG Raw Video Compressor
CinemaDNG lossless compression has been an interest of mine for some time. I’ve described a possible workflow before. But it had some issues and the (lack of) speed was annoying, to say the least. So I’ve been thinking about a better solution.
This led to the creation of slimRAW. A fast CinemaDNG compressor developed by yours truly.
slimRAW supports all types of uncompressed CinemaDNG and DNG raw video that I could get my hands on. It has been tested to work with footage from Blackmagic Design Cinema Camera (pre firmware 2.1); Digital Bolex D16; Sony FS700/FS7 CinemaDNG raw recorded through Convergent Design Odyssey 7Q/7Q+; Canon DSLR Magic Lantern raw converted to CinemaDNG/DNG through any of the available converters; Ikonoskop A-Cam dii; uncompressed CinemaDNG from Kinefinity cameras (including 6K from the KineMAX); Fastec Imaging TS and HiSpec high speed cameras; Indiecam indiePOV and indieGS2K (12-bit footage). Other sources are likely to work if the footage is 8, 12, 14 or 16-bit (but no guarantees).
slimRAW losslessly compressed CinemaDNG output has been tested to import fine in Blackmagic Design daVinci Resolve, Assimilate Scratch, The Foundry NUKE, Adobe Premiere Pro CC, Adobe SpeedGrade CC, and in Adobe software using Adobe Camera Raw for CinemaDNG/DNG import – After Effects, Photoshop, Lightroom. It should also work in any CinemaDNG conformant software.
An obvious application of slimRAW is to compress existing uncompressed CinemaDNG footage and free storage space without any loss of quality. But there is more to it. An important goal in development was to achieve faster than real time compression speeds. This would allow for practical offload-with-compression on location in one step: offloading video from camera or recorder storage to main storage while compressing it in the same time.
slimRAW compression was coded from scratch for performance. Processing is parallel and scales with available CPU cores. Example processing speeds (on a 3.4ghz Intel i7-4770 under Windows 7, running footage off SSD): around 110fps for Digital Bolex D16 2K CinemaDNG uncompressed footage and around 70fps for Blackmagic Design Cinema Camera 2.5K CinemaDNG uncompressed footage. slimRAW is fast enough that in a lot of cases performance is limited by storage bandwidth and not by the CPU. SSDs and RAID certainly help with that.
slimRAW also achieves the best compression ratios I’ve seen for lossless CinemaDNG compression (including software and hardware, i.e. cameras outputting losslessly compressed CinemaDNG). Lossless compression ratios aren’t fixed. They depend on the nature of source material: noisy and detailed footage will compress less than clean and defocused footage. With test footage from various cameras slimRAW compression ratios range from around 1.5:1 to 2.8:1 (a reduction of size to 66-35% of the original size). Some clips actually break the 3:1 barrier.
slimRAW preserves all the original metadata. This includes CinemaDNG specific metadata like time code, frame rate, T-stop, etc. It doesn’t touch the color matrices or any other color related metadata. This means that in any video production application supporting losslessly compressed CinemaDNG there is no difference between the original files and the losslessly compressed files. Moreover, the compressed files should be easy to swap in place of the originals in existing projects. Applications using Adobe Camera Raw are exceptions since ACR modifies metadata in the input files on import and records settings in there.
Since support for CinemaDNG in Premiere Pro CC and SpeedGrade CC is incomplete, slimRAW provides a user selectable option to output losslessly compressed CinemaDNG files compatible with these two applications. The trade-off is a small sacrifice of compression. As a curious bonus feature, slimRAW can actually convert some uncompressed DNG video which is unsupported in Premiere CC to losslessly compressed DNG video which works with Premiere CC. The most notable example is DNG video converted from Canon Magic Lantern raw with ML’s raw2dng: Premiere CC will reject the uncompressed DNG video, but the slimRAW losslessly compressed sequences will import fine.
To recap, a few nice points about slimRAW:
- Speed.
- Optimal compression ratios.
- CinemaDNG metadata preservation.
- Output compatible with a bunch of video production software.
- Can offload-with-compression.
- Checksum generation to help track data integrity through post.
- Versions for both Windows and OS X.
Read more on slimRAW’s website.
Thanks to Csaba Nagy and Frank Glencairn for testing and discussion.