JPG and RAW processing swings and roundabouts

Last weekend we had a large family gathering which I was asked to photograph (since I used to do events and other commercial type photography in a past life). I picked my G1 as my main camera and thought I'd toss in my film EOS for a couple of shots (which actually I really wanted to get). As mentioned in my previous blog post there are some situations where a larger capture format (or among digital folks "full frame") just works better. If I had a 5D or other full frame DSLR I would have used that for the event not the G1 for just those reasons.

background

However this is not the main reason for the post, this time its (mainly) about the processing pathways of RAW and JPG.

I know that a number of professional photographers just use JPG as their main capture and shun the RAW workflow. Others do use RAW capture and often they are also people who have a team who work with them in post processing that for them. After this weekend I really see why that is.


Now normally I use JPG on the camera, picking RAW for just a few shots here and there, so as a result I'm not normally processing 300 images from an event so much as hand tuning five or six (perhaps) at a time.


Previously my workflow (don't you love that buzzword) has been to open the RAW in Adobe Camera RAW within Photoshop and then operate on the file. This was clearly not going to work for this situation or I'd still be chained to the computer working on images (as if I had nothing else to do).

what I mean by quality photography

One of the things I expect in a series of images is consistency,from one shot to the next (in the same situation and lighting) I expect;

• consistent colour
• consistent exposure

... to me nothing looks worse than faces and suits changing colour casts, contrast or tone.

It just looks like crap and is (part of) what separates the dopes from those able to deliver a good product to the client.

Now back when I got my first film camera with Av on it (we called it aperture priority back in those days) I soon found that my exposures were all over the joint and I wondered

"what the hell is the benefit of this automation if it gets in the way more often than not"

It didn't take me long to be lamenting my previous "match needle" light meter built into my (manual) 35mm camera (yes, it was an OM-1) and the way of using that I had developed. Heck compared to a hand held light meter that system was really handy automation.

I quickly learned to plop the camera in manual and just take meter readings occasionally (as lighting situations or lenses changed).

The next issue was proofs ... if I took a roll of film to a commercial lab I'd get a strip of 6x4 inch proof prints along with my roll of film. These were consistently exposed and reflected the exposure of my film (you can guess I was using negative here cos you don't bother with this on chromes / slides / E-6), by which I mean that each print was made with the same exposure from the negative that the previous one had been ...

Now this is totally unlike what happens if you take your images to a typical mini-lab at the supermarket / chemist / corner store ... there every bloody print is made by a machine which knows nothing about the previous print and thus tries to "optimize" each print. No matter how consistent YOUR SHOTS are the prints go back to looking like grandma's patchwork quilt (lovely as it is) not anything you'd be proud to show to anyone (if you have any shred of professionalism).

the real question

So, how do I process my 300 odd RAW files to meet my criteria of quality consistent results.

Well you've essentially got two choices use a product such as Lightroom (ohh, don't you have that yet? well .. just whack another couple of hundred down and add another few hours of your time getting to know it too) or you can use a free product such as dcraw (which is a command-line utility that actually is what is used as the engine for many RAW converters anyway).

Now, as it happens, I have my camera set so that RAW files contain an embed medium res JPG image that would be what the camera would have produced if I didn't use RAW. That is to say all the settings are applied to it including stuff like white balance, the "curves" the camera prefers and any lens corrections which it might apply (some cameras are doing this for you now). Naturally dcraw allows me to extract these JPG's (which then load nicely into something like irfanview) to quickly evaluate everything.

So far so good.

Now, because I set my white balance manually for each shooting situation, I can then use this value to tell dcraw how to apply colour to the RAW when domosiac occurs (eg when making an image I can do something with).

So armed with a few parameters I set it loose to convert my RAW files into 16 bit TIFF's for tweaking in Photoshop.

This is where I hit my first hurdle .... you see, dcraw is also a simple binary, so it can't make use of multi-threading. This means that if you happen to have a dual or quad core machine then its all going to just hog the one processor. On my box (1.4Ghz per core) this was taking about 2 minutes per image.

Fark (said the crow) ... 300 x 2 minutes is TEN hours.

So I killed this and thought about it for a bit.

Its worth mentioning at this point that each image as a TIFF at 16 bit is about 70Meg ... so I hope you've got some spare space on your disk for this cos thats 21 Gig right there.

So I though Ok, maybe I don't need need 16 bit ... after all, I didn't think I'd be making huge edits to the files, so 8 bit would be likely to be enough. To cut it down even more, I decided to go through and weed out my stuff to process only the files I wanted to convert.

All in all this brought processing down to just 2.5 hours

Time for going and doing something else while my machine crunches all this for me.

Getting back to Lightroom for a minute, a fellow I know who uses it says that he can use it to manage which images he wants to render to JPG and apply similar parameters to "groups" which were in similar lighting conditions ... thus it doesn't matter if the images were shot with the right light balance as I used above ... he can leave it on Auto White Balance (shudder) and ignore those parameters (while still getting lens corrections, something that dcraw can't do).

Ok, now two and a half hours later I still need to open my images in Photoshop and make any changes such as sharpening or colour tweaks (because I already know that my sensor does not make reds and yellows just where I like them for skin tones and I have not used any system to profile my sensor ... more about that topic here for example).

I have enough experience with Photoshop to know to sort this out on one image, then create an action to apply that across a batch of files, so I again don't need to sit there like a monkey (thank God).

This took me about 15 minutes to sort out, and then another 2 hours for the machine to grind away applying this to my subset of converted images.

As it happens I have a few computers in the house (... don't ask) so while I was allowing the PC to do this, I used my other PC to scan some of the film images on my LS-4000.

Thank God this has ICE as my negatives naturally came back from the "lab" (no professional labs round this neck of the woods) with fingerprints, streaks and some fine scratches running the entire length of the roll .. and no, its not my EOS that did that!

A few things came from this process:

1. image processing of my negatives was a bit longer than that of the RAW
2. all of the film images looked great, had better snap and better colour straight off of the film (heh ... he says)
3. I am so glad I am not locked into scanner hell anymore, cos doing this in strips of 6 really takes my time for attended operations.

This last point makes me think (at first)

"why the hell don't I have a roll film adaptor to scan 36 at a time, unattended"

... which then reminds me of the time I got all the film processed and scanned by a lab with a Noritsu scanner at the time of developing. Dam they were great!

Seriously I got very high quality 6Megapixel images from my film for $15.00 per roll (processing and burning to CD) at the time of processing. No dust, no scratches and I still got the Negs if I wish to use them for optical enlargements or other uses.

Sitting here now I am thinkin ... Man I'd pay $9 extra per roll to save all this time!

So now I'm thinking

"has digital let us down, or did we shoot ourselves in the foot with it?"

Aren't these things meant to make our lives easier?

OK ... the computer beeps to let me know the final batch is completed and so I give them a quick look over and then send them off (via FTP) to a service to have them printed and posted back to me (man, there is no way I'm going to pay hundreds for a good desktop printer to have that to fuck about with ... inks to go dry, calibrations to perform...).

So a full day (and some of the evening) of my attention was needed to produce this result, so that the proofs can be posted off to my grandmother, and the CD's burnt for other relatives to get their own prints made if they wish.

Sitting here on the train to another town I'm thinking about all this process and wondering about all this.

I can really see why many professionals prefer to use JPG and save themselves all this hassle, but the problem is that having looked at the results of processed RAW files, I definitely got better contrast and colour out of the RAW images than I'd have got with the JPG.

For example, this is one I took in the beginning of winter (part of my 56 series)

Camera JPG


RAW lightly processed

Its not immediately clear what's different about those two images, but if you open them up into separate tabs and look around you'll see (as I've identified below) that I have much better shadow details in the RAW sourced image, while actually getting better contrast in the stone area where there was low contrast to begin with.


and this wasn't even an optimized (carefully exposed right) image for RAW, I've easily got another 1/3 stop of headroom that I didn't make use of (cos I was in a hurry and when it clips it gets ugly fast). Heck I've even added some curves to the JPG to make it less contrasty (and chosen a camera profile which was less contrasty ... and this isn't even contrasty by Australian outdoor landscape photography standards!

options and choices

Now looking at my weekends images (you know, the usual drama black suites and white stuff on white stuff) the camera JPG's however had superior noise processing (with little discernable loss in quality), so the in-camera processing there is superior to what I can do here (or I could spend more money and try Noise Ninja or something).

Another advantage of the in-camera JPG processing is that it is done in parallel to the capture process (and not to mention done by dedicated signal processing hardware), so by the time I get finished shooting I have finished products (more or less) ready to roll (and they take up less storage space than the RAW too). So assuming that you are able to get just what you want from the JPG first go then you're better off with that in many ways.

The question is, "should I just go with the JPG's and by not seeing the results of the RAW and never know what I was missing be happy, or should I spend the money (oh, I love this "Digital is free after you buy the camera" shit) on Lightroom yet another product (ohh, and need to upgrade my OS and hardware probably) to get a better work flow and stay chained to the desk??

Hmmm ...

Well just maybe, but when I look at the results of the negative film workflow (which just happend to make the loveliest prints) I'm left wondering ... you see the Negs produced images which (in the exact same conditions) have better shadows and held hilights better(in snow for gods sake) than the JPGs (by a mile) and in some cases the RAW too *(mainly in situations where I let the camera meter and didn't do it manually resulting in missing head room on the RAW).

Images such as this one from the 35mm negative turned out to be the sweetest images in the group (and didn't really need much more effort than the digitals)

If I could reliably get my C-41 processed and turned into scans as I have had by the above mentioned Noritsu process I would probably never shoot another wedding or funeral in digital again (and I know a few wedding photographers who've said just that before too).


So with many professional labs going or gone under (leaving us with no choice but to go for a full digital process, not a hybrid one which has film capture then digital for editing and printing), I'm wondering if Digital is having some hand in killing jobs and contributing to skills losses in our community (I dare you to find a professional photo finisher or high quality master print maker for me in your area).

Actually there are quite a many professional wedding photographers who swear by Cameras such as the Fuji S3 and S5 digital bodies for exactly the reason that the sensors capture as much as negative does (but of course are still stuck in the digital post processing mud like me).

For me as a photographer I now need to know heaps more about the process than I ever did before. Once I just needed to make a well exposed negative (or chrome), now I need to do almost everything in the process with the actual printing being done by machine from the file I send it.

So sure I love my digital camera, I have actually developed myself as a photographer in using it to enable me to take way more experimental images than ever I would have with film.

Clearly I can't say I don't want digital, but perhaps the hidden costs are spread further than we thought.

I recently saw a discussion where wedding photographers were being hit with a "per gig price" of $500 ... hah! Go out on site for 8 hours, provide all the gear, go home (cos you won't be affording an office at those rates) and process your ass off. You can hear the clients now ...

"well you don't have any film costs anymore".

Its no wonder that photographers like this are taking over the industry.

Ahh I remember the good olde daze, I'd shoot 12 rolls of chromes for an event, wander down to the proLab in town drop them off ... go grab a bite to eat, pick them up one hour later and sit down with the gig director to sort out in 20 minutes on the light table what was going to be put into the projector that night. Digital has made it all so easy now ... as long as you don't want the quality to go with that side order of new software.