File format for storage.

[ericmark]

I was taking photos of a wedding so opted to change the format from Pentax RAW (PNG) to Photoshop RAW (DNG) so they would not have a problem in the future in reading the files.

However when I came to look at the images I found the size had jumped up from just under 10M to 16M in size, and when I came to correct the images in Photoshop, there was no sidecar created, but instead it was included with the DNG file, and to return to as taken, one had to open up each picture again, not simply delete the sidecars.

Due to the increase of size I needed two DVD’s rather than one to store the pictures, and I have as a result returned to using Pentax RAW.

Since Pentax have in the past been good as far as being backwards compatible, and I can still use lenses bought in 1980 on my D-SLR, I can’t see there being a problem in the future, unlike the Canon and Nikon who seem to have changed many times.

To convert to DNG must be even worse with cameras that will not record it directly in that format. At least with my camera it will record directly in that format, so one asks, why should we use the DNG format? Especially since very few makes seem to have given the option? Maybe with other makes, as with my Pentax, it takes up so much extra card space, the manufactures have decided it’s not worth it?

For the wedding I still think I did the right thing taking in DNG format, but can’t convert back to PNG, where as if I had done the reverse, and taken in PNG, I could have converted to DNG.

So to the question. Do you think photos, which are likely to be kept for a long time should:-
a) Be taken in manufactures camera RAW and stored in that format.
b) Be taken in manufactures camera RAW and converted to DNG for storing.
c) Be taken in DNG format in camera and stored in that format.
d) Be converted into TIFF, and stored, as it’s still 16 bit, and been around for years.
e) Converted to Jpeg and stored in that mode as takes up even less room.
Of course assuming all options are available.
Although I realise there is less info in a Jpeg, I don’t have any idea as to if there is more or less information stored between TIFF, DNG, or PNG (or other manufactures format), but would guess there must be losses in the conversion. So I would have thought storing in the manufactures format is likely best option!

[Marko]

I'm not sure what prompted you to make that change....and isn't PNG just a different web file format...? I'm not familiar w/Pentax...but on my Nikon, there'es only 1 RAW format. The whole DNG thing, as far as i know is just another RAW format made by adobe...but PNG is a different beast AFAIK.

I shoot RAW 100% of the time and so do most of the pros that I personally know. There's no predicting the future but since millions of photographers currently shoot RAW native to their cameras...I expect the format (or some easy way to convert formats should the need ever arise) to be well supported for the rest of my life...

[Iguanasan]

Pentax has the ability to capture images in RAW or JPEG formats. Pentax RAW, unless I am mistaken are .PEF files and my opinion is that this is the best way to store your images for posterity at this time as it's the raw data from sensor. Converting to either PNG (Portable Network Graphics) or JPEG (Joint Photographic Experts Group) formats "throws away" information to create smaller file sizes.

Digital Negative, DNG, is a format that hasn't proven itself in my mind. The aim for the format was to provide a standard format for all RAW files as right now they are all proprietary formats. However, the effort to convert each and every image to DNG for storage is an onerous one without any real benefit.

Very long term storage of images is quite questionable at this point as who knows how long this technology will last. I have data on 5 1/4 inch floppies but no floppy drive to read the disk. So, to that end, I try to keep all my images on live media. That is I store them on hard drives and, in fact, multiple hard drives, and continue to move them each and every time the technology advances. My hope is that I will keep pace with the change and not leave any of my images in the "back closet" and leave myself in a state where I cannot get them back.

Hope this helps.

[ericmark]

You are correct it's PEF not PNG. We have a pro at the club who keeps talking about using DNG for storage so decided to change for wedding. Will not do it again will keep to Pentax RAW format.

[Barefoot]

d) TIFF, no compression. 'nuff said.

[flozaeliza]

PNG is more meant for graphics art, JPEG is better for photos.A PNG file is just as large as a DNG file, sometimes even larger
So, I always prefer JPEG format.

[ericmark]

An old post over a year, old but oddly selected just at the time when I have been modifying my ideas.

The reference to PNG was an error as already pointed out should have been PEF. I seem to remember that unlike Jpeg PNG with allow transparency but it was a mistake anyway.

So I have now four 16 bit formats. PEF, NEF, DNG (all considered as RAW files) and TIFF used by one of my HDR programs before I have applied tone mapping.

In all cases they will be converted into Jpeg once tone mapping is complete. I have over the last few years improved my skills at tone mapping and general adjustment of images before finally turned into Jpeg and I have gone back to old images and reworked them.

Had I not kept the originally RAW file then it would not be possible. For never to be repeated photos of weddings etc I see the point in retaining the original.

However with the old film photo I rarely could find the negative to produce a new print and even when I could in the main simple copying the print was good enough. So I have to ask is it really worth keeping the RAW file once the Jpeg has been made?

Much will depend on the operating system but with my old XP codec will allow me to see thumbnails of NEF and DNG files but not PEF and also I can’t read NEF files with CS4 so all NEF need changing to DNG. If I am doing this for the Nikon then I may as well use same file format for the Pentax.

[Marko]

Just blindly trust me on this one....KEEP ALL YOUR RAW FILES.

memory is dirt cheap these days - I just posted a deal on FB for a HD that I bought recently - a tiny portable HD that holds 2 TB - 119.
WD My Passport 2TB Portable External USB 3.0 Hard Drive with Case - Silver : Portable External Hard Drives - Best Buy Canada (http://www.bestbuy.ca/en-CA/product/-/b0000965.aspx?path=4f4d879cf6d44bce4b81b6b058c41ec 4en02)

(I see that u are in the UK - but a similar deal might be findable if u search a bit )

[Realist]

Personally I keep my RAW files. When I upload them on the internet I convert them to JPEG's, but when I print them I convert them to 8 bit TIFF's. Think the biggest thing I need to get better at is to delete the bad photos. Not all pictures are winners, so the losers get deleted for good.

[ericmark]

I will keep a RAW files but which one? PEF and NEF or as DNG? Can't see point of keeping both. And with Nikon I have to convert no option.

[Marko]

I'd keep the original proprietary files. (NEF, PEF, CR2 etc.)

[ericmark]

With the Pentax PEF I would agree as Pentax is well know for keeping to the standards it has set with lenses made back in 1980 still fitting a modern camera. However Nikon as I found to my cost seem to change things at the drop of a hat and although still with the label NEF clearly these formats have changed through the years with warning by Nikon about using older software with newer files seems it can cause corruption. So with the unstable nature of the Nikon format I think keeping just the DNG is the better option.

I have tried to give reasons wonder why you would keep Nikon rather than Adobe formats I am sure you have some good reasons?

[QuietOne]

:twocents: I'm going to sound a bit disjointed - it's getting late.

No one knows the file format of a proprietary file like the manufacturer. I see a lot of complaints similar to yours around the 'net. If the noise gets loud enough to be bad publicity, it may be they'll add those older formats to their latest, or at least have a converter of some kind written.

DNG doesn't seem to be catching on. I don't think it will go away - various open source projects will see to that. But it's a conversion, and any software that converts your RAW files is using a reversed engineered version of the original algorithm. Some versions are better than others, but there will be changes. It can't be helped.

You may well have to work with a DNG file, given your software and hardware limitations. But if you go back and rework a file, if you decide the settings aren't quite right, you might get just a little more starting with the original RAW converted to DNG fed into your photo editing program. It may also be that DNG conversions of your particular RAW get better in the future, because the reverse engineering was refined, or because Nikon decided to release at least some information on the format (unlikely, but not impossible, if it's the next marketing edge one of the camera companies decides to try). However it comes about, you won't have the original to reconvert from. I agree with Marko, get external hard drives. It's at the tail end of the holiday season, but you might still be able to pick some up on loss leader prices. Back-to-school is the other time they get really cheap. At the least, consider keeping the RAW for your best photos.

[Mike Guilbault]

Actually, if you're worried about losing the 'original' RAW file, you can embed the original RAW within the DNG. It's an option that can be applied when you do the conversion. So, you get the benefits of DNG as well as keeping your original RAW. Personally, I don't bother. I have LR automatically convert my NEFs to DNG upon import and then delete the original NEF (this is all done automatically on import with LR). All the original RAW data is kept, but for me, the big benefit of DNG is that the file format also holds all the Metadata and any adjustments you make to it with your RAW processor, LR for example. As soon as you make an adjustment to a RAW file, it creates a Sidecar file with a ".xmp" extension and the same filename as your RAW file. If, for some reason, you happen to move your RAW file but don't take the xmp file with it, you've just lost all your adjustments, keywords, and other Metadata. The DNG format retains all this information in one file. Consider the DNG format as an envelope that holds your RAW image as well as all the metadata and adjustments you've made to it, with no chance of loosing it from moving files around your computer. For me, it's DNG all the way.

[ericmark]

Thank you all for your input. As rightly said things move on now running Vista and CS5 with RAW 6.5 so can work direct with Nikon RAW files. Also found RawTherapeePortable which will process Nikon RAW and is free so there is no need to convert the DNG.

Oddly I like the sidecar idea it means I can remove all alterations and start again.