rscheffler
Offline
• • • • • •
Upload & Sell: Off
|
 p.1 #1 · Lightroom Classic v14.4+ new Denoise workflow optimization? | |
I finally updated to LRC 14.5.1 from a previous version that generated separate Denoise DNG files. Most of my event work is higher ISO and I've found Denoise very useful at what it does. Therefore, I use it on virtually 100% of my images.
My workflow with the previous Denoise DNG files was to move them to a separate folder at the end of a job/project and then run them through Adobe's DNG Converter app with lossy compression applied. I'd then save these lossy compressed files with the original RAW, simply as a source for all of the edits saved to the images if I ever had to reprocess them (at which time I'd generate new Denoise DNGs and copy the edits over from the lossy compressed files - but 99% of the time I never revisit client work to reprocess, so this clunky solution would rarely be used in a high volume situation). I wanted the lossy DNGs for the benefit of keeping a set of the edits that were independent of the catalog file and they would typically compress way down to a bit less than Canon's own CRAW lossy compression, so it wasn't a terrible storage hit, pretty much equivalent to saving a set of jpegs.
With the v14.4 move away from generating DNGs, I've noticed a few things and browsed the comments over at Lightroom Queen about the 14.4 update. It appears Denoise, AI masks, AI generative repairs, etc. are stored to the lrcat-data file in the catalog folder. But it seems like a nearly equivalent amount of data is saved to the lrcat file as well, but only after I do a 'save metadata to files' to also save the edits to sidecar XMP files. These XMPs jump in size to between 7-30MB (for Canon R5II 45MP files).
My current project is 3800 images large, about 75% are 45MP files and the rest are 24MP. All have had Denoise applied and the catalog is only this project, saved in the project folder and not in the default LR catalog location. After running the 'save metadata to file' step, the catalog folder ballooned to 70GB, virtually equally split between the lrcat and lrcat-data files. About 3GB was in the lrcat-wal file, which was temporary because it reset when the catalog was closed and reopened (not the case for the lrcat or lrcat-data files). Edits saved to the XMP files total about 40GB.
I should add that I always make a separate catalog for every project/job and given how the catalog now appears to balloon with Denoise data in it, the catalog will now always be in the project folder itself and off the boot drive (to save the limited space I have there).
So the point/question I'm getting to is: there seems to be data duplication. I really like having edits saved to XMP as a fallback in case of catalog corruption and it's easy to grab a few files and add them to another catalog and have all their edits in place. But saving to XMP appears to bloat the lrcat file. Has anyone figured out if we can trash the lrcat-data file IF we saved edits to XMP sidecar files?
(It appears the answer is YES.)
I explored a bit more and noticed if I unchecked Denoise for a file and did a 'save metadata to file' that the XMP discarded the saved Denoise data and shrank considerably in size (to sub-1MB). I made a test catalog of 50 images and when I deselected Denoise in all of them (had to do it one by one, making a batch deselection change didn't apply deselection to Denoise), and did another 'save' the XMP files all shrank to below 1MB each. But the lrcat-data file did not change size, likely because the Denoise data is being retained as a part of each file's edit history. When I imported the 50 images to the test catalog, all already had Denoise applied in another catalog, and that edit info was transferred via their XMP files to the new catalog. At this stage the lrcat file was only a few MB, but the lrcat-data file was about the same size as all the XMP files (with Denoise data) combined. After I did a 'save metadata to file' for all 50 files, the lrcat file shot up very similar in size to the lrcat-data file. After closing that catalog, I deleted the lrcat-data file and reopened the catalog. LRC presented an error message that if I continued, AI edits would be lost. I continued anyway and AI masks were still there, as was Denoise for each image. Presumably because the bloated lrcat file contained this information. Meanwhile, the lrcat-data file was only a few KB in size. I did a 'read metadata from files' step to see if anything would change with the images. There appeared to be no change but the lrcat-data file was now back to its previous size before I last closed LRC and deleted that file.
So it seems there's possibly a bug (according to Lightroom Queen) with how data is saved to the lrcat file if one also saves to XMP sidecar files. In any case, it appears that the bulky Denoise data is being duplicated in both the lrcat and lrcat-data files and when this is the case, deleting the lrcat-data file (at the end of a project) shouldn't discard the Denoise and other AI edits. And one would also have the edits backed up to XMP.
Also, given the usefulness of Denoise and given that it could add 7-30MB of data to the catalog for each 45-50MP file to which it has been applied, one can imagine that it wouldn't take too much to cause a LR catalog containing thousands of images to become very massive and potentially cause storage issues for those with relatively small boot drives. Additionally, simply turning off Denoise on an image by image basis may not remove this data from the catalog because it appears it's being retained as a part of each file's edit history.
|
