ACES Gamut VWG Participants(please add your own details here):
This document is meant to serve as a source of truth & summary of work done over the first ten meetings of the ACES Gamut Mapping Virtual Working Group(VWG). It will serve as a platform for group consensus and to reference as we move forward with gamut mapping algorithm development and testing.
HISTORY
A great deal of discussion has focused on spectral sensitivities of cameras, and the possibility of taking these into account, or using spectral reconstruction techniques as part of the gamut mapping approach. Any algorithm proposed in this group should be generalized and therefore should not presume to know the origin of every piece of image data being processed, or the spectral characteristics of the system which generated it.The gamut mapping approach we take should deal with the ACES image data“as is”, and simply strive to convert that into less problematic image data – what Daniele Siragusano refers to in his document as“gamut healing”. Any discussion of modifying the source data as it is transformed to ACES is the domain of a future IDT VWG, and outside the scope of this VWG. Therefore any such discussion which happened in the meetings will not be included in this document(unless a brief inclusion is necessary to place a relevant point in context) in order to keep things succinct and on-topic.
It was agreed fairly early on that although the possibility of creating a new ACES working space which mitigates common gamut issues should not be discounted, this would require a very strong case as to the benefits. Changing a core component of ACES would potentially introduce backwards compatibility issues, and would also be based only on the situation at the current time. It raises the possibility of having to change the working space repeatedly in future.
Likewise the option of using 2006 CMFs instead of 1931 was discounted. Harald Brendel’s paper shows that there is no obvious benefit. The ACES framework is based around the 1931 2 degree observer, and other common color spaces(e.g. sRGB) have no spectral definition. So there would be backwards compatibility issues to deal with. Also a conversion methodology would have to be defined, which is not something appropriate for this group.
This group is focused on gamut management of scene-referred data.While it is clear that some degree of gamut compression is beneficial as part of a display transform, that will fall under a different group. It has been noted that if the scene-referred data has already been compressed into a more reasonable domain, a“lighter touch” may be required of any display-referred mapping. It was also noted that while color appearance models may be useful for display-referred gamut mapping, these are not relevant to our scene-referred approach. Perceptual concepts such as hue, saturation and lightness do not have any real meaning for scene-referred data, as they are dependent upon the observer and their adaptation state. Daniele noted that HSV is a problematic projection for modifying colors in and instead suggested investigating opponent spaces. HSV might still be useful to provide the saturation component as a multiplication factor in the opponent space. Opponent color spaces that use a 0-1 domain could still be used by normalising the RGB components to the max value of the triplet. This could help in maintaining exposure invariance.
SETTING THE SCOPE
The aim is to change only as much as is necessary in order to achieve the goals for the current issues, and avoid the temptation to go back and tinker with prior decisions about how ACES functions. If future work(such as improvements to IDTs) affects work we do in this group, we will revise at that time.
Based on the history above, our general working assumptions are:
Samples are relative scene exposure values(i.e. scene-referred linear data) with no assumed min/max value range boundaries
The gamut mapping operator is per-pixel only(i.e. not spacial or temporal)
Some stated ideals for a gamut mapping algorithm are:
Exposure invariance – f(a·RGB) = a·f(RGB)
Source gamut agnosticism
Monotonicity
Simplicity – ideally suited to a fast shader implementation
Invertibility(see caveats later in document)
Colors in a“zone of trust” should be left unaltered
While a suitable algorithm should be able to map arbitrary gamut A into arbitrary gamut B, it should not be a requirement that all source data must be contained within gamut A. Nor is it necessarily a requirement that the output should be entirely bounded by gamut B. Indeed, allowing extreme source values to be mapped to output values close to, but not within, the target gamut means that the compression function does not need to tend to the horizontal at the boundary. This means that its inverse will not tend to the vertical, which is beneficial for invertibility.
Because the unreal colors which occur are a result of the mismatch between a camera and human observer(among other causes), and are outliers in the residual error of a transform optimized for a subset of important(memory colors / Pointer’s gamut?) colors, what they“should” look like is somewhat undefined. The important thing is to remap them into values which are plausible rather than“accurate”.
What is outside the scope:
Colorimetric accuracy or spectral plausibility of input device transforms(IDTs)
Full capture to display gamut mapping.(Required modifications to the RRT/ODT will need to be addressed by a subsequent group.)
Customizing for specific input/output gamuts
Working in bounded or volume-based gamuts
Anything which could limit creative choices further down the line(e.g. excessive desaturation)
RESOURCES
The original working group proposal can be found here.
For testing purposes, a range of imagery is needed, and images have been submitted to a DropBox folder from a range of cameras. Thomas Mansencal has created a Colab for generating synthetic images based on various camera sensitivities and light spectra. To add to this, Harald has provided a dataset of the spectra of common LEDs. While it was originally proposed that a test shoot could be set up to generate imagery, in the current situation we are limited to already existing or synthetically generated imagery.
A number of links to potentially relevant publications can be found here, here and here.
Daniele Siragusano wrote a detailed response to the original group proposal which can be read here.
Sean Cooper presented a notebook(which has also been converted into a Colab by Thomas Mansencal) investigating the paths of lines of constant hue based on the concept of the Rösch Color Solid. In this he showed firstly that these lines are considerably distorted as they approach the spectral locus as plotted using the Standard Observer, and secondly that due to the Luther-Ives Criterion failure of current capture devices, there are far greater distortions when the same lines of notional constant hue are plotted for a camera via an IDT based on a 3x3 matrix. Indeed, these lines double back on themselves as they approach the spectral locus, and cross over one another creating ambiguity as to the meaning of a given chromaticity. These findings lend weight to the working group’s argument for keeping IDTs as a separate topic, outside the scope of this group, and not being overly concerned with the colorimetric meaning of outlying chromaticities.
EXAMPLE WORK
Matthias has created a Nuke script, packaged together with a selection of images, demonstrating a few approaches to gamut mapping. It is described in this post on ACES Central, and is intended as a straw man to stimulate discussion, rather than an actual proposal for gamut mapping algorithms.
Daniele presented the gamut mapping operator from Baselight. Their research had found no one ideal universal gamut mapping operation, so they created a user-controllable operator, to be applied as necessary during the grade. He noted that it is gamut agnostic and exposure invariant, and that their algorithms stay within the RGB world - not using any perceptual spaces.
Justin Johnson proposed a method of gamut mapping based on known camera native primaries - the example is of Arri footage. It uses a saturation mask derived from the target gamut. It is not realistic for our purposes due to needing to know the source camera, but was a good discussion starter on requirements and process.
Jed Smith posted about a distance(saturation) based RGB approach referencing Daniele’s note about Filmlight’s algorithms. He notes that it’s preliminary, and would benefit from more robust methods of control, as well as weighting from secondary colors as well as RGB.
NOTES ON IMPLEMENTATION
Through the course of our discussion, several notes on future implementation details have been discussed. While this is an architecture group, and implementation will come once our work is completed, there are a few areas in implementation(parameterisation, invertibility, pipeline workflow) that might affect our design work on any gamut mapping algorithm(s). Therefore, we have included a few pertinent details in this report for reference.
While parameterisation may give the greatest flexibility, within most ACES implementations(especially ones such as OCIO) there is no method for passing user parameters to a CTL. Indeed, CTL is not implemented“as is” in any production system. Additionally, unless there is a robust way to ensure that user selected parameters are applied identically everywhere, this could introduce inconsistency. And consistency is a major selling point of ACES. AMF may make this simpler in future, but currently it is a concern. It was suggested that perhaps parameters might be chosen from a range of presets, so this could appear in a UI in a similar way to the choice of Input or Output Transform.
Invertibility of the transformation is another aspect that was discussed. The consensus was that while an inverse transform should be defined it comes with the caveat that gamut expansion can create undesirable results, especially when the gamut mapped image has been modified as part of post-production. A possible method for mitigating this could be an approximate inverse that reduces the slope of the gamut expansion in the boundary regions.
Alternatively, if the algorithm is of a type that is not entirely bounded to the target gamut, then an exact inverse could be less problematic.
While the ideal position in an ACES pipeline for scene-referred gamut mapping to be applied is still under discussion, the consensus seems to be that the best place is during the transform from AP0 to the AP1 working space.
Given that the interchange colorspace is ACES 2065-1, where does the discussion and integration of invertibility fit? Should the working assumption be that AP0 is still king, and possible imaginary pixel values should exist in the archive, regardless of their mapped working space? There is also the possibility of introducing new artifacts along the way(in VFX in particular) that did not exist in the camera source. Is this acceptable?
HISTORY
SETTING THE SCOPE
RESOURCES
EXAMPLE WORK
NOTES ON IMPLEMENTATION