kineMINI 4K Raw Explained in Detail (Dan Hudgins)
August 30, 2014 at 8:13 pm #703
Topic #1, “how is the raw data recorded”.
1A) In the DNG the data is recorded 12 bit linear, this may change in the future, but in that case tag 50712 would have a linearizion table so the DNG reader would restore the data to being ONLY and FOREVER converted back to linear. The black offset is different between the Cineform raw and DNG data after both are restored to linear, that was something Cheng thought up, and should be taken care of in the shot’s 3D-LUT for monitoring matching, if not Cheng should fix that as it would be classed as a bug. The linear data does not change when the monitoring look group or ISO curve changes, they have no effect on the recorded data.
1B) To reduce compression artifacts in the Cineform raw recordings the data is black offset adjusted, in this case the black offset is lower than in the DNG which take the ADC output as is, and a SINGLE “log90” curve applied, this is NOT the KineLOG curve, if they are telling people that they are misinforming them as far as I know. The STATIC “log90” curve is similar to Cineon ™ encoding but not the same, its an internal format for Cineform raw codec, and has noting to do with anything else, and because not matrix has been applied to equalize the primaries should NEVER be used directly for grading, in spite of Kinefinity not telling people that and doing it themselves in tests they have shown, for some reason they ignore common sense in things that I have tried my best to explain to them as not being the ideal method of doing.
The Cineform raw “log90” data, which DOES NOT CONFORM to Cineon standard and is NOT matrixed correctly for end use, has is NOT mid-tone normalized, that means that it converts the code values from the 12bit linear data to “log90” encoded data WITHOUT any normalization for the ISO or EI setting in the camera, that data does not change no matter what settings are used in the camera’s monitoring or the ISO curve selected. Because of this there is “one light” conversion from Cineform raw “log90” to Rec709, and I know of no conversion tables made that do this outside of the camera making them from the monitoring tables, although it would be possible do this if one shot a test chart with white and gray cards and color reference like the chart in the sample BMP I sent along with the additional look groups in the zip files.
The point of the 3D-LUT in the camera’s shot folders is that they convert the Cineform raw “log90” which is NOT ISO normalized, to some other format that IS ISO normalized. ISO normalized means that in the RGB result the code values for 18% gray are more or less constant, and so if you changed the ISO curve in the camera downstream grading would work on shots made with different ISO curves, and that when viewed the results would have more or less the same brightness, which is NOT the case for the Cineform raw “log90” where the brightness will vary depending on what ISO curve was used for monitoring if the brightness was constant on the camera’s monitor because the exposure was adjusted to compensate of the ISO curve change.
Topic #2, “the LUT are not for DNG”
That is a false statement, the linear to log LUT, was made for SpeedGrade to convert the linear data in the DNG files into something similar to the non ISO normalized data from the Cineform raw “log90”, so that the monitoring matching 3D-LUT could be applied and the input levels would match.
But there is a problem, most programs use the tags in the header to muck up the data you see in the RGB conversion from the linear sensor data, some apply AGC, some apply arbitrary gamma correction, so do god knows what to the data.
What you need is a program that can convert the DNG raw data into linear RGB WITHOUT any conversion of the ratio of the tone values. A program called DNG_validate.exe (pc program, don’t know if there is a MAC version) may be able to do that.
Some other program may, there may be a LINEAR option in Resolve that could work, I have not tried it.
So in that case the workflow is:
DNG 12 bit linear > RGB linear > linear-to-Cineform_raw_log90_encoding > Monitoring_3D_LUT > monitoring_matching_RGB
The monitoring_matching_RGB should match the BMP snapshot in each shot folder, if not there is a bug someplace that Kinefinity should fix.
It goes without saying that the playback from the Cineform raw in the Cineform codec should match the BMP shapshot in each shot folder if you have loaded the shot’s 3D-LUT into the Cineform codec, because there is a signature code that mates each shot’s MOV file to its matching 3D-LUT, so the Cineform codec will switch the 3D-LUT used for viewing the playback on the fly while editing and viewing and rendering.
In my software I ignore all the color related DNG tags and the other points and grade directly from the original ADC output to get the most grading range. You can with the right software ignore all of the monitoring matching and Cineform raw encoding quality issues and produce RGB results directly from the raw linear data in the DNG as you like, you are not locked into any prescribed processing of you shots.
Various things are being worked on to improve the monitoring matching for one light results. These depend on Cheng making some changes to the camera’s firmware. When this will happen is up to him, I have tried to help as much as they would let me, but its up to them.
If any parts of KineSTATION are not working correctly I don’t know about it, I don’t have KineSTATION and have not used it, nor do I need to. I just use HFSExplorer, a free program, to download the DNG frames to my PC for processing.
Topic #3, ISO normalization.
1) The current DNG header does not contain any information for ISO normalization, and the Cineform raw “log90” data is intentionally NOT ISO normalized. It is intended to add some information to improve the DNG header so that it will come out ISO normalized in at least Resolve, but that has not been calibrated and released yet, so loading DNG into any program can give very different results as to how mid-tone from the shooting translates into the mid-tone value in the end results. This is not a “bug” in the camera, its a fault in the whole concept of the DNG header which Adobe developed for still camera makers and poorly adapted for Cinema use. I have talked to Lars Borg at Adobe about adding an alternative processing that would be 100% stable and give an absolute conversion that could never give “wrong” results, but I have not gotten any reply to that email after some months, and at the DNG meeting at NAB 2014 I got hostel replies from some of the
participants for even suggesting the addition of this very needed addition to the CinemaDNG spec. I don’t know why Adobe would be apposed to a faultless conversion option, two reasons come to mind, first they would have to admit that there is a problem, second that its “not invented here” issue, and then its not in alignment with their programs intended for photo publishing in the print industry. Other than that, if they would be open minded and willing to have an adult conversation about improving CinemaDNG to the level of a stable format for profesional filmmaking I would be more than willing to do what I can to see that that could happen. Until then, the DNG need to be kludged around to get accurate matching to what you see on the camera’s monitoring, no matter which look group is used in the camera at the time of shooting.
2) The Cineform raw “log90” does not conform to any know cinema log format, and is intentionally not ISO normalized so that the grading range can take advantage of the equal distribution of the data so that the effective ISO can be changed AFTER shooting in the grading process by changing the conversion 3D-LUT used to normalize the data.
Because the Cineform raw “log90” is not effected by the look group used for monitoring it is a neutral recording format, and so any of the 3D-LUT made from any look group and ISO curve can be applied after shooting to simulate having done that change at the time of shooting. Kinefinity did not develop that feature yet in KineSTATION, but they could and should so that camera users would have the option of changing their mind AFTER shooting, in other words, the DNG could be converted to MOV with KineLOGC for ARRI like downstream work, or later the same DNG could be converted to Kine709B for later Rec709 type work. The same could be done converting Cineform raw to Cineform RGB or YUV, you keep the original “neutral” recording and make copies that conform to various standards as needed.
You may be able to do that if you swap the 3D-LUT from one shot folder to the other if KineSTATION can be “tricked” that way, I don’t know if Cheng checks the “signature” in the 3D-LUT to see if it matches, it might because he is probably using the Cineform raw codec for the transcode, I am not involved in the internal workings of KineSTATION because I was not part of the Cineform NDA so they can’t talk to me about some details.
With regard to the best results, I think shooting at ISO 320 looks better than the higher speeds, especially for hand held shots, because of the noise levels. But because of the decreased number of stops above 90% white subject, you would need some reflectors or fill lights for high contrast shots in direct sunlight or at night with bright lights in the background.
Your question before about which ISO gives the best log dynamic range is a very different question than which ISO give best looking images when the lighting is within the range of highlight before clip.
For KineLOG look group which is NOT what Cineform raw “log90” is, in spite of Kinefinity confusing people by telling them that it is, the first ISO that holds all data values from 95 to 1023 is at ISO 1280, but all speeds hold the same dynamic range, its just that mid-tone moves up and down that range. When mid-tone is lower down the ISO rating is higher because the foot of the ISO curve is more vertical and so the higher ISO curves show more noise within the same dynamic range.
==October 27, 2014 at 11:06 am #874
KineMarcoParticipantDecember 26, 2014 at 8:37 am #875
WesParticipantDecember 26, 2014 at 10:56 pm #876
According to the information in the DR & ISO test by Filmcyfrowy, the “!” means:
“Note that with some ISO values a “!” is displayed, which means that this is a minimum ISO value for the present Highlights settings.”December 29, 2014 at 6:12 am #877
You must be logged in to reply to this topic.