VMAF is super cool, although not the end-all-be-all of video quality measurement for sure. I'm currently evaluating switching over to the new v1.0 models.
The (unsolicited) slop PRs on that repository are a sight to behold, though. The same slopper posted an aggressive response to my PR: https://github.com/Netflix/vmaf/pull/1586
It's a flawed, and unreliable metric. Other metrics like SSIMULACRA2, CVVDP, or Butteraugli are more reliable/accurate and should be used if one cares about quality.
At this point I've accepted that streaming is for the stuff that already doesn't look great (a lot of streaming content frankly), and for the rest there's physical media.
The sensitivity to that difference is extremely varied. Customers buying TVs are attracted to ridiculously brightly colored videos, and go home to watch a murky movie with banding artifacts and say “I love my tv.” And they do.
I myself am not a golden eye, and I know some, and they can clap along to fragment boundaries during streaming.
Yes this is all very true, and I have my own limitations of course; I don't fret over $200 cables or any of that. The truth is that for some movies for the reasons you already stated there's no difference worth noting because the problems are with the grading, lighting, or some other production issue.
In some cases though I think with a decent OLED most people would tell the difference. A movie like Sinners in UHD from physical media gives you something much closer to a good theater experience than the same movie compressed and streamed. Would I say no to watching Sinners at a friend's place through streaming? Heck no, it's a great movie regardless, but when I have a choice and the convenience... I'd pick the disc.
Blog post about the latest version: https://netflixtechblog.com/vmaf-v1-good-is-not-good-enough-...
VMAF is super cool, although not the end-all-be-all of video quality measurement for sure. I'm currently evaluating switching over to the new v1.0 models.
The (unsolicited) slop PRs on that repository are a sight to behold, though. The same slopper posted an aggressive response to my PR: https://github.com/Netflix/vmaf/pull/1586
I was hoping this would be a post announcing the death of VMAF from Netflix, alas.
Why?
It's a flawed, and unreliable metric. Other metrics like SSIMULACRA2, CVVDP, or Butteraugli are more reliable/accurate and should be used if one cares about quality.
At this point I've accepted that streaming is for the stuff that already doesn't look great (a lot of streaming content frankly), and for the rest there's physical media.
The difference is STARK.
The sensitivity to that difference is extremely varied. Customers buying TVs are attracted to ridiculously brightly colored videos, and go home to watch a murky movie with banding artifacts and say “I love my tv.” And they do.
I myself am not a golden eye, and I know some, and they can clap along to fragment boundaries during streaming.
Yes this is all very true, and I have my own limitations of course; I don't fret over $200 cables or any of that. The truth is that for some movies for the reasons you already stated there's no difference worth noting because the problems are with the grading, lighting, or some other production issue.
In some cases though I think with a decent OLED most people would tell the difference. A movie like Sinners in UHD from physical media gives you something much closer to a good theater experience than the same movie compressed and streamed. Would I say no to watching Sinners at a friend's place through streaming? Heck no, it's a great movie regardless, but when I have a choice and the convenience... I'd pick the disc.