It's possible to save RAW files (mostly) unprocessed with iPhones, either via built-in functionality (Pros) or via apps like Halide.
But the aggressiveness of the de-noising in the native JPG/HEIF images otherwise is really unfortunate if you want to look at the images on a screen larger than the phone's screen. The amount of detail lost (other than in areas like people's faces where the phone knows to specialise) can be very considerable.
I'd really like a way to dial that aggressiveness down a fair bit, even at the cost of more noise/grain and larger file size (through less compression due to the extra noise).
Another thing is the amount of lens flare you can get when shooting at the sun for sunsets/rises, etc or other large bright light sources.
With very small lens elements, from a physics perspective it's understandable that suppressing the reflections and inter-reflections is very difficult on such a small surface area (even with special coatings to reduce the fresnel reflection ratios), but if you care about image quality and wanting to look at images on screen larger than the phone which took them, larger format cameras still have some benefit despite their larger and heavier size and therefore inconvenience (looks at 5D Mk IV on shelf).
Do you know Lumina [1]? It has a setting that turns off most post-processing, resulting in very natural-looking pictures. It has more features than I need, and it's free.
I wish there was a middle ground between what Android/Pixel camera saves as raw, and the in-camera JPEG. Sometimes I have a few quibbles with the JPEG and what I'd like to do is edit the raw file, but starting from something close to the JPEG. Unfortunately what you get as a starting point from raw is hideous, and it's never clear how to begin. I don't think I've ever got an acceptable result trying to edit raw photos from my Pixel.
Start by adjusting the black levels and the exposure. This is where the histogram can help to visualize how much you are adjusting (if you have one). As the exposure goes up, you can adjust the highlights to recover some of the areas trying to blow out. As you pull the black level down, you can recover some of the details getting crushed using with the shadows adjustment. You can then adjust the contrast/saturation/warmth/tint as needed. The order of adjustments in the iOS Photos editor pretty much follow that order.
In other words, you want either your camera app to select the initial tweaks for you to be able continue in the external editor (not going to happen, RAW editing software is incompatible by design), or your editing software to select the initial tweaks that "look good" (that depends on your software). In RAW mode, Google Camera's output is photometrically correct, even if it stacks multiple frames or denoises it. Which is the only way to do it that makes sense, any other RAW camera app or actual dedicated camera does this the same way.
That's exactly what I'm talking about, how do you imagine that working? Metadata is not compatible by design, because processing pipelines are all subtly different and your result will always look different in your editor. Trying to match some basic parameters with the JPEG is possible and some RAW software can do that, but the result is going to be subtly different for the same reason.
Search over all values of all parameters and choose the ones the minimise the mean square error of pixel values.
Obviously that will be slow, so probably do some kind of gradient descent, or perhaps depending on what the parameters are there may be a closed-form solution, I don't know.
Yes the result will be subtly different but it's just a starting point.
I've never had to write such software but in my imagination there is the sensor data, potentially from several exposures, and some static data about the camera, and a list of edits and parameters that the photo app is using to produce the in-camera JPEG. And I just want a way to intervene in that list of edits and parameters to produce my own result. There must be SOME way to do this otherwise how do I edit raws from my real camera? The starting point for camera raw in my photo editor always looks great if the file came from my camera, and always looks ghastly if it came from my mobile.
For Android, you can sort of get some of this with Snapseed. I occasionally use it, and it's "ok". I'm more frustrated by the fact that my preferred RAW editor (DxO) doesn't handle Android's DNG files. For me, at least, editing raw images on a phone screen is just not tolerable.
It's strange that in the age of AI, denoisers are still so bad. It's basically impossible to photograph snowing in the winter because the denoiser will remove 90% of the snowflakes. Machine learning models are already used for denoising ray traced graphics with substantially improved results, so why is it that cameras aren't using ML denoisers yet? At least for still images. Or do they perhaps already use them, only the quality is still bad for unknown reasons?
(As someone who worked closely with pathtracing renderers and de-noisers, I think I can answer this :) )
It's mostly because in the VFX/CG space for ray tracing/path tracing de-noisers, they almost always rely on extra outputs/AOVs of things like 'albedo' (diffuse reflectance), normal / world position, etc, to help guide them in many cases.
So they often can 'cheat' a bit, and know where the edges of things are (because say the object ID AOV changes - minus pixel filtering, which complicates things a bit).
They can also 'cheat' in other ways, by mixing back in some of the diffuse texture detail that the denoiser might have removed from the 'albedo' AOV channel.
Cameras don't really have anything to guide them, so they have to guess. And often, they seem to use very primitive methods like bi-lateral filters (or at least things which look very similar), to try and guide them, but it doesn't work very well.
Portrait cameras on phones can use depth sensors a bit to help if the camera has them, but for things like hair strands, it doesn't really work, and is mostly useful for fake-depth-of-field depth-based blurring.
Yeah, but surely ML models would at least work better than analytic algorithms. After all, when looking at a noisy picture, our brain is pretty good at distinguishing detail from noise, so it's not clear to me why an ML model couldn't have denoising performance similar to the human brain, even if it doesn't match the "cheating" denoisers used in ray tracing.
It would probably help you to compare what you can do on a phone vs what you can do with desktop software (Lightroom/Photoshop, DxO, Topaz, CaptureOne, etc). It's generally quite good, with the exception of challenging liminal areas (e.g. hair, foliage).
Fwiw, Topaz -- which I have a license for but essentially never use -- has pretty incredible denoising & upsizing features (for both photo & video), but to get the optimal quality output you offload the processing to their cloud infra (and buy credits from them to pay for it). It's roughly the equivalent of a SWE using a local LLM that's good enough" vs a frontier model that's SOTA but requires a consumption-based subscription.
I've got a 61mp camera, and an RX 7900XT. It takes about 15s/picture for DXO to denoise, which is a lot longer than people are willing to wait on a phone to take a photo. Topaz is even slower. A cloud service could be used to do it in post, but someone has to pay for that.
Yet in modern computer games, modern graphics cards denoise a scene in real-time at 60 frames per second using machine learning models [1][2] while doing all the other rendering at the same time. Granted, that's ray tracing, and the resolution is lower, and they technically cheat by using additional information, but it might be that DXO is not optimized very well.
Would you be happy if ML detects that you are trying to capture the moon and replaced your shot with a memorized picture of the moon with match ing size and orientation ?
Autoencoders can presumably do this except that it would be operating at levels of patches rather than the entire Moon.
I don't think this is even just an AI denoiser problem. Rain effectively looks like haze for the parts of the scene that are at sufficiently long distances. Therefore it's difficult to use dehaze with rainy scenes. The harder the rain is falling, the more limited your ability to dehaze. It's much the same with denoise. You won't be denoising at 100% in Topaz if it's dark and raining hard.
One day smartphone cameras might get there, but right now, the sensor technology isn't there yet. The problem isn't merely noise. It's that rain and snow are moving about in the scene, which means that the camera can't just do its usual trick of taking multiple exposures.
AI denoisers are nice, but they aren't strictly necessary for ILCs. Even those full-frame cameras without IBIS are able to take pictures of nighttime snow just fine.
Are we still talking about smartphone cameras? If yes, apps already heavily rely on much more advanced computational photography than your average photo editor can do, including but not limited to ML denoisers. The problem is that such apps are typically optimized for the "average case" and are as automated as possible, so they either remove snow, rain, and haze intentionally, or lose small moving particles as the result of stacking. That said, snow and rain are usually possible to capture in the apps that attempt to determine the scene type or have specific modes.
The post does a great job at explaining some of the details consumers often overlook. Especially in regards to sensor size, light and accutance.
The best camera is the one you have with you. And this is why smartphones are so great, but the author also does a great job of expressing the limitations and problems.
I've worked as a professional photographer and videographer when between work. I've owned pro Nikon, canon and currently roll with some bonkers expensive leica gear. What makes smartphones so special as cameras is how easy they are taking unobtrusive photos of friends, family and kids. There are few things more frightening to me than a kid charging my way to rip the leica out of my hands because it looks so interesting. They don't do that with smartphones. Smartphones are especially stealthy because it's not clear what the user is doing with them. They could be just browsing the web or whatever. Are they taking video? Or a photo ect.
This is why I ended up picking up an (admittedly quite expensive) Ricoh GR IV. It's tiny enough to take with me everywhere, has a modern APS-C sensor and great IBIS.
Lens for image quality and sensor size and density for resolution, but we hit pretty hard limits on those a long time ago. Software on top of that has been the major differentiator for quite some time. Exposure stacking and intelligent detail control produce more improvement for less investment than a super-complex lens assembly or exotic sensor. Though it brings its own risks.
Not to say there is no movement on the other fronts. Glass was pushing for a crazy anamorphic lens and far larger sensor that would have been a serious improvement, but I don't know if it went anywhere.
I wish "real" camera companies were more aggressive about offering computational photography post-processing, at least as an option. I've gotten spoiled by an iPhone. Despite my Sony and Fuji having huge wonderful lenses, I am pretty disappointed when using a dedicated camera and interior lighting leads to slight blur, or a cloudy day produces washed-out skies.
My theory is that doing so would cannibalize their user base.
I think that those who buy a pro camera nowadays do it because they care about photography itself. For that public post-processing is a touchy subject: besides the philosophical aspect of "is this a true representation of the moment I captured?", there's a real chance that their work will be under-appreciated or rejected altogether if their camera is known as "the one that retouches your photo automatically".
I'm not sure that Nikon's sales would improve that much if they offered auto-post-processing (is that something amateurs a tively want?), but I can imagine that their current customers would be unhappy.
If you shoot manual and raw, there are ways you can fix those things. But you need to understand what you want to achieve before you take the photo and then do the right things in post processing. If you go for jpg and auto, the camera is making choices for you and those might simply be the wrong choices for what you want to achieve.
There is no right or wrong. Generally the sky is not the most important thing. So cameras will expose for the subject below the sky. In a high dynamic range situation (outside scene with sunlight), that means over exposing the photo. The Fuji will actually under expose, which is a good thing. But then it immediately throws the baby away with the bathwater with the one size fits all tone curve it applies to produce the jpg. Try shooting raw + jpg. It's usually fixable in the raw version if the jpg is over exposed.
With raw, you'd expose such that the sky has no blown out highlights. And then in post you adjust your tone mapping to bring up the darker parts without destroying the sky. There are many different ways of doing tone mapping and a gazillion ways you can configure that. That's because different scenes call for different ways to deal with over/under exposed areas.
If you expose correctly, use the aperture appropriately for the lens, scene, direction of light, pick the right lens, etc. you can control the outcome. But it requires knowing and understanding how all of that interacts. You can get very different results for the same scene just by fiddling with aperture and exposure.
The mark of a good photographer is that they don't spend a lot of time in post and instead switch lenses to deal with different scenes. They know how it's going to come out before they click the shutter. It used to be that they wouldn't even see the end results until after developing the film. So, they'd be measuring light and calculating optimal aperture and exposure settings given the scene, light, and lens. Modern cameras make this a lot more interactive and easy. If it looks alright in the digital view finder, it probably is alright. If you have an SLR, pay attention to the exposure indicator.
If you are not interested in doing or learning that, stick to your smart phone. The lens and sensor are not amazing but the camera AI is probably a lot better than what comes with your bigger camera and it will compensate by doing smarter things in post processing.
Its funny how the best camera changes. Most of my photos used to be nature or random trinkets throughout life. Now nearly all my photos are of my little girl.
I care way less about strict objective/subjective quality in comparisons and more about which one chooses the fastest shutter speed.
You can have the best sensor, colour science and dynamic range in the world but if there is movement blur its unusable.
My pixel has been okay at this but I'm apprehensive about how to find good comparisons when I need to replace this phone.
The part he didn't mention is interpolation at the low end "specs are mere suggestions" end of things. I have a backup Android phone - a true "brand X" type of thing, vanilla android, bought at a garage sale. Nice enough phone, but claims a 40MP camera. The merest glance at a picture taken by it shows it has an ordinary-for-its-time 13MP camera in it and the pictures are interpolated to 40MP.
Hopefully the camera doesn't upscale and then downscale again if told so save at its actual native-ish resolution.
> in the darkness, your camera will need to use a longer shutter speed
the alternative, which many smartphone cameras do now, is to capture a burst of many photos of a short shutter speed and then combine them in software. For static things, this is equivalent to a longer shutter speed (with the additional advantage of not blowing out the highlights), and for moving things, we can filter in software to avoid smearing them out.
To expand on the HDR example: There’s this interesting lecture series about computational photography by Marc Levoy, who worked on the earlier Pixel cameras: https://youtube.com/watch?v=y7HrM-fk_Rc
From what I remember, the core thesis is “take a lot of pictures and take the best parts”, which works for a surprising number of cases.
As a hobbyist photographer that still likes using a nice phone camera, the thing that is most important to me is that the phone camera accurately reflects skin tones in photos of people. The only things I'm really photographing with my phone are snapshots of myself and my family and friends doing things together. For everything else I use a real camera. I think the current generation iPhones are pretty good at this and I don't really need it to get any better than it already is, if I want to take a serious photo I will go use a serious camera.
the main issue is when you blow the image up, the details in the highlights and shadows don't hold up, you need to study Chroma subsampling to understand this. Sensor size is still important, but they're getting closer
It's not the Chroma subsampling, it's the agressive de-noising removing the detail (noise is technically 'detail' you don't normally want).
410/411/422 is the least of the problems. If it was just that, it'd largely just be compression artifacts around red/blue things like you often see on streaming / TV new text banners at the bottom. i.e. things like Stop signs, etc...
Solid overview, but it underweights what's arguably the single biggest determinant of whether a phone camera produces a "good" photo: ISP tuning. The post frames software processing as a short list of named features (HDR, Night Sight, AI moon shots), when in reality almost every pixel has already been through dozens of tuned pipeline stages before any of those "features" run.
The ISP pipeline that turns raw Bayer data into a JPEG includes black level correction, lens shading correction, defect pixel correction, demosaic, auto white balance, color correction matrix, tone mapping, multi-stage noise reduction (raw domain, luma/chroma in YUV, temporal across frames), sharpening, local contrast, and multi-frame fusion. Each stage has dozens of parameters and most of them are scene-dependent.
"Tuning" means months in a light booth shooting color charts at every CCT from roughly 2300K to 7500K, then more months outdoors capturing skin tones across ethnicities, foliage, sky at different times of day, neon, candles, fluorescent shop lighting, mixed lighting. Every sensor + lens module gets its own calibration. Then you tune the perceptual layer on top: skin tone preservation, sky and foliage segmentation feeding AWB, AE metering weights with face priors, highlight roll-off, the chroma vs luma noise tradeoff across frequency bands.
This is why two phones with the same Sony IMX sensor look completely different. Sony ships a reference tuning. Apple and Google throw it away. Pixel phones famously have small sensors and beat phones with 2-3x the sensor area on real-world output, almost entirely on the strength of ISP tuning and computational photography stacked on top.
The headcount is genuinely large. Apple and Google each have camera organizations in the hundreds, with a substantial fraction doing nothing but tuning: color scientists, ISP tuning engineers, perceptual quality engineers running blind A/B against competitor output. It's why nobody else -- not Samsung, not Xiaomi, not the smaller players -- quite matches them even when they buy the same or better sensors. (Disclosure: I run a camera company and we live this problem.)
The sensor size advice in the article is correct but easy to misread. A small sensor with great tuning regularly beats a big sensor with mediocre tuning, and the gap is bigger than people expect.
where can i learn more about what you know (besides LLMs): books, wikis etc. ?
For reference, I have a ok-to-mediumish background in image signal processing.
All that really matters is that it exists. If you really care about the quality of your camera, you're going to want to get a dedicated camera. For everyone else (i.e. basically everybody except photographers), literally any phone camera is as good as another.
It's possible to save RAW files (mostly) unprocessed with iPhones, either via built-in functionality (Pros) or via apps like Halide.
But the aggressiveness of the de-noising in the native JPG/HEIF images otherwise is really unfortunate if you want to look at the images on a screen larger than the phone's screen. The amount of detail lost (other than in areas like people's faces where the phone knows to specialise) can be very considerable.
I'd really like a way to dial that aggressiveness down a fair bit, even at the cost of more noise/grain and larger file size (through less compression due to the extra noise).
Another thing is the amount of lens flare you can get when shooting at the sun for sunsets/rises, etc or other large bright light sources. With very small lens elements, from a physics perspective it's understandable that suppressing the reflections and inter-reflections is very difficult on such a small surface area (even with special coatings to reduce the fresnel reflection ratios), but if you care about image quality and wanting to look at images on screen larger than the phone which took them, larger format cameras still have some benefit despite their larger and heavier size and therefore inconvenience (looks at 5D Mk IV on shelf).
Do you know Lumina [1]? It has a setting that turns off most post-processing, resulting in very natural-looking pictures. It has more features than I need, and it's free.
[1] https://apps.apple.com/us/app/lumina-manual-camera/id1617117...
Wow, this is a nice camera app. I usually stick with the stock iOS camera but this is a tangible upgrade. Generous to offer it for free.
Ooh, this looks really good. I use OpenCamera on Android but it's pretty limited... I wonder if there's anything like Lumina there.
I wish there was a middle ground between what Android/Pixel camera saves as raw, and the in-camera JPEG. Sometimes I have a few quibbles with the JPEG and what I'd like to do is edit the raw file, but starting from something close to the JPEG. Unfortunately what you get as a starting point from raw is hideous, and it's never clear how to begin. I don't think I've ever got an acceptable result trying to edit raw photos from my Pixel.
> and it's never clear how to begin
Start by adjusting the black levels and the exposure. This is where the histogram can help to visualize how much you are adjusting (if you have one). As the exposure goes up, you can adjust the highlights to recover some of the areas trying to blow out. As you pull the black level down, you can recover some of the details getting crushed using with the shadows adjustment. You can then adjust the contrast/saturation/warmth/tint as needed. The order of adjustments in the iOS Photos editor pretty much follow that order.
In other words, you want either your camera app to select the initial tweaks for you to be able continue in the external editor (not going to happen, RAW editing software is incompatible by design), or your editing software to select the initial tweaks that "look good" (that depends on your software). In RAW mode, Google Camera's output is photometrically correct, even if it stacks multiple frames or denoises it. Which is the only way to do it that makes sense, any other RAW camera app or actual dedicated camera does this the same way.
Or you could provide the RAW and the JPEG and it would start you off at a point that most closely matches the JPEG?
That's exactly what I'm talking about, how do you imagine that working? Metadata is not compatible by design, because processing pipelines are all subtly different and your result will always look different in your editor. Trying to match some basic parameters with the JPEG is possible and some RAW software can do that, but the result is going to be subtly different for the same reason.
Search over all values of all parameters and choose the ones the minimise the mean square error of pixel values.
Obviously that will be slow, so probably do some kind of gradient descent, or perhaps depending on what the parameters are there may be a closed-form solution, I don't know.
Yes the result will be subtly different but it's just a starting point.
Isn't that what the "auto" button does, and then you can tweak from there?
I've never had to write such software but in my imagination there is the sensor data, potentially from several exposures, and some static data about the camera, and a list of edits and parameters that the photo app is using to produce the in-camera JPEG. And I just want a way to intervene in that list of edits and parameters to produce my own result. There must be SOME way to do this otherwise how do I edit raws from my real camera? The starting point for camera raw in my photo editor always looks great if the file came from my camera, and always looks ghastly if it came from my mobile.
For Android, you can sort of get some of this with Snapseed. I occasionally use it, and it's "ok". I'm more frustrated by the fact that my preferred RAW editor (DxO) doesn't handle Android's DNG files. For me, at least, editing raw images on a phone screen is just not tolerable.
It's strange that in the age of AI, denoisers are still so bad. It's basically impossible to photograph snowing in the winter because the denoiser will remove 90% of the snowflakes. Machine learning models are already used for denoising ray traced graphics with substantially improved results, so why is it that cameras aren't using ML denoisers yet? At least for still images. Or do they perhaps already use them, only the quality is still bad for unknown reasons?
(As someone who worked closely with pathtracing renderers and de-noisers, I think I can answer this :) )
It's mostly because in the VFX/CG space for ray tracing/path tracing de-noisers, they almost always rely on extra outputs/AOVs of things like 'albedo' (diffuse reflectance), normal / world position, etc, to help guide them in many cases.
So they often can 'cheat' a bit, and know where the edges of things are (because say the object ID AOV changes - minus pixel filtering, which complicates things a bit).
They can also 'cheat' in other ways, by mixing back in some of the diffuse texture detail that the denoiser might have removed from the 'albedo' AOV channel.
Cameras don't really have anything to guide them, so they have to guess. And often, they seem to use very primitive methods like bi-lateral filters (or at least things which look very similar), to try and guide them, but it doesn't work very well.
Portrait cameras on phones can use depth sensors a bit to help if the camera has them, but for things like hair strands, it doesn't really work, and is mostly useful for fake-depth-of-field depth-based blurring.
Yeah, but surely ML models would at least work better than analytic algorithms. After all, when looking at a noisy picture, our brain is pretty good at distinguishing detail from noise, so it's not clear to me why an ML model couldn't have denoising performance similar to the human brain, even if it doesn't match the "cheating" denoisers used in ray tracing.
It would probably help you to compare what you can do on a phone vs what you can do with desktop software (Lightroom/Photoshop, DxO, Topaz, CaptureOne, etc). It's generally quite good, with the exception of challenging liminal areas (e.g. hair, foliage).
Fwiw, Topaz -- which I have a license for but essentially never use -- has pretty incredible denoising & upsizing features (for both photo & video), but to get the optimal quality output you offload the processing to their cloud infra (and buy credits from them to pay for it). It's roughly the equivalent of a SWE using a local LLM that's good enough" vs a frontier model that's SOTA but requires a consumption-based subscription.
Interesting, so it seems to be an issue with heavy compute or RAM requirements.
I've got a 61mp camera, and an RX 7900XT. It takes about 15s/picture for DXO to denoise, which is a lot longer than people are willing to wait on a phone to take a photo. Topaz is even slower. A cloud service could be used to do it in post, but someone has to pay for that.
Yet in modern computer games, modern graphics cards denoise a scene in real-time at 60 frames per second using machine learning models [1][2] while doing all the other rendering at the same time. Granted, that's ray tracing, and the resolution is lower, and they technically cheat by using additional information, but it might be that DXO is not optimized very well.
1: https://blogs.nvidia.com/blog/ai-decoded-ray-reconstruction/
2: https://gpuopen.com/amd-fsr-rayregeneration/
Would you be happy if ML detects that you are trying to capture the moon and replaced your shot with a memorized picture of the moon with match ing size and orientation ?
Autoencoders can presumably do this except that it would be operating at levels of patches rather than the entire Moon.
No would be unhappy :(
ML models on some Samsung phones did exactly that.
https://news.ycombinator.com/item?id=35107601
It was probably a result of an explicit company charter but such things can happen with modern ML models quite unintentionally.
> Yeah, but surely ML models would at least work better than analytic algorithms.
Bear in mind, a modern flagship phone doesn't just need to take photos - it also needs to record 4k video at 60fps.
Can't go too hard with the ML when you've only got 1/60th of a second to do it.
I don't think this is even just an AI denoiser problem. Rain effectively looks like haze for the parts of the scene that are at sufficiently long distances. Therefore it's difficult to use dehaze with rainy scenes. The harder the rain is falling, the more limited your ability to dehaze. It's much the same with denoise. You won't be denoising at 100% in Topaz if it's dark and raining hard.
One day smartphone cameras might get there, but right now, the sensor technology isn't there yet. The problem isn't merely noise. It's that rain and snow are moving about in the scene, which means that the camera can't just do its usual trick of taking multiple exposures.
AI denoisers are nice, but they aren't strictly necessary for ILCs. Even those full-frame cameras without IBIS are able to take pictures of nighttime snow just fine.
Are we still talking about smartphone cameras? If yes, apps already heavily rely on much more advanced computational photography than your average photo editor can do, including but not limited to ML denoisers. The problem is that such apps are typically optimized for the "average case" and are as automated as possible, so they either remove snow, rain, and haze intentionally, or lose small moving particles as the result of stacking. That said, snow and rain are usually possible to capture in the apps that attempt to determine the scene type or have specific modes.
Would you have an example of an app which can photograph snow and rain?
The post does a great job at explaining some of the details consumers often overlook. Especially in regards to sensor size, light and accutance.
The best camera is the one you have with you. And this is why smartphones are so great, but the author also does a great job of expressing the limitations and problems.
I've worked as a professional photographer and videographer when between work. I've owned pro Nikon, canon and currently roll with some bonkers expensive leica gear. What makes smartphones so special as cameras is how easy they are taking unobtrusive photos of friends, family and kids. There are few things more frightening to me than a kid charging my way to rip the leica out of my hands because it looks so interesting. They don't do that with smartphones. Smartphones are especially stealthy because it's not clear what the user is doing with them. They could be just browsing the web or whatever. Are they taking video? Or a photo ect.
> The best camera is the one you have with you.
This is why I ended up picking up an (admittedly quite expensive) Ricoh GR IV. It's tiny enough to take with me everywhere, has a modern APS-C sensor and great IBIS.
So keen on one of these! Hard to beat on a performance/quality to size ratio.
Lens for image quality and sensor size and density for resolution, but we hit pretty hard limits on those a long time ago. Software on top of that has been the major differentiator for quite some time. Exposure stacking and intelligent detail control produce more improvement for less investment than a super-complex lens assembly or exotic sensor. Though it brings its own risks.
https://techcrunch.com/2018/10/22/the-future-of-photography-...
Not to say there is no movement on the other fronts. Glass was pushing for a crazy anamorphic lens and far larger sensor that would have been a serious improvement, but I don't know if it went anywhere.
https://techcrunch.com/2022/03/22/glass-rethinks-the-smartph...
I wish "real" camera companies were more aggressive about offering computational photography post-processing, at least as an option. I've gotten spoiled by an iPhone. Despite my Sony and Fuji having huge wonderful lenses, I am pretty disappointed when using a dedicated camera and interior lighting leads to slight blur, or a cloudy day produces washed-out skies.
My theory is that doing so would cannibalize their user base.
I think that those who buy a pro camera nowadays do it because they care about photography itself. For that public post-processing is a touchy subject: besides the philosophical aspect of "is this a true representation of the moment I captured?", there's a real chance that their work will be under-appreciated or rejected altogether if their camera is known as "the one that retouches your photo automatically".
I'm not sure that Nikon's sales would improve that much if they offered auto-post-processing (is that something amateurs a tively want?), but I can imagine that their current customers would be unhappy.
> I'm not sure that Nikon's sales would improve that much if they offered auto-post-processing (is that something amateurs a tively want?)
Ah, but what if instead they could get that $700 0.3kg lens you own to perform as well as that $2000 1kg lens you don't own?
If you shoot manual and raw, there are ways you can fix those things. But you need to understand what you want to achieve before you take the photo and then do the right things in post processing. If you go for jpg and auto, the camera is making choices for you and those might simply be the wrong choices for what you want to achieve.
There is no right or wrong. Generally the sky is not the most important thing. So cameras will expose for the subject below the sky. In a high dynamic range situation (outside scene with sunlight), that means over exposing the photo. The Fuji will actually under expose, which is a good thing. But then it immediately throws the baby away with the bathwater with the one size fits all tone curve it applies to produce the jpg. Try shooting raw + jpg. It's usually fixable in the raw version if the jpg is over exposed.
With raw, you'd expose such that the sky has no blown out highlights. And then in post you adjust your tone mapping to bring up the darker parts without destroying the sky. There are many different ways of doing tone mapping and a gazillion ways you can configure that. That's because different scenes call for different ways to deal with over/under exposed areas.
If you expose correctly, use the aperture appropriately for the lens, scene, direction of light, pick the right lens, etc. you can control the outcome. But it requires knowing and understanding how all of that interacts. You can get very different results for the same scene just by fiddling with aperture and exposure.
The mark of a good photographer is that they don't spend a lot of time in post and instead switch lenses to deal with different scenes. They know how it's going to come out before they click the shutter. It used to be that they wouldn't even see the end results until after developing the film. So, they'd be measuring light and calculating optimal aperture and exposure settings given the scene, light, and lens. Modern cameras make this a lot more interactive and easy. If it looks alright in the digital view finder, it probably is alright. If you have an SLR, pay attention to the exposure indicator.
If you are not interested in doing or learning that, stick to your smart phone. The lens and sensor are not amazing but the camera AI is probably a lot better than what comes with your bigger camera and it will compensate by doing smarter things in post processing.
Its funny how the best camera changes. Most of my photos used to be nature or random trinkets throughout life. Now nearly all my photos are of my little girl.
I care way less about strict objective/subjective quality in comparisons and more about which one chooses the fastest shutter speed. You can have the best sensor, colour science and dynamic range in the world but if there is movement blur its unusable.
My pixel has been okay at this but I'm apprehensive about how to find good comparisons when I need to replace this phone.
The part he didn't mention is interpolation at the low end "specs are mere suggestions" end of things. I have a backup Android phone - a true "brand X" type of thing, vanilla android, bought at a garage sale. Nice enough phone, but claims a 40MP camera. The merest glance at a picture taken by it shows it has an ordinary-for-its-time 13MP camera in it and the pictures are interpolated to 40MP.
Hopefully the camera doesn't upscale and then downscale again if told so save at its actual native-ish resolution.
My DroidX from 2010 took better pics than most modern phones. Zooming in on those photos look better! Crazy
> in the darkness, your camera will need to use a longer shutter speed
the alternative, which many smartphone cameras do now, is to capture a burst of many photos of a short shutter speed and then combine them in software. For static things, this is equivalent to a longer shutter speed (with the additional advantage of not blowing out the highlights), and for moving things, we can filter in software to avoid smearing them out.
I sometimes use the 200MP mode on my phone - it does render more detail in images and sometimes that's what I want.
To counter the unnatural look of noise reduction I often add a film grain effect.
Film grain is great for dealing with crappy noise reduction, but I found no good fix for oversharpening yet. Gaussian blur doesn't do it.
Yes, good point. When I look at prints from the 200MP files I like the amount of detail, but the sharpening is quite obvious.
To expand on the HDR example: There’s this interesting lecture series about computational photography by Marc Levoy, who worked on the earlier Pixel cameras: https://youtube.com/watch?v=y7HrM-fk_Rc
From what I remember, the core thesis is “take a lot of pictures and take the best parts”, which works for a surprising number of cases.
Any modern flagship has good enough camera capabilities, as long as it has okay low light performance that’s enough for me.
Having tried the iPhone Pro lenses, while impressive, the sensor size is never going to match a full-frame mirrorless.
As a hobbyist photographer that still likes using a nice phone camera, the thing that is most important to me is that the phone camera accurately reflects skin tones in photos of people. The only things I'm really photographing with my phone are snapshots of myself and my family and friends doing things together. For everything else I use a real camera. I think the current generation iPhones are pretty good at this and I don't really need it to get any better than it already is, if I want to take a serious photo I will go use a serious camera.
Light your subject
the main issue is when you blow the image up, the details in the highlights and shadows don't hold up, you need to study Chroma subsampling to understand this. Sensor size is still important, but they're getting closer
It's not the Chroma subsampling, it's the agressive de-noising removing the detail (noise is technically 'detail' you don't normally want).
410/411/422 is the least of the problems. If it was just that, it'd largely just be compression artifacts around red/blue things like you often see on streaming / TV new text banners at the bottom. i.e. things like Stop signs, etc...
Works instantly the moment I tap it
Solid overview, but it underweights what's arguably the single biggest determinant of whether a phone camera produces a "good" photo: ISP tuning. The post frames software processing as a short list of named features (HDR, Night Sight, AI moon shots), when in reality almost every pixel has already been through dozens of tuned pipeline stages before any of those "features" run.
The ISP pipeline that turns raw Bayer data into a JPEG includes black level correction, lens shading correction, defect pixel correction, demosaic, auto white balance, color correction matrix, tone mapping, multi-stage noise reduction (raw domain, luma/chroma in YUV, temporal across frames), sharpening, local contrast, and multi-frame fusion. Each stage has dozens of parameters and most of them are scene-dependent.
"Tuning" means months in a light booth shooting color charts at every CCT from roughly 2300K to 7500K, then more months outdoors capturing skin tones across ethnicities, foliage, sky at different times of day, neon, candles, fluorescent shop lighting, mixed lighting. Every sensor + lens module gets its own calibration. Then you tune the perceptual layer on top: skin tone preservation, sky and foliage segmentation feeding AWB, AE metering weights with face priors, highlight roll-off, the chroma vs luma noise tradeoff across frequency bands.
This is why two phones with the same Sony IMX sensor look completely different. Sony ships a reference tuning. Apple and Google throw it away. Pixel phones famously have small sensors and beat phones with 2-3x the sensor area on real-world output, almost entirely on the strength of ISP tuning and computational photography stacked on top.
The headcount is genuinely large. Apple and Google each have camera organizations in the hundreds, with a substantial fraction doing nothing but tuning: color scientists, ISP tuning engineers, perceptual quality engineers running blind A/B against competitor output. It's why nobody else -- not Samsung, not Xiaomi, not the smaller players -- quite matches them even when they buy the same or better sensors. (Disclosure: I run a camera company and we live this problem.)
The sensor size advice in the article is correct but easy to misread. A small sensor with great tuning regularly beats a big sensor with mediocre tuning, and the gap is bigger than people expect.
where can i learn more about what you know (besides LLMs): books, wikis etc. ? For reference, I have a ok-to-mediumish background in image signal processing.
All that really matters is that it exists. If you really care about the quality of your camera, you're going to want to get a dedicated camera. For everyone else (i.e. basically everybody except photographers), literally any phone camera is as good as another.
“The best camera in the world is the one you have with you”
isn't that orthogonal to the topic though? it would still be nice to improve the image quality of the camera you have with you as much as possible
One that I can physically turn off with a switch.