Canon autoramp is a mess after C - 2.0.0 upgrade [fixed in A2.2.3]

We tested as good as we can. But there will always be things we didn't find. If you found things that apparently don't work as they should, you can report them here so we can take care of them as soon as possible.
zoom
Posts: 44
Joined: August 27th, 2020, 8:20 am

August 16th, 2022, 5:49 am

Oliver wrote:
August 16th, 2022, 1:35 am
Hmm, that's very interesting. The drop from 2.8 looks like it's at the correct point in time (if you look at that sawtooth shape of the line, the vertical drop is pretty much where it's expected, but the odd thing is that it should have just been 1/3rd of a stop. instead it was 1.33 stops.
It then looks like that all those spikes are exactly that 1 stop (in the opposite direction.
This is the sunrise algorism, the exposure should not decrease backwards.
I just saw that you used the Modified LRT algorithm! (LRT+)
do you remember if you added any adjustment value, or tried to do so? We recently made a change with the Adjustment setting for the Modified LRT Algorithm, and it looks like a bug might have crept in there.
I always use LRT+. This was working fine before 2.0.0

I did not touch anything with the Adjustment setting. I did set the initial exposure one stop lower than the metered number in my camera to avoid any over exposure (I assume this should be maintained during my entire sequence).
Just an explanation for the short interval thought: We have found that it takes quite a while for Canon cameras to be ready for the next shot after finishing the previous one. The calculations for the LRT algorithms take some time, reading and changing camera settings is fast, but not instant. Add a possibly slow card write speed to that, and you could be cutting it close with a 3s interval and 1s shutter speed. Of course we did implement everything robust to these restrictions. But since bugs are always unexpected, that would be the first place I'd have looked without any infos leading me elsewhere.
Again, this interval thing doesn't explain why the shutter speed changed from 0.5 to 1sec backwards during a sunrise.
So, for now, switch to the standard LRT algorithms, and/or try to avoid the adjustment setting if you can.
Are you asking me to try different things instead of fixing the real issue? Earlier you suggested 'EV-meter smoothing algorithm'. Now you ask to switch to standard LRT. I am confused which one is reliable.
Oliver
Posts: 1138
Joined: October 9th, 2018, 4:17 pm

August 16th, 2022, 11:07 am

It sounded like you need a solution ASAP, so my suggestion was to try using the LRT Algorithm until we find and fix the bug with the LRT+ Algorithm. It will take us some time to reproduce and fix the issue, so I was trying to help you find a temporary solution until then. We only made a change with how the adjustment works on the LRT+ algorithm, so that's the most likely place for a bug to occur
Initially I thought you found a bug with our LRT implementation, hence I suggested using the EV-based algorithm, that works completely differently. But since I saw that you used LRT+, I'm now assuming the bug has something to do with the LRT+ adjustment that we worked on a few weeks ago. Therefore I assume LRT will work, so you can also try that, if you prefer an image-based algorithm over the EV meter based algorithm.

again, the short interval is no explanation for what you're seeing - you're clearly seeing a bug in our firmware. I was merely saying that the short interval has a higher chance of being responsible for unexpected behaviour, not that it's definitely the cause of exactly what you were seeing.
Founder & CEO of Foolography, Hardware & Firmware developer.
zoom
Posts: 44
Joined: August 27th, 2020, 8:20 am

August 16th, 2022, 6:35 pm

Oliver wrote:
August 16th, 2022, 11:07 am
Initially I thought you found a bug with our LRT implementation, hence I suggested using the EV-based algorithm, that works completely differently. But since I saw that you used LRT+, I'm now assuming the bug has something to do with the LRT+ adjustment that we worked on a few weeks ago. Therefore I assume LRT will work, so you can also try that, if you prefer an image-based algorithm over the EV meter based algorithm.

It sounds like your put some changes into a final product without some QA cycles, and replies on customer to find issues.

I searched the forums, some one reported the issues with EV-based algorithm as well. I just need a reliable auto ramping means which wont ruin my next vacation again.

As the customers, we cannot really afford trying different things during an expensive vacation. Most of our shots are once in a lifetime opportunity.
Oliver
Posts: 1138
Joined: October 9th, 2018, 4:17 pm

August 16th, 2022, 7:02 pm

Of course we go through QA for everything we work on.
But we support about 70 different cameras, and each have nearly infinite combinations of different settings. Multiply that by the infinite number of possible scenes to photograph, and you'll see it's impossible to test every possible situation that might or might not cause a bug to rear its head. So no matter how much we test, there's always a chance that something slips through our tests. But if a user does experience a bug that we missed in the implementation and testing phases, we're always sorry about that, and do our best to fix it as soon as possible.
Founder & CEO of Foolography, Hardware & Firmware developer.
zoom
Posts: 44
Joined: August 27th, 2020, 8:20 am

August 17th, 2022, 5:38 am

Oliver wrote:
August 16th, 2022, 7:02 pm
and do our best to fix it as soon as possible.
We will see how soon you will fix this serious degrade.
Oliver
Posts: 1138
Joined: October 9th, 2018, 4:17 pm

September 22nd, 2022, 9:14 pm

This is odd - for some reason my post did not get published. I wrote this on the 30th of August:


Hi again.
So, for the past week or so, we've really been racking our brains, trying to really find the root cause of the issue you're seeing. We really had a hard time, as no-one else has ever reported, nor have we ever seen anything like it ourselves.

Today, we finally cracked it, and it is indeed a bug (or to be more specific, it's actually a combination of about several separate bugs) in our Firmware, which only ever comes to light in a rare edge case. At first, we were almost certain that the issue that triggers the failure was the short 3 second interval! However, we found that while this can help bring the issue to light, it’s not actually the root cause.
We did find that this is the reason why you saw this problem, and no-one else has reported anything similar:

- you’re running a sunrise timelapse.
- your ISO day limit was 100
- your cameras NEXT step after ISO 100 would have been a full stop (Lo 1.0), not 1/3rd of a stop!

The 5D Mark IV is by no means a seldom camera, nor is it odd to want to use the lowest native ISO as the limit, though our default limit is the very lowest value - Lo 1.0 or ISO 50 equivalent, so I guess there’s just not that many of our customers shooting sunrises AND changing the default ISO limit. So thank you for reporting this odd case with all these details.

A bug in our firmware remembered the last “NEXT step”, which is never a problem, except when that last next step is different than the actual next step. This is what caused the 1.3EV jump down instead of 0.3EV when the Unleashed finished ramping ISO, and started with the aperture. We implemented it so that want to ramp by 0.3 stops, then check each of the three settings in a row whether we can and want to ramp this setting and by how much. If yes we then check whether we SHOULD (ie check the day limit)and should ramp this setting,. on sunrise, we check ISO first, and realise we can't ramp by 0.3, but only by 1.0 stops. Then we realise that we're at the limit, so we don't ramp iso and move onto aperture. We find out we can ramp 0.3, it's OK to do so, so we do that. Unfortunately, due to the aforementioned bug, we checked whether one value is greater than or equal to another, instead of just greater than, so even after applying the 0.3 stop change to aperture, the two values were then equal, so we also ramped the 1.0 stops from ISO, which we still remembered due to another bug. So this explains the big dip when you reached the ISO limit.

Then, another bug comes into play, which creates the spikes. We made all parts of our implementation as independent of each other as possible - the input for the calculation, the calculation itself including determining which settings to change, and then actually changing the settings happens quite independently, also from the interval itself. This does allow for short intervals, that might cause us to miss an opportunity of changing settings before the next picture is taken. After the calculation, we save how many EV steps are remaining to adjust by, then after adjusting, subtract how many steps we DID adjust by, and save how many steps are remaining that we’ll need to adjust by next time. Unfortunately our bug reset this “remaining” value to 0, and when we got the confirmation by how much we did adjust, we’d suddenly have a remaining value in the opposite direction of the one we’re actually ramping in. We only do the next calculation when the threshold is passed, and that’s why there’s the spike upward, only each time we actually want to ramp downward. Its 1 full stop because of the first bug our “remaining” value was reset to 0, then 0- (-1.3EV) = +1.3EV, and when we wanted to adjust by -0.3EV, we added the two together and got +1.0EV. on the next photo we do ramp down again, and because of the first bug, it’s by 1.3EV again, luckily getting back to our desired value.

This repeats on each adjustment.

We’ve implemented the fix, and will be releasing a new firmware version soon.

That said, while a short interval not uncommon for certain timelapses, and while you're of course allowed to be creative in whatever way you like, I just wanted to note that it is very, very uncommon for holy grail timelapses. So I wanted to offer an idea on how to "save" your ruined timelapses: You could simply ignore 2 out of 3 images, resulting in a 9s interval which is close to the much more commonly used 10s interval. Then make sure you always delete the two over exposed images where the each of those "spikes" in the graph are. This might result in the odd 6 or 3 second difference between two frames, instead of the usual 9s but that should not be be visible in the final result. I think it would be quite easy to do in LRTimelapse, where you can easily spot the problematic two pictures.

Nonetheless, sorry again for the trouble, and thanks for the detailed report and helping us find these hidden bugs.
Founder & CEO of Foolography, Hardware & Firmware developer.
Oliver
Posts: 1138
Joined: October 9th, 2018, 4:17 pm

September 22nd, 2022, 9:17 pm

This has been fixed in Firmware A2.2.3, released a few days ago. It's already available in the iOS app, coming soon in the Android app.
Sorry it has taken a little longer than we had anticipated.
Founder & CEO of Foolography, Hardware & Firmware developer.
zoom
Posts: 44
Joined: August 27th, 2020, 8:20 am

September 23rd, 2022, 4:18 am

>>So I wanted to offer an idea on how to "save" your ruined timelapses: You could simply ignore 2 out of 3 images, resulting in a 9s interval which is close to the much more commonly used 10s interval.

Thanks for the tips. No, I cannot use a 10sec interval. 3sec is already too long to catch up the changing lights and shadows on the buildings and mountains. I usually use 1 sec interval and do a manual adjustment.

This 1 stop spike issue is reproducible with both the regular LRT and the modified LRT+ algorisms as I emailed you another failure yesterday using the regular LRT algorisms (as you know, the forum just started working this afternoon).

As I said in the email, I did another sunrise session with the EV 24h algorism this morning. Unfortunately I got a similar issue. Now it always drops one full stop to the adjusted meter value:
EV24h.jpg
EV24h.jpg (173.76 KiB) Viewed 9298 times
I started with an exposure of iso400, f5.6, 1/10 (one stop lower than the camera metering value). At first, the ISO was ramped from 400 to 100 without any issue, then it stopped ramping(see the yellow line). I made the adjustment to -1.0 manually in the App to lower the exposure. Then the ramping started again, and the problems showed up:

The exposure kept going up about 1.3 stop higher than the adjusted value, following by an one stop sudden drop.

The Red dots are the adjusted -1.0 EV points, and the orange dots are the -2.0 EV points that I set at the two green lines respectively. It supposed to ramp to these red and orange dots within 1/3 stop.

So, basically none of the algorism works so far for an entire year after ver2.0. My sunset stopped ramping when I was in Hawaii last Oct with the beta version. Then the modified LRT+ didn't work during my vacation to Seattle. now It's the regular LRT algorism. And today the EV 24h didn't work either.

I just updated the FW to A2.2.3. The release notes said this version will fix the LRT algorism issues. I assume the fix is for both the regular LRT and the modified LRT+? But it didn't mention the EV24h issue I just posted.

Do the one stop spike in LRT/LRT+ algorisms and the one stop drop issue in EV24h share the same cause?

And by the way, after updating to the new version of firmware, the App still shows a red warning sign (after a restart of the App), and if I go to the firmware, it has the same version even an update button is showing up. But when I click the button, there's no response. Hope this is just a minor UI glitch.
Oliver
Posts: 1138
Joined: October 9th, 2018, 4:17 pm

September 26th, 2022, 10:41 pm

Hi,

It's exactly the same issue. The adjustment part of our algorithms are independent of the input (EV meter or LRT/Image histogram). So yes, I'd expect the same issues on the EV 24h algorithm, and they'll be solved too now. It will behave a bit differently, because we actually use the values of the EV meter to add to/subtract from the "remaining" steps value I talked about, unlike the fixed amount we adjust by on the LRT algorithms. So this might have inverted the order of the two bugs, resulting in a spike down compared to the other Algorithms.

In other words, this is also fixed by A2.2.3!
Founder & CEO of Foolography, Hardware & Firmware developer.
zoom
Posts: 44
Joined: August 27th, 2020, 8:20 am

September 29th, 2022, 6:45 am

Oliver,

I asked you whether this will affect sunsets or not before I went for a very rare colorful sunset with low fogs around the lake.

Here are what you replied in email:

"'In theory it could have happened with sunsets too, though there’s only one case we could think of that could have triggered these bugs."
"But as I wrote, the bug is fixed, so to our knowledge, there are no more bugs in Sunrise, sunset, EV-24h or LRT based algorithms!"

Well, unfortunately my sunset sequence today was ruined again with you new firmware A2.2.3,
iso spikes.png
iso spikes.png (198.82 KiB) Viewed 8978 times
The iso jumped from 250 all the way to the night limit, 1600. Way over exposed (3 stops higher than the camera metering):
Over exposed.JPG
Over exposed.JPG (87.16 KiB) Viewed 8978 times
I stopped the sequence after I found out the issue, then I manually took below picture with the camera's metering to the center:
EVcenter.png
EVcenter.png (188.75 KiB) Viewed 8978 times
But I already lost the peak of the beautiful sunset with the very rare low fogs.

Very very disappointed. Nothing worked for me since last October. This has wasted a lot of my energy, time, especially these once in a lifetime opportunities. I would have enjoyed all of these great opportunities if I did them manually.

Seriously, can I have a refund please? I can mail you back the Unleashed device at my expense. It's really frustrating to struggle with Unleashed for an entire year. I hope this is a reasonable request after all of these pains.
Post Reply