Previous fix in #2547 altered id before they were handed off to
callbacks and v4l2 wont strip its own flags from ids resulting in
invalid ids in the ioctl.
Instead add cleanup section and jump all branches there before looping.
(Jim edit: I'm changing this entire function so that it isn't horrible.)
b822faf73b