As we get a new set of objects each frame anyway, we
do not gain anything by keeping the modifier constant.
This helps with capturing when switching your setup a
bit, e.g. from ingame to desktop or from X11 to wayland.
Signed-off-by: Mark Thompson <sw@jkqxz.net>