Video: Harmonise video annotation UI with time series

[abdullah-ukaea]

This PR aligns the video annotation UI with the time series view by replacing the floating icon toolbar with a clear right-side annotation toolbar and introducing View and Edit modes.

• View mode: users can zoom, pan, and select existing annotations.
• Edit mode: users can also move and reshape existing annotations.
• Edit mode + Ctrl: users can draw new annotations using the selected tool, including over existing annotations.

Users can switch modes using the toolbar or the E keyboard shortcut. Existing video features such as propagation, hiding annotations, class selection, and instance tracking remain unchanged.

Closes issue #272

Video view before PR

Video view after PR

Time series view

[abdullah-ukaea]

@wk9874 @jblake42 Hi guys, please give this PR a look and let me know what you think.

Here is the to do list:

• User functional test (Matt)
• UI developer functional test (Josh)
• Code review (Josh)

[abdullah-ukaea]

To Do:

• Get Video: Fix race condition during rapid video frame navigation #333 merged into dev
• Merge dev into this branch video/general-ui-improvement
• Add feature to disable annotations during frame loading (when a user clicks next to go to next frame for high latency data loaders)

[wk9874]

Overall looks great! Nice to see the two views being harmonized together. A few points from my functionality test:

1. When loading a new frame, the existing frame is no longer overlayed with a grey box along with the loading wheel. Eg compare the video here: Video: Fix race condition during rapid video frame navigation #333 (comment) to the video below. Think it looked nicer before!

toktagger_loading.mp4

2. I personally think that one of the tools on the right hand side should be auto selected for the user on first load, and its not possible to have no tool selected (basically how it was before harmonization). I think its more intuitive to the user to see something drawn on the screen instead of a popup asking you to select a tool. (But this new way is consistent with the current time series implementation, so happy for this to remain as is and raise an issue to change both later. Or if you / @jblake42 think its better the way it is then happy to leave it)
3. Related to 2 - selecting a tool, then going to View mode, then back to Edit mode, deselects the tool. I can see this getting quite annoying for the user having to constantly reselect the tool every time they switch modes. I think it should just put you back into the mode you were in last time you were in Edit mode. But again this is consistent with time series atm, so happy to raise an issue and tackle separately
4. For 'Add feature to disable annotations during frame loading', while the buttons now do become greyed out, I can still draw:

toktagger_draw_while_loading.mp4

5. When going to next sample, the help box will pop back up if in edit mode. Can this be suppressed? Dont need to be told the info on every navigation :)
6. Bounding boxes & polygons can be deleted in View mode. That shouldn't be the case - I think the ideal scenario is that the pop up still appears in View mode if the item is clicked, but the delete button is disabled, Also mentioned in [WIP] Video: xy point improvements  #322 but should be addressed here
7. This one's slightly more of a feature request, so happy for it to go into an issue if you don't want to tackle it now. I think the 'right click to change mode' feature should also support right clicking an existing annotation to change its class label. Ie if I've drawn a bounding box, then realise my class label was set to 'Major UFO' instead of 'Minor UFO', I want to be able to change that label without redrawing the box completely

[abdullah-ukaea]

Overall looks great! Nice to see the two views being harmonized together. A few points from my functionality test:

1. When loading a new frame, the existing frame is no longer overlayed with a grey box along with the loading wheel. Eg compare the video here: Video: Fix race condition during rapid video frame navigation #333 (comment) to the video below. Think it looked nicer before!

toktagger_loading.mp4
2. I personally think that one of the tools on the right hand side should be auto selected for the user on first load, and its not possible to have no tool selected (basically how it was before harmonization). I think its more intuitive to the user to see something drawn on the screen instead of a popup asking you to select a tool. (But this new way is consistent with the current time series implementation, so happy for this to remain as is and raise an issue to change both later. Or if you / @jblake42 think its better the way it is then happy to leave it)
3. Related to 2 - selecting a tool, then going to View mode, then back to Edit mode, deselects the tool. I can see this getting quite annoying for the user having to constantly reselect the tool every time they switch modes. I think it should just put you back into the mode you were in last time you were in Edit mode. But again this is consistent with time series atm, so happy to raise an issue and tackle separately
4. For 'Add feature to disable annotations during frame loading', while the buttons now do become greyed out, I can still draw:

toktagger_draw_while_loading.mp4
5. When going to next sample, the help box will pop back up if in edit mode. Can this be suppressed? Dont need to be told the info on every navigation :)
6. Bounding boxes & polygons can be deleted in View mode. That shouldn't be the case - I think the ideal scenario is that the pop up still appears in View mode if the item is clicked, but the delete button is disabled, Also mentioned in Video: xy point improvements  #322 but should be addressed here
7. This one's slightly more of a feature request, so happy for it to go into an issue if you don't want to tackle it now. I think the 'right click to change mode' feature should also support right clicking an existing annotation to change its class label. Ie if I've drawn a bounding box, then realise my class label was set to 'Major UFO' instead of 'Minor UFO', I want to be able to change that label without redrawing the box completely

Thank you for the functional testing of this branch I appreciate it! @wk9874

Regarding your points:

1. I have been experimenting with grey box on / off as it is trivial to change (I just make the div transparent with 1 line) and to be honest I think I disagree with you. I find the grey box to look "glitchy" and not very aesthetic. Currently a user sees a spinner in the centre of the frame to indicate they are still in a frame loading state, I think this is enough. Perhaps @jblake42 can make the final decision here.
2. Auto selection of tools from the beginning of an annotation session is interesting and I guess my first question is how do you decide what tool to select from the beginning? If you simply hard code to always use bounding box by default, annotators using polygon or point annotations still have to select there tool anyway. Perhaps in the future we can add a "power user" optional setting where you can select what tool to autoselect for every sample in video view? But again to be consistent with time series I will only make this change for video if we do it for time series also. @jblake42 let me know what you think.
3. Agreed and done in this PR
4. Good catch, I have now patched this in this PR
5. Yep I also found the help box popping up to be quite annoying. I have disabled it to automatically popup it now opens via a mouse click manually.
6. I have now fixed this, in this PR. Now in view mode the annotation popup delete button is disabled. It still works the same in edit mode.
7. So I think I know how to do this, but it is non trivial to implement. I would prefer to create an issue for this if there isn't one already and I will tackle this in a future PR.

[wk9874]

Have had another test, all looks good!

For 7, raised #361

Happy for 1 & 2 to stay as they are

Tested 3, 4, 5, 6 are fixed

Only other comment - should we put the annotation list under the image, in the same format as we have it on Time Series & Profile 2D? Ie should this:

Be made to look more like this:

I would still want the delete & jump to buttons though - in fact this could be useful to add to the time series & profile2D modes as well... (different PR?)

[wk9874]

Raised #362 for that last point

[abdullah-ukaea]

This PR is ready for a code review @jblake42

[jblake42]

This looks great - functionality is as expected! Only some small suggestions

[abdullah-ukaea]

@jblake42 ready for re-review