Optical flow estimation, supervised and unsupervised ways.
Optical Flow is one of the classic tasks of Image Processing. It consists, from a pair of successive images, to retrieve a map of velocities that occurred between the couple of images. Without additional information about the acquisition scene, it is an ill-posed problem that is classically addressed by Tikhonov regularization in the literature.
In 2015, a first deep neural network, Flownet, is proposed. Flownet and its successors are supervised. Is the case of natural video with rigid moving objects it remains possible to build datasets having a ground truth (Sintel, Kitty).
They provide very good performances but a train set with ground truth is mandatory. In the field of Geoscience for instance, ground truth are not available. For this reason, we are interested by unsupervised approaches (Yu et al. 2016; de Bezenac et al. 2020).
In the task of rain forecast at a short time horizon (Che et al. 2022), or of sea surface height reconstruction (work under submission), ED-DRAP, an architecture based on residual attention, drew very promising results. Second, get inspiration from ED-DRAP to benchmark the standard optical flow datasets.
Third, experiment unsupervised learning on rain precipitation or ocean surface circulation.
