Skip to content

Latest commit

 

History

History
183 lines (173 loc) · 8.11 KB

README.md

File metadata and controls

183 lines (173 loc) · 8.11 KB
Raw

FlowNet2

FlowNet 2.0: Evolution of Optical Flow Estimation with Deep Networks

Abstract

The FlowNet demonstrated that optical flow estimation can be cast as a learning problem. However, the state of the art with regard to the quality of the flow has still been defined by traditional methods. Particularly on small displacements and real-world data, FlowNet cannot compete with variational methods. In this paper, we advance the concept of end-to-end learning of optical flow and make it work really well. The large improvements in quality and speed are caused by three major contributions: first, we focus on the training data and show that the schedule of presenting data during training is very important. Second, we develop a stacked architecture that includes warping of the second image with intermediate optical flow. Third, we elaborate on small displacements by introducing a sub-network specializing on small motions. FlowNet 2.0 is only marginally slower than the original FlowNet but decreases the estimation error by more than 50%. It performs on par with state-of-the-art methods, while running at interactive frame rates. Moreover, we present faster variants that allow optical flow computation at up to 140fps with accuracy matching the original FlowNet.

Results and Models

Models Training datasets FlyingChairs Sintel (training) KITTI2012 (training) KITTI2015 (training) Log Config Download
clean final EPE Fl-all EPE
FlowNet2CS FlyingChairs 1.59 - - - - - log Config Model
FlowNet2CS Flying Chairs + FlyingThing3d subset - 1.96 3.69 3.50 28.28% 8.23 log Config Model
FlowNet2CSS FlyingChairs 1.55 - - - - - log Config Model
FlowNet2CSS Flying Chairs + FlyingThing3d subset - 1.85 3.57 3.13 25.76% 7.72 log Config Model
FlowNet2CSS-sd Flying Chairs + FlyingThing3d subset + ChairsSDHom - 1.81 3.69 2.98 25.66% 7.99 log Config Model
FlowNet2 FlyingChairs 1.15 - - - - - log Config Model
FlowNet2 FlyingThing3d subset - 1.78 3.31 3.02 25.18% 8.02 log Config Model
Models Training datasets ChairsSDHom Log Config Download
Flownet2sd ChairsSDHom 0.37 log Config Model

Citation

@inproceedings{ilg2017flownet,
  title={Flownet 2.0: Evolution of optical flow estimation with deep networks},
  author={Ilg, Eddy and Mayer, Nikolaus and Saikia, Tonmoy and Keuper, Margret and Dosovitskiy, Alexey and Brox, Thomas},
  booktitle={Proceedings of the IEEE conference on computer vision and pattern recognition},
  pages={2462--2470},
  year={2017}
}