Awesome-Optical-Flow

This is a list of awesome articles about optical flow and related work. Click here to read in full screen.

The table of contents is on the right side of the "README.md".

Recently, I write A Survey on Future Frame Synthesis: Bridging Deterministic and Generative Approaches, welcome to read.

Optical Flow

Supervised Models

Time	Paper	Repo
CVPR24	MemFlow: Optical Flow Estimation and Prediction with Memory	MemFlow
CVPR23	DistractFlow: Improving Optical Flow Estimation via Realistic Distractions and Pseudo-Labeling
CVPR23	Masked Cost Volume Autoencoding for Pretraining Optical Flow Estimation	FlowFormerPlusPlus
NeurIPS22	SKFlow: Learning Optical Flow with Super Kernels	SKFlow
ECCV22	Disentangling architecture and training for optical flow	Autoflow
ECCV22	FlowFormer: A Transformer Architecture for Optical Flow	FlowFormer
CVPR22	Learning Optical Flow with Kernel Patch Attention	KPAFlow
CVPR22	GMFlow: Learning Optical Flow via Global Matching	gmflow
CVPR22	Deep Equilibrium Optical Flow Estimation	deq-flow
ICCV21	High-Resolution Optical Flow from 1D Attention and Correlation	flow1d
ICCV21	Learning to Estimate Hidden Motions with Global Motion Aggregation	GMA
CVPR21	Learning Optical Flow from a Few Matches	SCV
TIP21	Detail Preserving Coarse-to-Fine Matching for Stereo Matching and Optical Flow
ECCV20	RAFT: Recurrent All Pairs Field Transforms for Optical Flow	RAFT
CVPR20	MaskFlownet: Asymmetric Feature Matching with Learnable Occlusion Mask	MaskFlownet
CVPR20	ScopeFlow: Dynamic Scene Scoping for Optical Flow	ScopeFlow
TPAMI20	A Lightweight Optical Flow CNN - Revisiting Data Fidelity and Regularization	LiteFlowNet2

Multi-Frame Supervised Models

Time	Paper	Repo
ECCV24	Local All-Pair Correspondence for Point Tracking
CVPR24	FlowTrack: Revisiting Optical Flow for Long-Range Dense Tracking
CVPR24	Dense Optical Tracking: Connecting the Dots	dot
ICCV23	Tracking Everything Everywhere All at Once	omnimotion
ICCV23	AccFlow: Backward Accumulation for Long-Range Optical Flow	AccFlow
ICCV23	VideoFlow: Exploiting Temporal Cues for Multi-frame Optical Flow Estimation	VideoFlow
ECCV22	Particle Video Revisited: Tracking Through Occlusions Using Point Trajectories	PIPs

Semi-Supervised Models

Time	Paper	Repo
ECCV22	Semi-Supervised Learning of Optical Flow by Flow Supervisor

Data Synthesis

Time	Paper	Repo
ECCV22	RealFlow: EM-based Realistic Optical Flow Dataset Generation from Videos	RealFlow
CVPR21	AutoFlow: Learning a Better Training Set for Optical Flow	autoflow
CVPR21	Learning Optical Flow from Still Images	depthstillation
arXiv21.04	Optical Flow Dataset Synthesis from Unpaired Images

Unsupervised Models

Time	Paper	Repo
ECCV22	Optical Flow Training under Limited Label Budget via Active Learning	optical-flow-active-learning-release
CVPR21	SMURF: Self-Teaching Multi-Frame Unsupervised RAFT with Full-Image Warping	smurf GoogleResearch
CVPR21	UPFlow: Upsampling Pyramid for Unsupervised Optical Flow Learning	UPFlow_pytorch
TIP21	OccInpFlow: Occlusion-Inpainting Optical Flow Estimation by Unsupervised Learning	depthstillation
ECCV20	What Matters in Unsupervised Optical Flow	uflow GoogleResearch
CVPR20	Learning by Analogy: Reliable Supervision from Transformations for Unsupervised Optical Flow Estimation	ARFlow
CVPR20	Flow2Stereo: Effective Self-Supervised Learning of Optical Flow and Stereo Matching

Joint Learning

Time	Paper	Repo
arXiv21.11	Unifying Flow, Stereo and Depth Estimation	unimatch
CVPR21	EffiScene: Efficient Per-Pixel Rigidity Inference for Unsupervised Joint Learning of Optical Flow, Depth, Camera Pose and Motion Segmentation
CVPR21	Feature-Level Collaboration: Joint Unsupervised Learning of Optical Flow, Stereo Depth and Camera Motion

Special Scene

Time	Paper	Repo
CVPR23	Unsupervised Cumulative Domain Adaptation for Foggy Scene Optical Flow	UCDA-Flow
ECCV22	Deep 360∘ Optical Flow Estimation Based on Multi-Projection Fusion
AAAI21	Optical flow estimation from a single motion-blurred image
CVPR20	Optical Flow in Dense Foggy Scenes using Semi-Supervised Learning
CVPR20	Optical Flow in the Dark	Optical-Flow-in-the-Dark

Special Device

Event Camera event-based_vision_resources

Time	Paper	Repo
ArXiv23.03	Learning Optical Flow from Event Camera with Rendered Dataset
ECCV22	Secrets of Event-Based Optical Flow	event_based_optical_flow
ICCV21	GyroFlow: Gyroscope-Guided Unsupervised Optical Flow Learning	GyroFlow

Scene Flow

Time	Paper	Repo
CVPR21	RAFT-3D: Scene Flow Using Rigid-Motion Embeddings
CVPR21	Just Go With the Flow: Self-Supervised Scene Flow Estimation	Just-Go-with-the-Flow-Self-Supervised-Scene-Flow-Estimation
CVPR21	Learning to Segment Rigid Motions from Two Frames	rigidmask
CVPR20	Upgrading Optical Flow to 3D Scene Flow through Optical Expansion	expansion
CVPR20	Self-Supervised Monocular Scene Flow Estimation	self-mono-sf

Applications

Video Synthesis/Generation

Time	Paper	Repo
ECCV24	Clearer Frames, Anytime: Resolving Velocity Ambiguity in Video Frame Interpolation	InterpAny-Clearer
arXiv23.11	MoVideo: Motion-Aware Video Generation with Diffusion Models
CVPR24	FlowVid: Taming Imperfect Optical Flows for Consistent Video-to-Video Synthesis
WACV24	Scale-Adaptive Feature Aggregation for Efficient Space-Time Video Super-Resolution	SAFA
CVPR23	A Dynamic Multi-Scale Voxel Flow Network for Video Prediction	DMVFN
CVPR23	Conditional Image-to-Video Generation with Latent Flow Diffusion Models	LFDM
CVPR23	A Unified Pyramid Recurrent Network for Video Frame Interpolation	UPR-Net
CVPR23	Extracting Motion and Appearance via Inter-Frame Attention for Efficient Video Frame Interpolation	EMA-VFI
WACV23	Frame Interpolation for Dynamic Scenes with Implicit Flow Encoding	frameintIFE
ACMMM22	Neighbor correspondence matching for flow-based video frame synthesis
ECCV22	Improving the Perceptual Quality of 2D Animation Interpolation	eisai
ECCV22	Real-Time Intermediate Flow Estimation for Video Frame Interpolation	RIFE
CVPR22	VideoINR: Learning Video Implicit Neural Representation for Continuous Space-Time Super-Resolution	VideoINR
CVPR22	IFRNet: Intermediate Feature Refine Network for Efficient Frame Interpolation	IFRNet
TOG21	Neural Frame Interpolation for Rendered Content
CVPR21	Deep Animation Video Interpolation in the Wild	AnimeInterp
CVPR20	Softmax Splatting for Video Frame Interpolation	softmax-splatting
CVPR20	Adaptive Collaboration of Flows for Video Frame Interpolation	AdaCoF-pytorch
CVPR20	FeatureFlow: Robust Video Interpolation via Structure-to-Texture Generation	FeatureFlow

Video Inpainting

Time	Paper	Repo
ECCV22	Flow-Guided Transformer for Video Inpainting	FGT
CVPR22	Inertia-Guided Flow Completion and Style Fusion for Video Inpainting	isvi

Video Stabilization

Time	Paper	Repo
CVPR20	Learning Video Stabilization Using Optical Flow	jiyang.fun

Low Level Vision

Time	Paper	Repo
ICCV21	Deep Reparametrization of Multi-Frame Super-Resolution and Denoising	deep-rep
CVPR21	Deep Burst Super-Resolution	deep-burst-sr
CVPR20	Efficient Dynamic Scene Deblurring Using Spatially Variant Deconvolution Network With Optical Flow Guided Training
TIP20	Deep video super-resolution using HR optical flow estimation	SOF-VSR

Stereo and SLAM

Time	Paper	Repo
3DV21	RAFT-Stereo: Multilevel Recurrent Field Transforms for Stereo Matching	RAFT-Stereo
CVPR20	VOLDOR: Visual Odometry From Log-Logistic Dense Optical Flow Residuals	VOLDOR

Before 2020

Classical Estimation Methods

Time	Paper	Repo
IJCAI1981	An iterative image registration technique with an application to stereo vision
AI1981	Determining optical flow
TPAMI10	Motion Detail Preserving Optical Flow Estimation
CVPR10	Secrets of Optical Flow Estimation and Their Principles
ICCV13	DeepFlow: Large Displacement Optical Flow with Deep Matching	Project
ECCV14	Optical Flow Estimation with Channel Constancy
CVPR17	S2F: Slow-To-Fast Interpolator Flow

Others

Time	Paper	Repo
NeurIPS19	Volumetric Correspondence Networks for Optical Flow	VCN
CVPR19	Iterative Residual Refinement for Joint Optical Flow and Occlusion Estimation	irr
CVPR18	PWC-Net: CNNs for Optical Flow Using Pyramid, Warping, and Cost Volume	PWC-Net
CVPR18	LiteFlowNet: A Lightweight Convolutional Neural Network for Optical Flow Estimation	LiteFlowNet
CVPR17	FlowNet 2.0: Evolution of Optical Flow Estimation with Deep Networks	flownet2-pytorch flownet2 flownet2-tf
CVPR17	Optical Flow Estimation using a Spatial Pyramid Network	spynet
ICCV15	FlowNet: Learning Optical Flow with Convolutional Networks	FlowNetPytorch
AAAI19	DDFlow: Learning Optical Flow with Unlabeled Data Distillation	DDFlow
CVPR19	SelFlow: Self-Supervised Learning of Optical Flow	SelFlow
CVPR19	Unsupervised Deep Epipolar Flow for Stationary or Dynamic Scenes	EPIFlow
CVPR18	Unsupervised Learning of Dense Depth, Optical Flow and Camera Pose	GeoNet
ICCV19	RainFlow: Optical Flow under Rain Streaks and Rain Veiling Effect
CVPR18	Robust Optical Flow Estimation in Rainy Scenes
NIPS19	Quadratic Video Interpolation
CVPR19	Depth-Aware Video Frame Interpolation	DAIN
CVPR18	Super SloMo: High Quality Estimation of Multiple Intermediate Frames for Video Interpolation	Super-SloMo
ICCV17	Video Frame Synthesis using Deep Voxel Flow	voxel-flow
CVPR19	DVC: An End-to-end Deep Video Compression Framework	PyTorchVideoCompression
ICCV17	SegFlow: Joint Learning for Video Object Segmentation and Optical Flow	SegFlow
CVPR18	End-to-end Flow Correlation Tracking with Spatial-temporal Attention
CVPR18	Optical Flow Guided Feature: A Fast and Robust Motion Representation for Video Action Recognition	Optical-Flow-Guided-Feature
GCPR18	On the Integration of Optical Flow and Action Recognition
CVPR14	Spatially Smooth Optical Flow for Video Stabilization