`VideoCapture`: change `CAP_PROP_FOURCC` to fix #22876 #23540

cudawarped · 2023-04-25T04:53:56Z

Address #22876 by adding a CAP_PROP_CODEC_FOURCC to the VideoCapture FFmpeg backend.
Merge with opencv/opencv_contrib#3488

Pull Request Readiness Checklist

See details at https://github.com/opencv/opencv/wiki/How_to_contribute#making-a-good-pull-request

I agree to contribute to the project under Apache 2 License.
To the best of my knowledge, the proposed patch is not based on a code under GPL or another license that is incompatible with OpenCV
The PR is proposed to the proper branch
There is a reference to the original bug report and related work
There is accuracy test, performance test and test data in opencv_extra repository, if applicable
Patch to opencv_extra has the same branch name.
The feature is well documented and sample code can be built with the project CMake

opencv-alalek · 2023-04-25T05:46:04Z

modules/videoio/include/opencv2/videoio.hpp

       CAP_PROP_CODEC_EXTRADATA_INDEX = 68, //!< Positive index indicates that returning extra data is supported by the video back end.  This can be retrieved as cap.retrieve(data, <returned index>).  E.g. When reading from a h264 encoded RTSP stream, the FFmpeg backend could return the SPS and/or PPS if available (if sent in reply to a DESCRIBE request), from calls to cap.retrieve(data, <returned index>).
       CAP_PROP_FRAME_TYPE = 69, //!< (read-only) FFmpeg back-end only - Frame type ascii code (73 = 'I', 80 = 'P', 66 = 'B' or 63 = '?' if unknown) of the most recently read frame.
       CAP_PROP_N_THREADS = 70, //!< (**open-only**) Set the maximum number of threads to use. Use 0 to use as many threads as CPU cores (applicable for FFmpeg back-end only).
+       CAP_PROP_CODEC_FOURCC = 71, //!< 4-character code representing type of the encoded data (the codec used).  This differs from CAP_PROP_FOURCC which can represent additional  information about the codec. e.g. CAP_PROP_FOURCC can return divx, xvid, mp4v etc. for an MPEG-4 video where CAP_PROP_CODEC_FOURCC would return only FMP4 (applicable for FFmpeg back-end only).


There is some mess with:

CAP_PROP_FOURCC =6, //!< 4-character code of codec. see VideoWriter::fourcc .

Both describes the "codec".
We could deprecate the old one, but we need robust and clear solution.

BTW, there are 3 different FOURCC could be used in some pipelines: camera firmware/device level(MJPEG compression), codec(YUV), output of VideoCapture(BGR24).

Its currently confusing as CAP_PROP_FOURCC is a mixture of the codec_id and the codec_tag

/** * Specific type of the encoded data (the codec used). */ enum AVCodecID codec_id; /** * Additional information about the codec (corresponds to the AVI FOURCC). */ uint32_t codec_tag;

this makes sense for a fourcc code as the codec_tag is the fourcc and if it doesn't exist we fallback to generating one from the codec_id. That said the description 4-character code of codec is not correct.

My initial idea for the PR was to simply to generate the fourcc with the codec_id instead of the codec_tag and fall back to codec_tag if there were any issues. This would be clearer but it could break existing code which relies on the existing values.

As different video codecs can have the same codec_tag, e.g. the codec_tag returned for the three test video's below is mp4v

highgui/video/big_buck_bunny.mp4
highgui/video/sample_322x242_15frames.yuv420p.mpeg2video.mp4
highgui/video/sample_322x242_15frames.yuv420p.mjpeg.mp4

when they use MPEG-4, MPEG-2 and MJPG codec respectively. It might be better to remove codec_tag completely and return false (-1) if a fourcc cannot be generated from the codec_id.

@opencv-alalek updated to translate codec_id to fourcc and fallback to codec_tag if this doesn't exist. This seems like a robust solution, the returned value will now "always" (apart from when the codec_id field doesn't exist) be the 4-character code of codec.

cudawarped · 2023-06-14T09:45:51Z

@opencv-alalek Are you happy with the approach in this PR? Forcing CAP_PROP_FOURCC to be the codec if there is a codec_id to be consistent with the property description?

…c_tag

asmorkalov

👍

asmorkalov · 2023-06-20T13:38:09Z

The issue is still there:

[ RUN      ] CUDA_Codec/Video.Reader/8, where GetParam() = (NVIDIA GeForce GTX 1080, "highgui/video/sample_322x242_15frames.yuv420p.libaom-av1.mp4")
[ERROR:[email protected]] global cap_ffmpeg_impl.hpp:1237 open Could not find decoder for codec_id=32797
[ERROR:[email protected]] global cap_ffmpeg_impl.hpp:1286 open VIDEOIO/FFMPEG: Failed to initialize VideoCapture
[ WARN:[email protected]] global cap.cpp:206 open VIDEOIO(FFMPEG): backend is generally available but can't be used to capture by name
[ERROR:[email protected]] global video_decoder.cpp:127 create Video source is not supported by hardware video decoder.
[ERROR:[email protected]] global video_parser.cpp:169 HandleVideoSequence Attempt to configure Nvidia decoder failed!
[ERROR:[email protected]] global video_parser.cpp:83 parseVideoData Call to cuvidParseVideoData failed!
unknown file: Failure
C++ exception with description "OpenCV(4.7.0-dev) /home/alexander/Projects/OpenCV/opencv_contrib/modules/cudacodec/src/video_reader.cpp:212: error: (-2:Unspecified error) Parsing/Decoding video source failed, check GPU memory is available and GPU supports hardware decoding. in function 'internalGrab'
" thrown in the test body.
[  FAILED  ] CUDA_Codec/Video.Reader/8, where GetParam() = (NVIDIA GeForce GTX 1080, "highgui/video/sample_322x242_15frames.yuv420p.libaom-av1.mp4") (66 ms)
[ RUN      ] CUDA_Codec/Video.Reader/9, where GetParam() = (NVIDIA GeForce GTX 1080, "cv/tracking/faceocc2/data/faceocc2.webm")
[       OK ] CUDA_Codec/Video.Reader/9 (145 ms)
[ RUN      ] CUDA_Codec/Video.Reader/10, where GetParam() = (NVIDIA GeForce GTX 1080, "highgui/video/sample_322x242_15frames.yuv420p.mpeg2video.mp4")
Ошибка сегментирования (стек памяти сброшен на диск)

cudawarped · 2023-06-20T14:26:08Z

@asmorkalov See my reply in the other PR opencv/opencv_contrib#3488 (comment)

opencv-alalek reviewed Apr 25, 2023

View reviewed changes

cudawarped force-pushed the add_CAP_PROP_CODEC_FOURCC branch 2 times, most recently from 72b4425 to 88a438e Compare April 25, 2023 14:59

cudawarped changed the title ~~VideoCapture: add CAP_PROP_CODEC_FOURCC to fix #22876~~ VideoCapture: change CAP_PROP_FOURCC to fix #22876 Apr 25, 2023

cudawarped changed the title ~~VideoCapture: change CAP_PROP_FOURCC to fix #22876~~ VideoCapture: change CAP_PROP_FOURCC to fix #22876 Apr 25, 2023

cudawarped mentioned this pull request Apr 30, 2023

cudacodec::VideoReader FORCC update with main #23540 opencv/opencv_contrib#3488

Merged

6 tasks

cudawarped force-pushed the add_CAP_PROP_CODEC_FOURCC branch 2 times, most recently from 1ae8920 to 46c91ad Compare June 13, 2023 14:49

opencv-alalek added test category: videoio RFC labels Jun 14, 2023

opencv-alalek added this to the 4.8.0 milestone Jun 14, 2023

asmorkalov assigned opencv-alalek Jun 16, 2023

cv::VideoCapture: change CAP_PROP_FOURCC to prefer codec_id over code…

024c836

…c_tag

asmorkalov removed the RFC label Jun 20, 2023

asmorkalov self-requested a review June 20, 2023 07:37

asmorkalov assigned asmorkalov and unassigned opencv-alalek Jun 20, 2023

asmorkalov approved these changes Jun 20, 2023

View reviewed changes

cudawarped force-pushed the add_CAP_PROP_CODEC_FOURCC branch from 46c91ad to 024c836 Compare June 20, 2023 12:06

asmorkalov merged commit 61d48dd into opencv:4.x Jun 22, 2023

asmorkalov mentioned this pull request Jul 27, 2023

(5.x) Merge 4.x #24070

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Uh oh!

`VideoCapture`: change `CAP_PROP_FOURCC` to fix #22876 #23540

`VideoCapture`: change `CAP_PROP_FOURCC` to fix #22876 #23540

Uh oh!

cudawarped commented Apr 25, 2023 •

edited by asmorkalov

Loading

Uh oh!

opencv-alalek Apr 25, 2023

Uh oh!

cudawarped Apr 25, 2023

Uh oh!

cudawarped Apr 25, 2023

Uh oh!

cudawarped commented Jun 14, 2023

Uh oh!

asmorkalov left a comment

Uh oh!

asmorkalov commented Jun 20, 2023

Uh oh!

cudawarped commented Jun 20, 2023 •

edited

Loading

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Uh oh!

VideoCapture: change CAP_PROP_FOURCC to fix #22876 #23540

VideoCapture: change CAP_PROP_FOURCC to fix #22876 #23540

Uh oh!

Conversation

cudawarped commented Apr 25, 2023 • edited by asmorkalov Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Pull Request Readiness Checklist

Uh oh!

opencv-alalek Apr 25, 2023

Choose a reason for hiding this comment

Uh oh!

cudawarped Apr 25, 2023

Choose a reason for hiding this comment

Uh oh!

cudawarped Apr 25, 2023

Choose a reason for hiding this comment

Uh oh!

cudawarped commented Jun 14, 2023

Uh oh!

asmorkalov left a comment

Choose a reason for hiding this comment

Uh oh!

asmorkalov commented Jun 20, 2023

Uh oh!

cudawarped commented Jun 20, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

`VideoCapture`: change `CAP_PROP_FOURCC` to fix #22876 #23540

`VideoCapture`: change `CAP_PROP_FOURCC` to fix #22876 #23540

cudawarped commented Apr 25, 2023 •

edited by asmorkalov

Loading

cudawarped commented Jun 20, 2023 •

edited

Loading