I try to develop my own virtual webcam. Therefore I play with v4l2loopback and try to understand this tutorial: https://arcoresearchgroup.wordpress.com/2020/06/02/virtual-camera-for-opencv-using-v4l2loopback/
I have encountered some problems, when I use the exact same program like in this tutorial. The program will be compiled and get started, but when I try to watch my virtual device with ffplay or vlc I just get only one frame and a core dumped error. I don't know why this happens, maybe one of you can help me.
#include <opencv2/opencv.hpp>
#include <fcntl.h>
#include <unistd.h>
#include <sys/ioctl.h>
#include <linux/videodev2.h>
#ifndef VID_WIDTH
#define VID_WIDTH 640
#endif
#ifndef VID_HEIGHT
#define VID_HEIGHT 480
#endif
#define VIDEO_IN "/dev/video0"
#define VIDEO_OUT "/dev/video6"
int
main(int argc, char* argv[]) {
// open and configure input camera (/dev/video0)
cv::VideoCapture cam(VIDEO_IN);
if (not cam.isOpened()) {
std::cerr << "ERROR: could not open camera!\n";
return -1;
}
cam.set(cv::CAP_PROP_FRAME_WIDTH, 480);
cam.set(cv::CAP_PROP_FRAME_HEIGHT, 640);
// open output device
int output = open(VIDEO_OUT, O_RDWR);
if(output < 0) {
std::cerr << "ERROR: could not open output device!\n" << strerror(errno);
return -2;
}
// configure params for output device
struct v4l2_format vid_format;
memset(&vid_format, 0, sizeof(vid_format));
vid_format.type = V4L2_BUF_TYPE_VIDEO_OUTPUT;
if (ioctl(output, VIDIOC_G_FMT, &vid_format) < 0) {
std::cerr << "ERROR: unable to get video format!\n" << strerror(errno);
return -1;
}
size_t framesize = VID_WIDTH * VID_HEIGHT * 3;
vid_format.fmt.pix.width = cam.get(cv::CAP_PROP_FRAME_WIDTH);
vid_format.fmt.pix.height = cam.get(cv::CAP_PROP_FRAME_HEIGHT);
// NOTE: change this according to below filters...
// Chose one from the supported formats on Chrome:
// - V4L2_PIX_FMT_YUV420,
// - V4L2_PIX_FMT_Y16,
// - V4L2_PIX_FMT_Z16,
// - V4L2_PIX_FMT_INVZ,
// - V4L2_PIX_FMT_YUYV,
// - V4L2_PIX_FMT_RGB24,
// - V4L2_PIX_FMT_MJPEG,
// - V4L2_PIX_FMT_JPEG
vid_format.fmt.pix.pixelformat = V4L2_PIX_FMT_RGB24;
vid_format.fmt.pix.sizeimage = framesize;
vid_format.fmt.pix.field = V4L2_FIELD_NONE;
if (ioctl(output, VIDIOC_S_FMT, &vid_format) < 0) {
std::cerr << "ERROR: unable to set video format!\n" << strerror(errno);
return -1;
}
// create GUI window
const char* gui = "gui";
cv::namedWindow(gui);
cv::setWindowTitle(gui, "OpenCV test");
// loop over these actions:
while (true) {
// grab frame
cv::Mat frame;
if (not cam.grab()) {
std::cerr << "ERROR: could not read from camera!\n";
break;
}
//cam.retrieve(frame);
cam >> frame;
// apply simple filter (NOTE: result should be as defined PIXEL FORMAT)
// convert twice because we need RGB24
cv::Mat result;
cv::cvtColor(frame, result, cv::COLOR_RGB2GRAY);
cv::cvtColor(result, result, cv::COLOR_GRAY2RGB);
// show frame
cv::imshow("gui", result);
// write frame to output device
size_t written = write(output, result.data, framesize);
if (written < 0) {
std::cerr << "ERROR: could not write to output device!\n";
close(output);
break;
}
// wait for user to finish program pressing ESC
if (cv::waitKey(10) == 27)
break;
}
std::cout << "\n\nFinish, bye!\n";
}
The issue trace:
ffplay /dev/video6
ffplay version 4.2.7-0ubuntu0.1 Copyright (c) 2003-2022 the FFmpeg developers
built with gcc 9 (Ubuntu 9.4.0-1ubuntu1~20.04.1)
configuration: --prefix=/usr --extra-version=0ubuntu0.1 --toolchain=hardened --libdir=/usr/lib/x86_64-linux-gnu --incdir=/usr/include/x86_64-linux-gnu --arch=amd64 --enable-gpl --disable-stripping --enable-avresample --disable-filter=resample --enable-avisynth --enable-gnutls --enable-ladspa --enable-libaom --enable-libass --enable-libbluray --enable-libbs2b --enable-libcaca --enable-libcdio --enable-libcodec2 --enable-libflite --enable-libfontconfig --enable-libfreetype --enable-libfribidi --enable-libgme --enable-libgsm --enable-libjack --enable-libmp3lame --enable-libmysofa --enable-libopenjpeg --enable-libopenmpt --enable-libopus --enable-libpulse --enable-librsvg --enable-librubberband --enable-libshine --enable-libsnappy --enable-libsoxr --enable-libspeex --enable-libssh --enable-libtheora --enable-libtwolame --enable-libvidstab --enable-libvorbis --enable-libvpx --enable-libwavpack --enable-libwebp --enable-libx265 --enable-libxml2 --enable-libxvid --enable-libzmq --enable-libzvbi --enable-lv2 --enable-omx --enable-openal --enable-opencl --enable-opengl --enable-sdl2 --enable-libdc1394 --enable-libdrm --enable-libiec61883 --enable-nvenc --enable-chromaprint --enable-frei0r --enable-libx264 --enable-shared
libavutil 56. 31.100 / 56. 31.100
libavcodec 58. 54.100 / 58. 54.100
libavformat 58. 29.100 / 58. 29.100
libavdevice 58. 8.100 / 58. 8.100
libavfilter 7. 57.100 / 7. 57.100
libavresample 4. 0. 0 / 4. 0. 0
libswscale 5. 5.100 / 5. 5.100
libswresample 3. 5.100 / 3. 5.100
libpostproc 55. 5.100 / 55. 5.100
Input #0, video4linux2,v4l2, from '/dev/video6':
Duration: N/A, start: 104476.472687, bitrate: 221184 kb/s
Stream #0:0: Video: rawvideo (RGB[24] / 0x18424752), rgb24, 640x480, 221184 kb/s, 30 fps, 30 tbr, 1000k tbn, 1000k tbc
Speicherzugriffsfehler (Speicherabzug geschrieben)B sq= 0B f=0/0
In addition, I changed pixel format and get with the parameter MJPEG this issue, when I try the command ffplay /dev/video6:
ffplay /dev/video6
ffplay version 4.2.7-0ubuntu0.1 Copyright (c) 2003-2022 the FFmpeg developers
built with gcc 9 (Ubuntu 9.4.0-1ubuntu1~20.04.1)
configuration: --prefix=/usr --extra-version=0ubuntu0.1 --toolchain=hardened --libdir=/usr/lib/x86_64-linux-gnu --incdir=/usr/include/x86_64-linux-gnu --arch=amd64 --enable-gpl --disable-stripping --enable-avresample --disable-filter=resample --enable-avisynth --enable-gnutls --enable-ladspa --enable-libaom --enable-libass --enable-libbluray --enable-libbs2b --enable-libcaca --enable-libcdio --enable-libcodec2 --enable-libflite --enable-libfontconfig --enable-libfreetype --enable-libfribidi --enable-libgme --enable-libgsm --enable-libjack --enable-libmp3lame --enable-libmysofa --enable-libopenjpeg --enable-libopenmpt --enable-libopus --enable-libpulse --enable-librsvg --enable-librubberband --enable-libshine --enable-libsnappy --enable-libsoxr --enable-libspeex --enable-libssh --enable-libtheora --enable-libtwolame --enable-libvidstab --enable-libvorbis --enable-libvpx --enable-libwavpack --enable-libwebp --enable-libx265 --enable-libxml2 --enable-libxvid --enable-libzmq --enable-libzvbi --enable-lv2 --enable-omx --enable-openal --enable-opencl --enable-opengl --enable-sdl2 --enable-libdc1394 --enable-libdrm --enable-libiec61883 --enable-nvenc --enable-chromaprint --enable-frei0r --enable-libx264 --enable-shared
libavutil 56. 31.100 / 56. 31.100
libavcodec 58. 54.100 / 58. 54.100
libavformat 58. 29.100 / 58. 29.100
libavdevice 58. 8.100 / 58. 8.100
libavfilter 7. 57.100 / 7. 57.100
libavresample 4. 0. 0 / 4. 0. 0
libswscale 5. 5.100 / 5. 5.100
libswresample 3. 5.100 / 3. 5.100
libpostproc 55. 5.100 / 55. 5.100
[mjpeg @ 0x7f159c002280] No JPEG data found in image= 0B f=0/0
[mjpeg @ 0x7f159c002280] bits 246 is invalid
[mjpeg @ 0x7f159c002280] bits 247 is invalid 0KB sq= 0B f=0/0
[mjpeg @ 0x7f159c002280] No JPEG data found in image= 0B f=0/0
[mjpeg @ 0x7f159c002280] No JPEG data found in image= 0B f=0/0
[mjpeg @ 0x7f159c002280] No JPEG data found in image= 0B f=0/0
[video4linux2,v4l2 @ 0x7f159c000bc0] decoding for stream 0 failed
[video4linux2,v4l2 @ 0x7f159c000bc0] Could not find codec parameters for stream 0 (Video: mjpeg (JPEG LS), none(bt470bg/unknown/unknown), 640x480, lossless): unspecified pixel format
Consider increasing the value for the 'analyzeduration' and 'probesize' options
Input #0, video4linux2,v4l2, from '/dev/video6':B sq= 0B f=0/0
Duration: N/A, start: 106356.321454, bitrate: N/A
Stream #0:0: Video: mjpeg (JPEG LS), none(bt470bg/unknown/unknown), 640x480, lossless, 30 fps, 30 tbr, 1000k tbn, 1000k tbc
[mjpeg @ 0x7f159c00a500] invalid id 238
[mjpeg @ 0x7f159c00a500] invalid id 242
[mjpeg @ 0x7f159c00a500] invalid id 238
[mjpeg @ 0x7f159c00a500] invalid id 242
[mjpeg @ 0x7f159c00a500] invalid id 238
[mjpeg @ 0x7f159c00a500] invalid id 242
[mjpeg @ 0x7f159c00a500] invalid id 247vq= 0KB sq= 0B f=0/0
[mjpeg @ 0x7f159c00a500] bits 248 is invalid 0KB sq= 0B f=0/0
[mjpeg @ 0x7f159c00a500] bits 248 is invalid 0KB sq= 0B f=0/0
[mjpeg @ 0x7f159c00a500] invalid id 244vq= 0KB sq= 0B f=0/0
[mjpeg @ 0x7f159c00a500] bits 245 is invalid 0KB sq= 0B f=0/0
[mjpeg @ 0x7f159c00a500] invalid id 244vq= 0KB sq= 0B f=0/0
[mjpeg @ 0x7f159c00a500] invalid id 245vq= 0KB sq= 0B f=0/0
[mjpeg @ 0x7f159c00a500] invalid id 215vq= 0KB sq= 0B f=0/0
[mjpeg @ 0x7f159c00a500] invalid id 244vq= 0KB sq= 0B f=0/0
[mjpeg @ 0x7f159c00a500] invalid id 244vq= 0KB sq= 0B f=0/0
[mjpeg @ 0x7f159c00a500] invalid id 247vq= 0KB sq= 0B f=0/0
[mjpeg @ 0x7f159c00a500] invalid id 240vq= 0KB sq= 0B f=0/0
[mjpeg @ 0x7f159c00a500] bits 241 is invalid 0KB sq= 0B f=0/0
[mjpeg @ 0x7f159c00a500] invalid id 247vq= 0KB sq= 0B f=0/0
[mjpeg @ 0x7f159c00a500] No JPEG data found in image= 0B f=0/0
[mjpeg @ 0x7f159c00a500] No JPEG data found in image= 0B f=0/0
[mjpeg @ 0x7f159c00a500] invalid id 250vq= 0KB sq= 0B f=0/0
In addition: I tried several pixel formats and something happened. Pixel format "RGB24" worked, but just for one time. When I shutdown the program and restarted it, the same issue popped up. Maybe this detail helps you to help me. best regards
Edit
I need to update this post. The program works fine! When I push out the image over /dev/video6 with the pixel format RGB24, it works fine when I use another programm and fetch the stream with opencv. When I use vlc, ffmpeg or other programs, it did not work. When I try it in pixel format V4L2_PIX_FMT_GREY ffmpeg works very fine but vlc didn't work again. Have someone an idea where I should look now or has a hint for me what I can do now?
Best regards