0

I am trying to develop a multi-threaded server to handle video streams. Multiple users can be connected at the same time. Each user sends a frame from his camera to the server, the server processes it, sends a response to the user, and so on. There were 2 options:

  1. Use UDP protocol. The server in the main thread receives frames, throws them into the thread pool for processing, and from there, when ready, sends a response to the client. This approach is bad because the acceptance of a frame in the main thread is very long and there was an out of sync between users, which led to client hangs.
  2. Use TCP protocol. The server in the main thread listens for the connection of clients and puts into the thread pool not the processing of 1 frame, but the processing of all frames of this client. This approach solved the problem of synchronization, but slowed down the work by 5-6 times. In fact, it is not clear why, because the image processing takes place on the GPU, which forces all frames to be processed sequentially and the thread pool does not play a role in speeding up calculations, but support for multiuser operation.So I concluded that this slowdown was due to TCP (on localhost, there were almost no changes in the operating time - an increase of about 1 fps for TCP).

I would like to figure out whether such a slowdown is really possible and how to be in this case.

All frames are sent in an uncompressed state, which was chosen empirically because the compression took longer.

P.S. It is very disappointing when the main algorithm works in 14 ms and all efforts were directed to its optimization, and the bottleneck turned out to be data transfer - about 250-500 ms.

4
  • Both approaches should be good. But the devil is in the implementation. Not much more can be said without looking into the code. Oct 21, 2020 at 11:35
  • @Maxim Egorushkin Can I leave you a link to the github for a superficial review?
    – Egor
    Oct 21, 2020 at 11:40
  • Consider using webrtc
    – Equod
    Oct 21, 2020 at 11:44
  • Rules stackoverflow.com/help/minimal-reproducible-example say that the complete code must be in the question. If you want someone to code review your repo, that's probably asking too much. Oct 21, 2020 at 14:21

0

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Browse other questions tagged or ask your own question.