10

I am trying to use the output of Opencv's dense optical flow function to draw a quiver plot of the motion vectors but have not been able to find what the function actually outputs. Here is the code:

import cv2
import numpy as np

cap = cv2.VideoCapture('GOPR1745.avi')

ret, frame1 = cap.read()
prvs = cv2.cvtColor(frame1,cv2.COLOR_BGR2GRAY)
hsv = np.zeros_like(frame1)

hsv[...,1] = 255
count=0

while(1):
    ret, frame2 = cap.read()
    next = cv2.cvtColor(frame2,cv2.COLOR_BGR2GRAY)
    flow = cv2.calcOpticalFlowFarneback(prvs,next,None, 0.5, 3, 15, 3, 10, 1.2, 0)
    mag, ang = cv2.cartToPolar(flow[...,0], flow[...,1])

    hsv[...,0] = ang*180/np.pi/2
    hsv[...,2] = cv2.normalize(mag,None,0,255,cv2.NORM_MINMAX)
    rgb = cv2.cvtColor(hsv,cv2.COLOR_HSV2BGR)
    if count==10:
        count=0

        print "flow",flow

    cv2.imshow('frame2',rgb)
    count=count+1
    k = cv2.waitKey(30) & 0xff
    if k == 27:
        break
    elif k == ord('s'):
    prvs = next

cap.release()
cv2.destroyAllWindows()

This is effectively the same code as given in the OpenCv tutorial on dense optical flow. I receive the following output from the print function:

flow [[[  0.00000000e+00   0.00000000e+00]
  [  0.00000000e+00   0.00000000e+00]
  [  0.00000000e+00   0.00000000e+00]
  ..., 
  [  0.00000000e+00   0.00000000e+00]
  [  0.00000000e+00   0.00000000e+00]
  [  0.00000000e+00   0.00000000e+00]]

 ..., 
 [[ -3.54891084e-14  -1.38642463e-14]
  [ -2.58058853e-14  -1.54020863e-14]
  [ -5.56561768e-14  -1.88019359e-14]
  ..., 
  [ -7.59403916e-15   1.16633225e-13]
  [  7.22156371e-14  -1.61951507e-13]
  [ -4.30715618e-15  -4.39530987e-14]]

 [[ -3.54891084e-14  -1.38642463e-14]
  [ -2.58058853e-14  -1.54020863e-14]
  [ -5.56561768e-14  -1.88019359e-14]
  ..., 
  [ -7.59403916e-15   1.16633225e-13]
  [  7.22156371e-14  -1.61951507e-13]
  [ -4.30715618e-15  -4.39530987e-14]]

I would like to know what exactly these values are? Original X,Y coordinates? Final X,Y coordinates? Distance moved?

I plan to try and find the initial and final coordinates to make a quiver plot using code from the following page: https://www.getdatajoy.com/examples/python-plots/vector-fields This is because in python there is no function that i am aware of that plots an optical flow map for you.

Thank you in advance!

2 Answers 2

12

You were almost there. Lets first take a look at the calcOpticalFlowFarneback Documentation it says there:

flow – computed flow image that has the same size as prev and type CV_32FC2.

So what you are actually getting is a matrix that has the same size as your input frame.
Each element in that flow matrix is a point that represents the displacement of that pixel from the prev frame. Meaning that you get a point with x and y values (in pixel units) that gives you the delta x and delta y from the last frame.

3
  • 2
    Okay awesome, Thank you. So the flow matrix would be a matrix the same size as the original image, and the X and Y values stored in it at certain positions would be the X and Y distance traveled to get where they currently are. So if i were to find out where they moved from I would simply use their current pixel location in the flow matrix and subtract the X and Y distances they traveled? Jul 1, 2016 at 7:35
  • 1
    do all optical flow algorithms give output in pixel units ?
    – asdf
    Feb 24, 2018 at 9:57
  • Yes. Since it captures the pixel motion Aug 27, 2020 at 10:52
1

I'm going to hijack this because it is the same topic.

If units are pixels as stated by @shravya, why this code does not show maximum flow equal to one?

I really dont get the units

Code


import numpy as np
import cv2
import seaborn as sns

# Generating img
img = np.zeros(shape=(3,50,50)) # 3 time frames, 50x50

center0 = np.array((10,10))
center1 = np.array((30,30))
for each_time, each_x, each_y in itertools.product(range(img.shape[0]), range(img.shape[1]), range(img.shape[2])): 
    img[each_time, each_x, each_y] =  img[each_time, each_x, each_y] + 1000 *  1/(  0.1* ((center0[0]+each_time*displacement_x - each_x)**2 + 1*(center0[1]+each_time*displacement_y - each_y)**2)**0.5 + 1)    
    img[each_time, each_x, each_y] =  img[each_time, each_x, each_y] + 1000 * 1/(  0.1* ((center1[0]+each_time*displacement_x - each_x)**2 + 1*(center1[1]+each_time*displacement_y - each_y)**2)**0.5 + 1)


img = (img - img.min())/(img.max()-img.min()) # Normalizing

## Ploting
fig, axs = plt.subplots(ncols=3, squeeze=True, figsize=(20,5))
for i in range(3):
    im = sns.heatmap(img[i,:,:], ax = axs[i], vmin=0, vmax=np.max(img))

fig.suptitle('Image') 

def calc_flow(img):

    ## Optical flow
    img = img.astype(np.int16)

    prev = np.zeros(img[0, :, :].shape).astype(np.int16)
    flows = np.zeros(shape=(img.shape[0], img.shape[1], img.shape[2], 2))

    for i, each_frame in enumerate(img):
        if i > img.shape[0]:
            break

        next_ = each_frame
        flow = cv2.calcOpticalFlowFarneback(prev, next_, None,
                                           pyr_scale = 0.5,
                                            levels = 3,
                                            winsize = 12,
                                            iterations = 5,
                                            poly_n = 5,
                                            poly_sigma = 1.2,
                                            flags = 0) 

        flows[i, :, :, 0] = flow[..., 0]
        flows[i, :, :, 1] = flow[..., 1]

        prev = next_
    
    return flows

flow = calc_flow(img)

fig, axs = plt.subplots(ncols=3, nrows=2, squeeze=True, figsize=(20,10))
for i in range(3):
    im = sns.heatmap(flow[i,:,:, 0] ,ax = axs[0,i], vmin=0, vmax = np.max(flow)) 
    im = sns.heatmap(flow[i,:,:, 1] ,ax = axs[1,i], vmin=0, vmax = np.max(flow)) 
    
fig.suptitle('Flow x and y plots')

mag_img, pha_img = cv2.cartToPolar(flow[..., 0], flow[..., 1]) 

fig, axs = plt.subplots(ncols=3, nrows=2, squeeze=True, figsize=(20,10))
for i in range(3):
    im = sns.heatmap(mag_img[i,:,:], ax=axs[0,i], vmin=0, vmax = np.max(mag_img)) 
    im = sns.heatmap(pha_img[i,:,:], ax=axs[1,i], vmin=0, vmax = np.max(pha_img))  
    
fig.suptitle('Magnitude and phase plots')

## Output
print(flow.max()) # This should be equal to displacement!
print(np.abs(flow).min()) # this should be zero


print(mag_img.max()) # This should be equal to displacement!
print(mag_img.min()) # this should be zero

Your Answer

Reminder: Answers generated by Artificial Intelligence tools are not allowed on Stack Overflow. Learn more

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.