Object Tracking & Prediction -Image Processing

This  project  is  aimed  at  object  tracking,  and  it involves  tracking  a  predefined  object  based  on  its  color  and predict  its  trajectory  to  expect  the  location  it  appears  again  in the  event  of  an  occlusion  by  a  different  larger  object. The  project  consists  of  3  main  parts  which  will  make  use  of basic  image  processing  techniques  in  openCV  with  python  and MATLAB.  Firstly,  we  try  to  detect  our  object  of  interest  which in  our  case  is  a  ball  of  a  specific  color.  The  detection  is  based on  its  color  which  can  be  changed  as  desired  before  running the  code.  Second  part  is  building  an  array  for  the  balls  center location (x,y) as a function of time which we feed to the third part; the predictor. Last part is about predicting the trajectory of the ball  which  is  done  using  mathematical  curve  fitting  techniques. Obviously,  the  more  points  we  get,  the  better  results  we  obtain for  prediction. The process of detecting and tracking the ball will be simplified by  detecting  the  ball  based  on  its  color  which  will  initially involve  converting  the  color  space  from  RGB  to  HSV.  Then an  image  thresholding  is  done  with  boundaries  to  get  a  binary image  output.  We  will  then  need  to  draw  and  find  contour  and extract  the  biggest  bound  of  the  contour  on  the  binary  image. This process will be continuous for each frame received from the camera  stream. The  project  output  would  be  interacting  with  a  live  video, tracing the ball and drawing a circle around it. Then drawing a line (or curve) for tracing the balls location and finally drawing another  line  (or  curve)  with  a  different  color  that  shows  the expected  path.  The  program  should  track  the  ball  as  long  as  it can  see  it  and  when  its  blocked  by  an  obstacle,  its  showing  the expected  path  for  it  and  then  when  it  is  visible  again,  it  follows the  same  procedures  to  track  and  predict. Ultimately,  the  project  will  be  able  to  detect  balls  based  on any color and can be expanded to detect any predefined object. A  graphical  user  interface  will  also  be  designed  to  select  the specific  color  of  ball  needed  to  be  tracked  and  will  contain several  options  that  will  also  enhance  visualization  as  needed.

For coding, MATLAB is used.

A function is created to detect the object of interest and it passes the center and radius to the main script which use them to track the path of the ball motion and also to predict the behavior of such  motion. For the tracking part, we save the coordinates of the ball center as (x,y) points and plot the last n points where n is predetermined. Also, we create a circle around the ball using the radius found in the ball detection part. To predict the ball motion, we first differentiate the nature of the motion whether it’s a line or a curve and if it’s a line, then we determine if it’s horizontal or vertical one. We make that identification by checking the difference between x values to know if it’s a vertical line; if the difference is close to a specified tolerance value that is relatively close to zero, then it’s considered a vertical line and we plot the predicted path as a vertical line starting from the last position of the ball center to the end of the frame. We adapt the same technique for the horizontal line case. If this is not the case, then we check the ratio of the difference in y values, ∆y, and in x values, ∆x, for the last 3 points which represent 2 slopes. If the 2 slopes are close to each other, then we may consider the path as a linear one and plot a line with a slope equal to the average of the 2 slopes. If the slopes are not close, then we consider it as a curvilinear motion and we use the curve fitting technique to predict the motion. We adapt a second order polynomial function to create our predicted path.