Hi there, last
few blogs were hardcore machine learning and AI. Today let’s learn something
interesting, lets do some magic using computer vision. I hope you all know
about Harry Potter’s ‘invisible cloak’, the one he uses to become invisible. We
will see how we can do the same magic trick with the help of computer vision. I
will code with python and use the opencv library.
Below is the
video for your reference:
The
algorithm is very simple, we will separate the foreground and background image
with segmentation. And then remove the foreground object from every frame. We
are using a red coloured cloth as foreground image; you can use any other color
of your choice but need to tweak the code accordingly. We will use the
following steps:
- Import necessary libraries, create output video
- Capture and store the background for every frame.
- Detect the red coloured part in every frame.
- Segment out the red coloured part with a mask image.
- Generate the final magical output.
Step1: Import necessary libraries,
create output video
Import the
libraries. OpenCV is a library of programming functions mainly aimed at
real-time computer vision. NumPy is the fundamental package for scientific
computing with Python. In machine learning as we need to deal with a huge
amount of data, we use NumPy, which is faster than normal array. Prepare for
the output video.
Step2: Capture and store the
background for every frame
The main
idea is to replace the current frames’ red pixels with background pixels to
generate the invisible effect. To do that first we need to store the background
image for every frame.
cap.read()
method is used to capture the current frame and stores the variables in ‘background’.
The method also returns a Boolean True/False store in ret, if the frame is read
correctly it returns Trues else false.
We are
capturing the background in a for loop, so that we have several frames for
background as averaging over multiple frames also reduces noise.
Step3: Detect the red coloured part
in every frame
Now we will
focus on detecting the red part of the image. As RGB (Red-Green-Blue) values are
highly sensitive to illumination we will convert the RGB image to HSV (Hue –
Saturation – Value) space. After we convert the frame to HSV space we will
specify, some specific color range to detect the red color.
In general,
the Hue values are distributed over a circle ranging between 0-360 degrees, but
in OpenCV the range is from 0-180. And the red colour is represented by 0-30 as
well as 150-180 values. We use the range 0-10 and 170-180 to avoid detection of
skin as red. And then combine the masks with a OR operator(for python + is
used).
Step4: Segment out the red coloured part
with a mask image
Now that we
where the red part is in the frame from the mask image, we will use this mask
to segment that part from the whole frame. We will do a morphology open and
dilation for that.
Step5: Generate the final magical output
Finally, we
will replace the pixels of the detected red coloured region with corresponding
pixel values of the static background, which we saved earlier and finally
generate the output which creates the magical effect.
So now you
can create your own video with invisible cloak. You can download the running
python code from here: full code
Hope you
enjoyed the magical aspect of computer vision. Do let me know your feedback and
suggestion in the comment below. Thank you
