Even with the advancements made in technology, many developing countries are still subject to patriarchal crimes like molestation and rape or other crime against women. In the developing world, women are not permitted to leave their workplaces at night, and aren’t allowed to be alone during the day. It is imperative, in order to make the remaining half of humanity more productive and safe. Technology advancements can be used to empower them with greater power. Picture a woman who’s trapped in an office, clinic or shop and needs immediate rescue. It is possible that she may not be able to call. A surveillance camera must be able to recognize her hand gestures and use them as an SOS signal. The elderly can use the same solution at home to make requests and call for assistance.
Let’s evaluate our choices, whether to use image processing, deep learning, or arithmetic algorithms to analyze the incoming video frames from an SoC with a camera, to trigger an alert. This project has three main components. a) Locate the object to trigger an event. b) Analyze motion to identify the signal. c) Installation on the SoC. It is difficult because we require a cost-effective solution that can be adopted quickly, without compromising accuracy. We deal with emergency situations here, so this problem is not easy.
It is simpler to find a specific object, but more efficient to use your body to alert the authorities. Let’s now discuss the solutions to Localization & Gesture Detection. A) Object localization methods i) Object detection We can easily use the hardware-optimized YOLO device to detect, for example, a mobile phone. This will give us 5-6FPS on a 4GB Raspberry Pi4B using Movidius NCS 2. You can teach YOLO how to recognize a specific object, such as a hand. The problem is that NCS sticks will increase the price of the product. Vanilla YOLO only gives a mere 1 FPS with 4GB RPi4B. ii. Multi-scale Template Matching template Matching uses 2D-convolution to locate the image of a specific template in a larger picture. We can make template matching translation-invariant and scale-invariant as well. Adaptive thresholding allows you to generate a binary mask from a template image. The template image has objects that must be identified. Take the frame and create a binary mask. Use cv2.matchTemplate to create a match template.
Draw a rectangle around the area to be matched. Multi-scale Template Matching for Hand Detection. However, to recognize gestures, which are a series of actions of an object or person, it is necessary that all frames be consistent. Hand template multi-scale match was not consistent in detecting an object in each frame, according to experiments. Template matching can be problematic if the object you’re trying to match is rotated or has non-affine transforms. iii. Using Computer Vision to Create a Mask for an Object Color to Identify the Object Based on Its Color.
This is extremely efficient. To confirm our find, we can check the shape and size of the contour. To avoid false positives, it would be wise to choose an object of a different color. This technique is highly accurate and efficient. It also allows for gesture recognition with pure mathematical models. Edge users will find it an excellent solution. Below is a demonstration of the method and mathematical gesture recognition. B. Gesture Recognition Techniques A simple gesture can be defined so it is easy to recognize. We will now try to recognize the circular motion gesture. Auto-Correlated Scale Matching Generating ‘n points around a circle with radius =r ‘i’ denotes the ‘nth’ point on the circle.
Compute the slopes of each line linking the points in the sequence. Generate point clouds using one of the above object localization methods. Add the location of each localized object in the frame to the point cloud. Calculate the slopes of each line that connects the points in the sequence. Calculate the correlation between slope curves created in previous steps. Calculate the index of maxima in the correlation curve with np.argmax. Rotate point cloud queue according to index value. Compute circle similarity = 1 cosine distance. If correlation is > threshold, alert will be triggered. The above algorithm works once it is implemented. In practice however, distances between localized points vary depending on the rotation speed and FPS. Linear Algebra is another simple way to determine if points are moving convexally or concave. ii. Concavity Estimation with Vector Algebra: If all vectors within the point cloud appear concave it means that there is circular motion. Calculate the vectors using np.linalg.norm by dividing adjacent points. Calculate the vector length by using np.linalg.norm. If there is a distance between the points < threshold, then ignore the motion Take the cross product of vectors to detect left or right turns. Right Hand Rule: if B is counterclockwise from A, then AxB points upwards, otherwise downwards If any consecutive value has a different sign, then the direction is changing. Hence, compute a rolling multiplication. Compute vector, Take determinant, Find sign after mult Find out the location of direction change (where ever indices are negative) Compute the variance of negative indices If all rolling multiplication values >, then the motion is circular. If the variance of the negative indices is greater than threshold, motion will be non-circular. If the percentage of negative values is more than threshold, motion will not be circular. Based on these conditions, the circular gesture can easily been detected. The above discussion shows that we can both use Object Color Masking to detect objects and efficient algebra-based concavity estimates (to detect gestures). A small circle will be visible if the object is too far away. We need to adjust the vectors to account for object depth so that the gesture is not missed. These algorithms will be tested on an RPi equipped with a camera. Let’s now see the code for gesture detection.
If a circular motion is detected, then messages are pushed to people concerned and an alarm is triggered. Flashing the’red light’ on a Pimoroni Blinkt is how you demonstrate that an event has occurred. Signaled by MQTT messaging and SMS to mobile via Twilio integration. The video below shows the entire project. This solution is efficient and useful for gesture detection using edge devices. You can substitute any other object or gesture for the tennis ball. You will need to adjust the math formula as needed. The alarm may not work if there is extreme light or if you have to use an object outside (but not your body) to activate it. To
