The depth module contains classes that hide some of all the repeated
code associated with processing depth data. The main component
DepthProcessing class, which is used to process
continuously retrieved data from the kinect.
The background model used in this class is obtained through running average via
If code already exists to retrieve the data extracting the bounding boxes proposals can be reduced to as little as the following:
# DepthProcessing instance depth = DepthProcessing(IMAGE_SHAPE) while True: # retrieve the depth information depth.current_frame = cam.get_depth_matrix() if first_run: # in first run moving average start from first frame depth.background_model = depth.current_frame.astype(depth.background_model.dtype) first_run = False # get depth background depth.update_background_model(depth.current_frame) # get depth foreground depth.extract_foreground_mask_from_run_avg(depth.current_frame) # apply opening to remove noise depth.foreground_mask = bg_models.apply_opening(depth.foreground_mask, BG_OPEN_KSIZE, cv2.MORPH_ELLIPSE) depth_proposal_bbox = depth.extract_proposal_bbox(depth.ACCUMULATOR)
Depth Processing Class
Extract depth foreground mask from running average computed substracting current_frame from background model where the difference is above BG_MASK_THRESHOLD
Parameters: current_frame – current frame from which extract foreground Returns: binary mask with 1 for foreground and 0 for background Return type: np.int64
Compute bounding boxes for connected components from foreground masks that remain constant for AGG_DEPTH_MAX_E frames.
To keep track of the bounding boxes over time the function uses an aggregator depending on the method specified
Parameters: method –
method used to keep track of the bounding boxes history. Methods available are:
- ACCUMULATOR: to use an image accumulator for each pixel (fastest method). The bounding boxes are extracted from the pixels accumulated AGG_DEPTH_MAX_E times.
- RECT_MATCHING/RECT_MATCHING2: to keep track of the number of times a particular bounding box occurs over time (slower method but more accurate). Two bounding boxes in different frames are considered the same if their placement and dimension remain within a tolerance threshold.
Returns: list of bounding boxes in the form of (x,y, width, height) where (x,y) is the top left corner Return type: List Raise: NotImplementedError: if a method different from ACCUMULATOR or RECT_MATCHING or RECT_MATCHING2 is specified
Update aggregator with the provided foreground. Each pixel of the image has a value that keeps the number of times it has been seen as foreground.
- aggregator – an image of uint8
- foreground_current – mask of the current foreground