Gain insights into using Python to read and write optical labels, explore 1D and 2D barcodes, and fiduciary markers for augmented reality, and discover relevant Python libraries and applications.

Reading _ Writing Machine-Readable Optical Labels with Python .jpg

g.tar.gz

Course

Gradio

Gradio-2

Gradio-3

Machine-readable optical labels are visual representations of data that machines can read and interpret. These labels, often composed of simple geometric shapes or lines, are designed to encode various kinds of information, such as product details or reference numbers.

In this course, you’ll be introduced to Python-based reading and writing of 1D and 2D barcodes and fiduciary markers. You’ll then learn the practical applications of barcodes and fiduciary markers in augmented reality and the creation of user interfaces for reading barcodes. Next, you’ll learn about different Python libraries and their usage to process different optical labels.

By the end of the course, you’ll be proficient in using Python for handling barcodes and fiduciary markers. You’ll also have hands-on experience installing and using different Python libraries to manipulate optical labels.

Using Python for Reading and Writing Optical Labels

The code below is designed to detect ArUco markers in an image and then perform augmented reality by overlaying another image onto the detected ArUco markers.



# Replacing four fiduciary markers with an image

In the code below, we will replace the four markers in the above image with another image.

import cv2
import numpy as np
import pandas as pd

def order_points(pts):
    rect = np.zeros((4, 2), dtype = "int32")
    
    s = pts.sum(axis = 1)
    rect[0] = pts[np.argmin(s)]
    rect[2] = pts[np.argmax(s)]
    
    diff = np.diff(pts, axis = 1)
    rect[1] = pts[np.argmin(diff)]
    rect[3] = pts[np.argmax(diff)]
    return rect

def visualizationAR(image, corners, ids, camera_matrix, distortion_coeffs, rvecs, tvecs, marker_length):

    if len(ids)==4:
        
        corners = np.array([corner[0] for corner in corners]).reshape((16,2)).astype('int32')
        center = np.mean(corners, axis=0).astype('int32')
        
        distances = [(np.linalg.norm(center - corner), corner) for corner in corners]
        distances.sort(reverse=True)
        
        points = np.array([x[1] for x in distances[:4]])
        
        ordered_points = order_points(points)
        upper_left = ordered_points[0]
        
        ordered_points = ordered_points.reshape((-1, 1, 2))

        overlay_image = cv2.imread("./resources/QRcodeEducative.png")
        overlay_h, overlay_w = overlay_image.shape[:2]
        overlay_points = np.array([[0,0],[overlay_w, 0],[overlay_w, overlay_h], [0, overlay_h]])
        h, _ = cv2.findHomography(overlay_points, ordered_points)
        warped_image  = cv2.warpPerspective(overlay_image, h, (image.shape[1], image.shape[0]))
        
        mask = np.zeros([image.shape[0], image.shape[1]], dtype='uint8')
        cv2.fillConvexPoly(mask, ordered_points, (255, 255, 255), cv2.LINE_AA)
 
        element = cv2.getStructuringElement(cv2.MORPH_RECT, (3,3));
        mask = cv2.erode(mask, element, iterations=3);

        warped_image = warped_image.astype(float)
        mask3 = np.zeros_like(warped_image)
        for i in range(0, 3):
            mask3[:,:,i] = mask/255

        warped_image_masked = cv2.multiply(warped_image, mask3)
        frame_masked = cv2.multiply(image.astype(float), 1-mask3)
        im_out = cv2.add(warped_image_masked, frame_masked)
    
    return im_out

image_file = './resources/marker_images_ar/calibrate_20230616_173854.png'
image = cv2.imread(image_file)

calibration_data_file = "./resources/calibration_data.npz"
camera_matrix, distortion_coeffs = loadCalibrationData(calibration_data_file)
height, width = image.shape[:2]
new_camera_matrix, _ = cv2.getOptimalNewCameraMatrix(camera_matrix, distortion_coeffs, (width, height), 0, (width, height))
image = cv2.undistort(image, camera_matrix, distortion_coeffs, None, new_camera_matrix)
  
processed_image = preprocessImage(image)

marker_dict = cv2.aruco.Dictionary_get(cv2.aruco.DICT_5X5_250)
detector_params = cv2.aruco.DetectorParameters_create()
detections, ids = detectMarkers(processed_image, marker_dict, detector_params)

marker_length = .04
rvecs, tvecs = poseEstimation(detections, marker_length, camera_matrix, distortion_coeffs)

result = visualizationAR(image, detections, ids, camera_matrix, distortion_coeffs, rvecs, tvecs, marker_length)

cv2.imwrite(f"./output/marker_ar.png", result)

**Lines 1–3:** The code starts by importing the required Python modules: `cv2` for image processing and marker detection, `numpy` for numerical operations, and `pandas` for data manipulation.

We define two helper functions:

**Lines 5–15:** The `order_points()` function takes a list of 4 points and returns them in an ordered manner. The ordering is top-left, top-right, bottom-right, and bottom-left.

**Lines 17–55:** The `visualizationAR()` function overlays an image on top of the four detected ArUco markers. The function first checks if there are exactly four markers detected. It then calculates the center of these markers and determines the four corners that are furthest from the center. The points are ordered using the `order_points()` function. It then performs a perspective transformation to align the overlay image with the detected markers. Finally, it creates a mask of the warped overlay image and adds this to the original image.

The other helper functions were already described in a previous lesson.


**Lines 57–58:** Then, the code loads the image file that contains the ArUco markers.

**Lines 60–61:** The camera's calibration data, including the camera matrix and distortion coefficients, are subsequently loaded from a specified file.

**Lines 63–64:** The image is undistorted using the camera calibration data. This is performed with the `cv2.getOptimalNewCameraMatrix()` and  `cv2.undistort()` functions.

**Line 66:** Next, the undistorted image is converted to grayscale using the `preprocessImage()` function. 

**Lines 68–70:** The code gets a predefined dictionary of ArUco markers, creates detector parameters, and then detects the markers in the preprocessed image with `cv2.aruco.detectMarkers()`.

**Line 73:** The pose (rotation and translation vectors) of each marker is estimated using the `poseEstimation()` function. This function uses `cv2.aruco.estimatePoseSingleMarkers()` to estimate the pose.

**Line 75:** The `visualizationAR()` function is called to overlay the image `./resources/QRcodeEducative.png` onto the detected ArUco markers.

**Line 77:** Finally, the final AR image is saved to a file. The result shows the detected ArUco markers with the overlay image perspective transformed and drawn over the markers. The image that we’ll be using contains two ArUco markers.

Learn how to perform a simple augmented reality image overlay using Python and OpenCV.

Introduction to Machine-Readable Optical Labels

An Overview of Barcodes

Reading and Writing 1D Barcodes With Python

Reading and Writing 2D Barcodes With Python

Creating Fiduciary Markers With Python

Reading Fiduciary Markers With Python

Fiduciary Marker Use Cases

Build a User Interface With Gradio For Reading Barcodes

Build a Barcode Writing Application with Python

Roundup

Appendix

Augmented Reality with ArUco Markers