🧩 3.5 Computer Vision#
🔰 Tutorial#
In this module, you will develop software to:
Use OpenCV for image processing and object detection
Implement spatial referencing and ID lookup using AprilTags
Automate a motorized microscope for sample analysis using OpenCV
1. OpenCV Basics#
OpenCV (Open Source Computer Vision Library) is a powerful and versatile open-source computer vision and machine learning software library. It is widely used for a variety of image and video processing tasks, making it an essential tool for developers working on computer vision applications.
Key features and capabilities of OpenCV include:
Image Processing: Basic operations like filtering, color space conversion, and geometric transformations.
Object Detection: Identifying and locating objects within images or video streams.
Feature Detection and Description: Identifying distinctive features in images for tasks like image matching and stitching.
Video Analysis: Processing video streams, including motion detection and object tracking.
Camera Calibration: Correcting lens distortion and determining camera position.
3D Reconstruction: Creating 3D models from 2D images.
Machine Learning: Implementing various machine learning algorithms for classification and clustering tasks.
GPU Acceleration: Utilizing GPU processing power for faster computations.
OpenCV is cross-platform and supports multiple programming languages, including Python, C++, and Java. Its extensive functionality and active community make it a go-to choice for both beginners and experienced developers in the field of computer vision.
Bill of Materials#
OpenCV (for Python installation:
pip install opencv-python)[Motorized Microscope]
[USB-A to micro USB-B cable]
Printed AprilTags (can be generated and printed from AprilTag Generation)
Demo#
✅ Read the OpenCV Documentation
✅ Watch Introduction to OpenCV
In this task, you will use OpenCV to perform image preprocessing and identify regions of interest (ROI).
import cv2
import numpy as np
def preprocess_image(image):
gray = cv2.cvtColor(image, cv2.COLOR_BGR2GRAY)
blurred = cv2.GaussianBlur(gray, (5, 5), 0)
thresh = cv2.adaptiveThreshold(
blurred, 255, cv2.ADAPTIVE_THRESH_GAUSSIAN_C, cv2.THRESH_BINARY_INV, 11, 2
)
return thresh
def find_roi(image):
processed = preprocess_image(image)
contours, _ = cv2.findContours(
processed, cv2.RETR_EXTERNAL, cv2.CHAIN_APPROX_SIMPLE
)
contours = sorted(contours, key=cv2.contourArea, reverse=True)
rois = []
for contour in contours[:5]:
area = cv2.contourArea(contour)
perimeter = cv2.arcLength(contour, True)
circularity = 4 * np.pi * area / (perimeter * perimeter)
if circularity > 0.7 and area > 1000:
x, y, w, h = cv2.boundingRect(contour)
rois.append((x, y, w, h))
return rois
def main():
image = cv2.imread("sample_image.jpg")
if image is None:
print("Error: Could not load image")
return
rois = find_roi(image)
for roi in rois:
x, y, w, h = roi
cv2.rectangle(image, (x, y), (x + w, y + h), (0, 255, 0), 2)
cv2.imshow("ROI Detection", image)
cv2.waitKey(0)
cv2.destroyAllWindows()
if __name__ == "__main__":
main()
3. Motorized Microscope Automation with OpenCV#
In this section, we’ll combine OpenCV image processing techniques with a motorized microscope to implement automated monitoring of microscopic particles.
Bill of Materials#
OpenFlexure Microscope (https://openflexure.org/) The OpenFlexure Microscope is a customisable, open-source optical microscope, using either very cheap webcam optics or lab quality, RMS threaded microscope objectives. It uses an inverted geometry, and has a high quality mechanical stage which can be motorised using low cost geared stepper motors.
Demo#
✅ Watch Building the OpenFlexure Microscope
This example code combines OpenCV image processing techniques with simulated microscope functionality to implement automated monitoring of microscopic particles. The main features include:
Image preprocessing: Using Gaussian blur and adaptive thresholding to enhance image quality.
Particle detection: Using contour detection methods to identify potential microscopic particles.
Microscope scanning: Utilizing the simulated microscope to scan an area and obtain multiple images.
Growth or distribution analysis: Calculating the number of detected particles and estimating changes over time.
Visualization: Marking detected particles on each scanned image and displaying the results.
This program performs periodic scanning and analysis to continuously monitor the sample. You can adjust the scanning interval, particle detection parameters, and other settings according to your specific requirements and sample type.
Note: The visibility and detection of particles will depend on the microscope’s magnification and the size of the particles you’re observing. This example can be used for various types of samples, such as:
Microspheres (1-100 μm)
Pollen grains (10-100 μm)
Large bacteria (1-10 μm, visible at 400x magnification)
Small protozoans (10-100 μm)
Adjust the detection parameters in the code based on your specific sample and microscope configuration.
import cv2
import numpy as np
from microscope_demo_client import MicroscopeDemo
from my_secrets import (
HIVEMQ_HOST,
HIVEMQ_PASSWORD,
HIVEMQ_PORT,
HIVEMQ_USERNAME,
MICROSCOPE_NAME,
)
import time
from PIL import Image
import io
microscope = MicroscopeDemo(
HIVEMQ_HOST, HIVEMQ_PORT, HIVEMQ_USERNAME, HIVEMQ_PASSWORD, MICROSCOPE_NAME
)
def preprocess_image(image):
gray = cv2.cvtColor(image, cv2.COLOR_BGR2GRAY)
blurred = cv2.GaussianBlur(gray, (5, 5), 0)
thresh = cv2.adaptiveThreshold(
blurred, 255, cv2.ADAPTIVE_THRESH_GAUSSIAN_C, cv2.THRESH_BINARY_INV, 11, 2
)
return thresh
def detect_particles(image):
processed = preprocess_image(image)
contours, _ = cv2.findContours(
processed, cv2.RETR_EXTERNAL, cv2.CHAIN_APPROX_SIMPLE
)
particles = []
for contour in contours:
area = cv2.contourArea(contour)
if 10 < area < 1000: # Adjust based on your sample and magnification
particles.append(contour)
return particles
def analyze_growth(previous_count, current_count):
growth_rate = (
(current_count - previous_count) / previous_count * 100
if previous_count > 0
else 0
)
return growth_rate
def microscope_scan_and_analyze():
print("Starting area scan...")
scan_images = microscope.scan([2000, 2000], [-2000, -2000])
total_particles = 0
for i, img_data in enumerate(scan_images):
img = Image.open(io.BytesIO(img_data))
img_cv = cv2.cvtColor(np.array(img), cv2.COLOR_RGB2BGR)
particles = detect_particles(img_cv)
total_particles += len(particles)
img_with_particles = img_cv.copy()
cv2.drawContours(img_with_particles, particles, -1, (0, 255, 0), 2)
cv2.imshow(f"Scan {i+1}/{len(scan_images)}", img_with_particles)
cv2.waitKey(1000)
print(
f"Scan image {i+1}/{len(scan_images)}: Detected {len(particles)} microscopic particles"
)
cv2.destroyAllWindows()
return total_particles
def main_microscope():
try:
previous_particle_count = 0
while True:
microscope.move(0, 0)
microscope.focus()
current_particle_count = microscope_scan_and_analyze()
growth_rate = analyze_growth(
previous_particle_count, current_particle_count
)
print(f"Total detected microscopic particles: {current_particle_count}")
print(f"Growth rate: {growth_rate:.2f}%")
previous_particle_count = current_particle_count
time.sleep(3600) # Wait for 1 hour
except KeyboardInterrupt:
print("Monitoring stopped")
finally:
microscope.end_connection()
if __name__ == "__main__":
main_microscope()
Note: The visibility and detection of particles will depend on the microscope’s magnification and the size of the particles you’re observing. This example can be used for various types of samples, such as microspheres (1-100 μm), pollen grains (10-100 μm), large bacteria (1-10 μm, visible at 400x magnification), or small protozoans (10-100 μm).
Considering of more accurate camera calibration is crucial for precise AprilTag detection and spatial referencing, we provide the following options:
Default Calibration: For quick start, you can use these default parameters for the Raspberry Pi Camera Module:
camera_params = [1000, 1000, 320, 240] # fx, fy, cx, cy
Simple Calibration: Run the provided
calibrate_camera.pyscript, which uses AprilTags for calibration:python calibrate_camera.py
This will generate a
calibration.jsonfile with your camera’s parameters.
Optional: Include a ruler in your setup for size reference and manual verification of measurements.
For more information on camera calibration, refer to the OpenCV documentation on Camera Calibration.
Tips#
Improving AprilTag Spatial Referencing Accuracy:
Perform precise camera calibration using the provided calibration script.
Capture calibration images from various angles and distances for better results.
Use larger AprilTags to improve detection accuracy at greater distances.
Ensure consistent and adequate lighting conditions in your working environment.
Enhancing Particle Detection in Microscope Automation:
Fine-tune the threshold parameters in the
detect_particlesfunction for your specific samples.Experiment with advanced image processing techniques like edge detection or morphological operations.
Develop custom detection algorithms tailored to your particular type of sample if needed.
Consider using machine learning approaches for more complex particle recognition tasks.
Extending the System for Multi-Object Recognition and Tracking:
Integrate deep learning models such as YOLO or SSD for object detection and classification.
Combine these models with the AprilTag system for comprehensive scene understanding.
Implement a tracking algorithm to maintain object identity across frames.
Use a database to store and retrieve object information based on detected features or tags.
Handling Image Analysis at Different Microscope Magnification Levels:
Create separate sets of image processing parameters optimized for each magnification level.
Implement a calibration process using standard samples of known size.
Determine and store the pixel-to-actual size conversion ratio for each magnification level.
Develop a system to automatically adjust processing parameters based on the current magnification setting.