UAV 3D Crop Phenotyping: From Image Acquisition to Machine Learning Modeling

1. Flight Path Design and Image Acquisition

UAV image acquisition employs a CCO (Cross-Complementary Overlap) flight path design strategy. This strategy enhances viewpoint diversity through multi-directional cross-flight paths to strengthen geometric constraints for 3D reconstruction.

Key Design Points:

• Multi sets of flight paths in different directions: Ensures multi-angle capture of crop canopy structure
• Forward and side overlap rates higher than conventional orthophoto requirements: Provides sufficient matching points for SfM reconstruction
• Image acquisition primarily serves 3D reconstruction goals: Rather than only satisfying orthophoto mosaic requirements

Based on industry-grade UAV platforms, multi-view RGB images of farmland are collected to provide unified data sources for subsequent orthophoto mosaic and 3D modeling.

2. Orthophoto Mosaic and Spatial Reference Construction

Multi-view images are first orthorectified and mosaicked to generate high-resolution orthophotos. Orthophotos in this workflow are primarily used for:

Main Purposes:

• Constructing unified 2D spatial reference: Providing baseline coordinate system for subsequent analysis
• Supporting precise field boundary extraction: Field boundary identification based on orthophotos
• Providing projection alignment foundation for 3D results: Ensuring spatial consistency between 3D reconstruction results and 2D baseline

Orthophotos serve as 2D baseline and are not directly used for phenotypic analysis.

3. Field Boundary Extraction Based on Orthophotos

Field boundaries are extracted through image-driven processing using orthophotos, with the following workflow:

Extraction Steps:

1. Distinguish crop-covered areas from non-crop regions: Utilize vegetation indices and texture features
2. Remove roads, bare land, and boundary interference: Apply morphological operations and region growing algorithms
3. Generate closed polygons as actual field boundaries: Based on contour extraction and polygon fitting

Extracted field boundaries serve as spatial constraints for subsequent 3D reconstruction and phenotypic calculations.

4. Farmland Crop 3D Reconstruction

4.1 SfM Point Cloud Reconstruction

Structure from Motion (SfM) method is employed for 3D reconstruction from multi-view images, obtaining crop canopy point clouds. SfM reconstruction results are primarily used for:

Application Scenarios:

• Canopy height and spatial distribution analysis: Crop height distribution statistics based on point cloud elevation
• Population-scale structural phenotyping: Calculation of point cloud density, spatial heterogeneity metrics
• Joint analysis with orthophotos and DSM: Multi-source data fusion enhances phenotypic accuracy

4.2 3D Gaussian Splatting (3DGS)

Building upon SfM, 3D Gaussian Splatting (3DGS) method is introduced for continuous surface representation of crop canopies. 3DGS is primarily used for:

Technical Advantages:

• Fine structure expression: High-resolution continuous geometric representation
• Continuous geometric representation of complex canopies: Handling intricate structures like branch-leaf intersections
• 3D visualization and structure exploration: Supporting interactive 3D browsing and analysis

5. 3D Crop Phenotyping Quantification and Machine Learning Analysis

5.1 3D Structural Phenotypic Feature Construction

3D structural phenotypic features are constructed based on crop point clouds, including:

Phenotypic Features:

• Canopy height distribution characteristics: Height statistics (mean, variance, quantiles)
• Point cloud density and spatial heterogeneity metrics: Spatial distribution uniformity, aggregation degree
• Canopy surface undulation and roughness parameters: Surface roughness, undulation amplitude

5.2 Machine Learning Modeling

Machine learning methods are further employed to establish mapping relationships between 3D structural phenotypes and crop biological traits, achieving quantitative analysis of crop phenotypes.

Analysis Workflow:

1. Feature engineering: Extract multi-scale phenotypic features from 3D point clouds
2. Model training: Use random forest, gradient boosting, and other algorithms to build predictive models
3. Validation and evaluation: Assess model performance through cross-validation
4. Application and promotion: Apply models to large-scale crop phenotypic monitoring

6. CanopyPC: Plant Canopy Point Cloud Processing Tool

CanopyPC is a Python-based tool for processing and analyzing UAV-CCO (Unmanned Aerial Vehicle Cross-Circular Oblique) reconstructed plant canopy point clouds. This tool provides a complete pipeline for segmenting, analyzing, and visualizing 3D point cloud data of plant canopies.

Features

Ground-Canopy Segmentation

Separate ground and plant canopy points using Cloth Simulation Filter (CSF)

Point Cloud Preprocessing

• Noise removal using statistical outlier detection
• Small cluster removal using DBSCAN
• Landmark sign removal
• Manual point selection and removal

Canopy Row Segmentation

Automatically segment plant canopies into rows using K-means clustering

Geometric Analysis

• Convex hull volume calculation
• Oriented bounding box (OOBB) analysis
• Projected area calculation
• Plant height statistics

Visualization

Interactive 3D visualization of point clouds, convex hulls, and bounding boxes

Summary

This workflow constructs a complete UAV-based 3D crop phenotyping system, from image acquisition to machine learning modeling, achieving quantitative and intelligent analysis of crop phenotypes. This method offers the following advantages:

• High precision: 3D reconstruction provides rich spatial structural information
• High efficiency: Automated workflow supports large-scale monitoring
• Scalability: Machine learning models can be transferred to different crops and environments

---

This article introduces the complete workflow of UAV-based 3D crop phenotyping. For questions or collaboration opportunities, feel free to contact and exchange ideas.