Problem
A field phenotyping campaign produces hyperspectral cubes, LiDAR point clouds, RGB images and thermal images for every plot. Each type is normally processed in separate software with its own parameters and export formats, which makes multi-modal trait analysis slow and hard to reproduce.
MCTP (Multi-modal Crop Trait Processing) was developed for the field walking phenotyping platform of Shufeng Bio. I was responsible for system optimization and for the data-processing and analysis methods.
Approach
One platform hosts four modality modules with the same interface: a control panel, live preview and processing log, interactive parameter tuning, and JSON/CSV exports.
Hyperspectral
ENVI HDR/SPE reading and wavelength parsing; automatic band detection for vegetation indices; NDVI with NIR-threshold plant masks and percentile glare masks; leaf-region mean spectra (CSV) and summary statistics (JSON); batch processing of whole directories.
LiDAR
PLY, LAS, LAZ and text input; RANSAC ground rebasing, voxel downsampling, cropping and height colouring; DBSCAN plant segmentation tuned interactively in the SegTuner window; coverage, voxel occupancy, convex-hull volume and height percentiles H10/H50/H90; cropped point-cloud and JSON report export.
RGB
Vegetation segmentation from three colour indices (ExG, CIVE, VDI) with a joint threshold; glare, label and border removal; adaptive morphology and skeleton enhancement for thin plants; watershed and connected-component labelling of individual plants; overlay images, plot-level JSON metrics and per-plant CSV tables.
Thermal
BMP images with DDT temperature matrices, with flipping and calibration; threshold suggestions from Otsu's method and percentiles; sliders for temperature threshold, HSV range and morphology; overlay, heatmap and plant-only previews; PNG, NPY, CSV and JSON export.
Results
MCTP turns four separate processing chains into one consistent path from raw files to structured, statistics-ready outputs. Hyperspectral and RGB data are processed in batch; LiDAR and thermal parameters are tuned on a sample and reused across the campaign. The recommended workflow keeps every step inspectable:
- Archive the raw data and acquisition metadata unchanged.
- Open a representative sample and confirm file pairing, orientation and units.
- Tune parameters on the preview and check the difficult cases.
- Process a small batch and compare with the raw data, then scale up.
- Store thresholds, voxel sizes, clustering settings and the software version with the results.
Limitations
Each module processes its own modality; the modalities are not yet registered to each other. Results depend on calibration, ground selection, point density, occlusion and thresholds, which is why every module exposes its parameters and previews. The next steps are cross-modal registration, time-series analysis across campaigns, shared project configuration templates, and server-side batch processing and reporting.
