PhenoNet

PhenoNet

Algorithm

PhenoNet comprised the PhenoViT module and the LSTM network, which only took the RGB information of the image as input and generated the wheat phenophases as output. The PhenoViT module was a lightweight feature extraction module designed to extract high-level feature information from raw RGB images efficiently. The LSTM network sequentially processed the extracted feature information over time, facilitating effective learning of temporal dependencies for the ultimate phenology classification decision.

SOTA Performance

0.945

Accuracy

0.928

Kappa

0.941

F1-score

0.889

Params (M)

0.093

MAdds (G)

8.0

Memory (MB)

imgCompare

Explainability

Why

PhenoViT performs well in tracking the change of color and morphological features in different stages. The visualization results demonstrate that the inclusion of the LSTM network in PhenoViT enhanced its ability to discriminate between different phenophases. Two expected phenomena were shown in the feature space: (i) each data point from different phenophases was as far as possible, and (ii) each data point belonging to the same phenophase was as close as possible.

Transferability

Explore more

Compared to training from scratch, PhenoNet misclassifies fewer samples in each phenophase through transfer learning. For example, PhenoNet without transfer learning misclassified 11 images in the F phenophase. However, the misclassification samples were decreased to 3 through two-step transfer learning. Transfer learning also accelerates PhenoNet convergence. Only training on the WheatPheno2022 requires more than 300 epochs, while the curve of the transfer learning group tends to converge completely around 220 to 270 epochs.

Integratable

PhenoNet OpenAPI

Real-Time Phenology Analysis with PhenoNet Integration. For example, to classify the phenophase by invoking the application programming interfaces (APIs) through Python, users could package their image data and authorization key in a dictionary format. Then, they could upload the dictionary to the APIs endpoint using the HTTPS POST method to submit the classification task. Once the classification was completed, the status code and phenophase would be returned in JSON format. Detailed interface documentation and demo code were provided for users to employ the APIs of PhenoNet.

Deployable

Edge computation

PhenoNet can be deployed on an edge computation device for real-time phenophase classification in the field. The deployment process involves three steps. First, the network is exported to a middle model in the Open Neural Network Exchange (ONNX) format. Second, unnecessary operations and layers in the middle model are removed using a simple Python package called "onnx-simplifier". Third, the middle model is converted to the IR format using OpenVINO's model optimizer and then compiled into BLOB format.

Citation

If you find this work useful for your research, please consider citing our paper:

@article{zhang2024phenonet,
  title={PhenoNet: A two-stage lightweight deep learning framework for real-time wheat phenophase classification},
  author={Zhang, Ruinan and Jin, Shichao and Zhang, Yuanhao and Zang, Jingrong and Wang, Yu and Li, Qing and Sun, Zhuangzhuang and Wang, Xiao and Zhou, Qin and Cai, Jian and others},
  journal={ISPRS Journal of Photogrammetry and Remote Sensing},
  volume={208},
  pages={136--157},
  year={2024},
  publisher={Elsevier}
}

Ready to try?