Multispectral and hyperspectral cameras help measure biomass, stem thickness,
and leaf characteristics.
Rien den Boer
In 2017, the Department of Agriculture at Purdue University (West Lafayette, IN, USA; www.purdue.edu) required a facility that fosters the progression of research in the area of digital plant phenotyping—the measurement of plant features including biomass, stem thickness, number of leaves, and leaf angles.
Realizing that these developments would not take place in the old greenhouses on campus, the decision was made to build the Controlled Environment Phenotyping Facility (CEPF, ag.purdue.edu/cepf), where the environment is controlled in a climate chamber from Conviron (Winnipeg, MB, Canada; www.conviron.com). In this facility, plant data acquired helps the researchers study and understand the relationship between phenotype and genotype and enables them to find ways to adapt plants to climate change.
As a plant’s phenotype—the result of its genotype and environment—refers to the expression of observable characteristics, a vision system is a great tool to observe and quantify such features. The CEPF holds up to 256 plants up to 4 m tall, each of which is routinely sent via conveyor (Figure 1) to two imaging chambers installed by Aris (Eindhoven, The Netherlands; www.arisbv.nl). To do this, an operator uses an experiment management system from Indigo Logistics (Barendrecht, The Netherlands; www.indigologistics.nl/en/home) to schedule plants to the imaging chambers and decide when to water the plants with an automated dosing system.
Due of the size of the plants, the camera systems required custom components, including a 2 x 5 m lightbox holding 90 matrix LED panels from Lumitronix (Hechingen, Germany; www.lumitronix.de). The system uses proprietary white LED panels (32 pieces, 150 W peak power each) and LED tubes made from solid-state Oslon (32 pieces, 450 nm blue, 200 W peak power) from Osram (Munich, Germany; www.osram.com).
Inside of the first chamber—the RGB imaging chamber—plants are rotated continuously (Figure 2a), where short, intense LED strobe lighting illuminates the plant. Used for image capture in this chamber are two ace acA2440-20 machine vision cameras (one monochrome model and one color model) from Basler (Ahrensburg, Germany; www.baslerweb.com). This camera features the 5 MPixel Sony (Tokyo, Japan; www.sony.com) IMX264 global shutter CMOS sensor and can reach speeds of 23 fps via GigE interface.
The cameras use M1224-MPW2 and M2518-MPW2 fixed-optic lenses from Computar (Cary, NC, USA; www.computar.com) for side views where the distance to the plants is constant, and a Varioptic Caspian C-39N0-160 auto focus liquid lens from Corning (Corning, NY, USA; www.corning.com) to adapt the focus for the large size range of the plants (20 cm to 4 m).
Both cameras, along with the lenses, are placed inside an MSI4 beam splitter box from Ellips (Eindhoven, The Netherlands; www.ellips.com). The system (Figure 2b) uses “template lighting,” whereby a monochrome image provides an automatic background separation mask for the color image. Such a mask can be derived from a chlorophyll fluorescence image (Figure 2c) that results from a short flash of blue LEDs (excitation), allowing the system to record the far-red emission from the chlorophyll-containing plants through a longpass filter from Midwest Optical Systems (Palatine, IL, USA; www.midopt.com), which blocks the blue light.
The cameras are connected to a Super Micro Computer (San Jose, CA, USA; www.supermicro.com) server based on Ubuntu 16.04, running the OpenCV-based (www.opencv.org), vision application and the Aris Phenotyping Framework, which ensures that recipes from the experiment management software convert to commands for the cameras and strobe controller.
The vision software application captures the images, stores them for archiving on a server at Purdue, and extracts a list of important generic plant features, including width, height, area, and color. More specific or custom features can be computed offline by researchers by processing the raw but segmented images archived on the data storage.
After imaging through the RGB system, the plants enter the hyperspectral imaging chamber, which is built around the MSV-500 VNIR camera from Middleton Spectral Vision (Middleton, WI, USA; www.middletonspectral.com), which features a 5.5 MPixel scientific CMOS image sensor that offers 2000-point spatial resolution, 1000-point spectral resolution, and a frame rate of 90 fps over USB3 or 100 fps over Dual Camera Link interface.
This camera also offers either a V8 or V10 spectrograph from Specim (Oulu, Finland; www.specim.fi). The V8 spectrograph offers a spectral range of 380 to 800 nm and a spectral resolution of 6 nm, while the V10 offers a spectral range of 400 to 1000 nm and a spectral resolution of 2.8 nm.
In this chamber—which also features an array of tungsten lights—image stacks (hyper cubes) are captured by the hyperspectral camera and stored onto the university’s data storage server for later processing. Hyperspectral data captured in this application is used by the research team for determining levels of nitrogen or other nutrients in the plants.
The 7,300-square-foot CEPF facility is led by Dr. Yang Yang, Director of Phenomics, Purdue University, and Chris Hoagland, Facility Manager. Additional project partners include Agrinomix (Greenhouse technology; Oberlin, OH, USA; www.agrinomix.com), Bosman van Zaal (Greenhouse technology; Aalsmeer, The Netherlands; www.bosmanvanzaal.com/en), and Phenokey (Phenotyping products and technology; Gravenzande, The Netherlands, www.phenokey.com).
