3D Model-Vinobot-2016_08_16 3D Model-Vinobot-2016_08_29 3D Model-Vinobot-2016_09_01 4D-RGB Model-Vinoculer 3D Model-Vinoculer-2016_08_16 3D Model-Vinoculer-2016_08_29 3D Model-Vinoculer-2016_09_07

Automated Field Phenotyping

in our research, we developed two phenotyping platforms for in-situ characterization of plant responses to changes in their environment (e.g., management practices, drought, flood, heat, etc.). These platforms consist of two robotic systems: a mobile, observation tower, Vinoculer, for canopy characterization and general inspection of the crop; and a ground vehicle, Vinobot, for individual plant phenotyping.

Vinobot(left) Vinoculer(right)

Figure above shows the Vinobot and Vinoculer in a field at the Bradford Research Center near Columbia, Missouri, USA.

Vinobot (ViGIR-lab Phenotyping Robot) is a mobile platform equipped with a range of sensors designed to fulfill both the phenotyping tasks and its autonomous navigation needs.

3D model of individual plants over growing period

Vinoculer (ViGIR-lab Phenotyping Trinocular Observer) is a portable observation tower mounted in the center of a field. It is capable of turning 360 degrees while capturing data from a large area. To see live stream from the vinoculer click here. It was equipped with stereo RGB and IR cameras in order to perform measurements such as volume, leaf area, biomass, height, growth rate and other canopy characteristics.

3D model of entire field over growing period

Together, the Vinobot and Vinoculer are capable of collecting data from a large area at the canopy and individual plant level.

We proposed the use of a trinocular camera rig consisting of a stereo RGB pair plus a thermal camera (Vinoculer) to create multi-dimensional models of the plants in the field. That is, our system allows both appearance and temperature information to be stored and analyzed for plant phenotyping purposes. We refer to it as a 4D-RGB model, since structural (3D), thermal (+1D) and textural (RGB) information are accurately registered and stored.

4D model of entire field over growing period

Robotic Assistive Technology

Surface Electromyographic signals (sEMG) find applications in many areas such as rehabilitation, prosthesis and human-machine interaction. Systems reliant on these muscle-generated electrical signals require various forms of machine learning algorithms for recognition of specific signatures. Those systems vary in terms of the signal detection methods, the feature selection and the classification algorithm used, however, in all those cases, the use of multiple sensors and complex analysis and classification algorithms are constant requirements. In this research, we developed a power wheelchair control system that relies on a single sEMG sensor and a new technique for signature recognition that we call Guided Under-determined Source Signal Separation (GUSSS). Compared to other approaches in the literature, the proposed technique relies on a much simpler classifier and uses a very small number of features to achieve reasonable results.

EMG wheelchair

3D Modeling

Our framework for 3D object modeling consists of six majors steps including 1) Multiple pairs of stereo images are captured by 2 calibrated cameras while the object moves freely with respect to the cameras; 2) A SIFT-based feature extraction algorithm establishes the correspondence between various points on every stereo pair sampled; 3) The intersection between the sets of points from two consecutive pairs of images is determined. That is, common feature points present in both the left-right image pair at camera-object position i and the subsequent left-right image pair at camera-object position i+1 are identified; 4) The 3D coordinates of every point in the intersection above is calculated; 5) The transformation between camera-object poses are estimated using the 3D coordinates of the intersection; and 6) The previous transformations are used to create virtual poses of the camera and fed into a patched-base multi-view software to construct the 3D model of the object.

Illumination-invariant Background Subtraction

Foreground object detection is an essential task in many image processing and image understanding algorithms, in particular for video surveillance. Background subtraction is a commonly used approach to segment out foreground objects from their background. In real world applications, temporal and spatial changes in pixel values such as due to shadows, gradual/sudden changes in illumination, etc. make modeling backgrounds a quite difficult task.

source image for background subtraction result image for background subtraction

In our work, we propose an adaptive learning algorithm of multiple subspaces (ALPCA) to handle sudden/gradual illumination variations for background subtraction.

Articulated Human Body Tracking

Human motion capture has numerous application in human-robot interaction, law enforcement, surveillance, entertainment, sports, medicine, etc. Various methods have been developed to date and they can be categorized into: marker-based or markerless; articulated model-based or appearance-based; single view or multiple view; and so on. Marker-based methods are the simplest ones and therefore are also the methods with most success so far. However, it is obviously not always possible to add markers to the human subjects, and markerless approaches are without a doubt the most general and desirable methods.

human motion human motion

In our work, we explore markerless method for human motion capture. We propose a Bayesian estimation based method which falls into single view and articulated model-based category. The estimator, derived from Particle Filters, was expanded to a hierarchical model by introducing a new coarse-to-fine framework to deal with the computational complexity inherent to Particle Filters.

Image-based Visual Servoing

Any control system using visual-sensory feedback loops falls into one of four categories. These categories are derived from choices made regarding two criteria: the coordinate space of the error function, and the hierarchical structure of the control system. These choices will determine whether the system is a position-based or an image-based system, as well as if it is a dynamic look-and-move or a direct visual servo.

In our work, we present an image-based, dynamic look and move visual servoing system. The difference between our approach and other popular ones is in the use of quaternion representation, which eliminates the potential singularities introduced by a rotational matrix representation.