Q & A with Dr. Fred Hulme: Conducting an Effective Trial

Q: I’m interested in switching the controlled release fertilizer I’m currently using with a product from a different manufacturer. I know I should probably perform a trial first, but I’m not sure of how to get started. Do you have any guidelines to offer?

--Chris H., Nursery Grower, Michigan

A: Chris,
You’re absolutely correct in your assumption that conducting a trial should be the first step before making any significant change to your production practices. As I’m sure you’re aware, the purveyors of new products and crops will typically try to sell them to growers by mentioning a variety of desirable features and benefits, such as improved profitability and easier production, or offer solutions to common problems. Some features or benefits offered by these products may appear so lucrative that many growers adopt them immediately. However, in most cases, it’s necessary to get some hands-on experience first.

Before even getting started, thoroughly review the prospective supplier’s claim support pieces (internal and third party research data, other grower trials and testimonies, product sheets, etc.). You might also ask company representatives about other growers’ trials and talk to those growers directly about their experiences. The more you learn up front, the more likely it will be that you can conduct a meaningful trial on your own. If you decide to move forward with the trial, take these guidelines into consideration as you plan ahead:

  • Test only one new product or crop at a time. If you make multiple changes at once, you can’t positively know what factor or interaction of factors was responsible for the trial results.
  • Limit the number of treatments so the analysis doesn’t get too complicated. If you perform a trial with multiple products, rates and application methods, you might end up with a mountain of data that is difficult to sift through, leading to inconclusive results.
  • Develop an objective and a protocol that truly measures the stated benefits and claims of any new product or crop. Clearly define and document these objectives and timetables in advance. The more details you can include, the more useful this information will be later.
  • Do a financial assessment to estimate the potential benefits of any change to your current practices. This includes determining if long-term incorporation of the new product will require any change in irrigation, regulatory issues, handling, storage and equipment.
  • Determine what factors are most important to you, and measure them during the trial. Common decision factors include overall plant performance, plant size, foliar color, rooting, crop timing, nutrient availability, price, cost-in-use, product convenience and ease of application.
  • Plan to include a grower practice control (what you currently do) as a baseline. While you may be quite satisfied with your current production systems, if you don’t compare the new product against your current system, you may not see potential improvements. All the plants grown in your current program may be very uniform, but there may be hidden potential in a new product that you won’t notice unless you place your current practice side by side.

After doing all the pre-trial analysis and planning, it’s time to set up your trial and get it underway. Trials may seem to require a lot of tedious detail work in advance; but the fact is this—a poorly set trial can often lead to incorrect conclusions, cause confusion and end up being a big waste of time. Here are a few tips that can help your trial go more smoothly and deliver more accurate results:

  • Determine when you will make a final assessment of the trial. Most likely this will be at harvest, but it may be sooner.
  • Trial new products on crops types that will respond to potential features and benefits of new products. Test a variety of response types (e.g. heavy feeders compared to salt sensitive/ slow growing crops; plant species susceptible to a disease or pest compared to a more resistant crop).
  • Trial new product on crops/ situations where previous problems have occurred.
  • Conduct the trial at the correct time of year and under typical environmental conditions.
  • Set up an ample sample size. It should be big enough to be significant and observable, but does not include so many plants to be financially risky. Results will be more accurate if the sample size provides a representative test in your operation. Generally, a bench, half of a house or a whole bed is usually a good rule per plant species.
  • Use good quality, uniform liners/ plugs, and plant the same cultivars in the same container sizes on the same date across treatment comparisons.
  • Replicate treatments in blocks to minimize confounding variations in temperature, light, humidity and irrigation.
  • Mark trial plants clearly by labeling each pot if possible. If you don’t take this step and end up uncertain about which treatment is which, the whole trial could be an enormous waste of time and resources.
  • Try to keep plants from being moved during the trial.
  • Subject all treatments to normal cultural practices (e.g. spacing, shading, sprays, pruning, etc.) unless advised otherwise.
  • Make sure key employees engage in the trial and observe the test blocks frequently.
  • Document, document, document. Record data and observations and take photos throughout the trial for later use.
  • In the final assessment, involve a number of employees to get the best comprehensive overview. Include your head grower, sectional growers, decision makers (owners/ general managers) and any supply company representatives.

A well-conceived and conducted trial will yield very useful information about how a new product or practice will work in your current operation. The key is to think carefully ahead of time so you ask the right questions. Follow-up and thorough documentation are essential to ensure that you’ve collected the information you need before you make a significant change.

Have a question for Dr. Hulme? Simply e-mail it to fred.hulme@scotts.com. Your question may be featured in an upcoming issue of The Scotts Exchange™!