Developing a Taste for Interactions

The other day, I ran across the idea of making red velvet cinnamon rolls on Pinterest. I giggled when I learned that Keely's reaction essentially matched mine:

Put "red velvet" in the title of pretty much any recipe, and we're going to try it. We've made traditional red velvet cakes from scratch, red velvet truffles, red velvet whoopie pies, red velvet pancakes, etc. Instead of using the given recipe for the cinnamon rolls above, however, my plan was to experiment with my basic recipe of cinnamon rolls that I've been making for years.

Given the fact that I'm teaching a class this week on experimental design techniques, I thought this would be the perfect time to do some experimentation myself. I was speculating that my students and co-workers would gladly be my test subjects (a.k.a. guinea pigs), and I was right! :)

There are several questions that I wanted this experiment to answer:
1. Is the red velvet dough preferred to the basic plain dough that I normally use?
2. Is a cream cheese icing better than the plain sugar glaze that I normally use?
3. Is it worth it to put chopped pecans on top or should I just leave them off?

For those of you who just want to bypass the details and simply find the answers to these questions, simply scroll to the bottom of this post. For you technical types who actually enjoy having more detail about how I conducted this experiment, keep reading. (Hey, the rest of you, stop rolling your eyes!)  :D

Here is another way of listing the several factors, or changes, that I wanted to try:

Factor:          Current Condition:        Change to Try:
A: Dough        Basic                           Red velvet
B: Icing           Plain sugar glaze        Cream cheese icing
C: Pecans      No pecans on top        Add pecans on top

I made 4 different batches of cinnamon rolls, each with differing combinations of above. Specifically, I made the following batches:

Batch 1: Basic dough with plain sugar glaze, and chopped pecans on top

Batch 2: Red velvet dough with the plain sugar glaze, but with no pecans on top

Batch 3: Basic dough with cream cheese icing, but with no pecans on top

Batch 4: Red velvet dough with cream cheese icing, and chopped pecans on top

I asked each student/co-worker who wanted to participate in this experiment (trust me . . . no twisting arms on this request) to taste a particular batch (or batches) of cinnamon rolls. Then, each was asked to provide a "likeability" score (using a 1-5 scale) by completing the following survey:

I collected 10 surveys for each batch, for a total of 40 observations. Thankfully, most participants enjoyed all 4 different batches of cinnamon rolls. Here is one of my students, Donna, getting ready for her first sample. I didn't realize until later that I caught one of my co-workers, Brian, "licking his chops" in the background! In fact, it was Brian who wrote above: "Holy cow!" :D

After analyzing the results, I learned that the dough that I used significantly affected the likeability score. This is depicted in the chart below by the very long bar that exceeds the reference line.

Even though most of the responses were very favorable, I learned that changing the dough had a negative effect on the likeability score. That is, the average likeability score from those batches where I used the red velvet dough was statistically significantly lower than the average likeability score where I used my basic dough (see first graph below). This surprised me!

Although changing the icing wasn't technically significant, the average likeability score from those batches with the cream cheese icing appears to be heading in the wrong direction, too, as shown in the second graph above. Since it isn't significant, I would keep it at the status quo condition anyways. That is, stick with the plain sugar glaze icing.

Lastly, it appears that the average likeability score is about the same, whether or not you put pecans on top. See the last graph above. So, at this point I would leave it at the status quo condition (no pecans) and save money.

Uncovering potential interactions is one of the benefits of running this type of experiment where you change multiple things simultaneously. That is, what would happen to the likeability score if you combine two of these changes? In this experiment, I found a clue where possibly the icing and the pecans interact, which was a little odd I thought.

When there were no pecans on top, the cream cheese icing was preferred over the glaze. When there were pecans on top, then the glaze icing was preferred over the cream cheese icing. Hmmmm. Since I'm planning on sticking with the glaze icing, now I wonder if I should invest in the pecans after all? It's just a clue, though. In fact, it is most likely that the significance is due to changing dough type anyways.

Bottom line: It looks like I should leave well enough alone. In other words, stick with the basic dough, stick with the plain sugar glaze icing, and don't worry about investing in pecans. I guess this particular experiment supports the old adage:

"If it ain't broke, don't fix it!"