Third Annual Cinnamon Roll DOE

When I teach experimental design concepts, I could discuss varying machine settings such as machine speed, temperature, pressure, product mixes, etc. to see how each one affects some important output measure such as production rates or off-quality.  Or, I could show the students how these concepts come alive by varying the basic recipe of my cinnamon rolls. No brainer, right? So, for three years in a row, I’ve been doing just that. Today marks the Third Annual Cinnamon Roll DOE (Design of Experiments).

Here are the new questions I wanted this experiment to answer:

1. Do females have different preferences re: cinnamon rolls than males? Here, I was speculating that females preferred chocolate variations whereas males would not.

2. Is a chocolate chip dough preferred to the basic plain dough that I normally use?
3. Is a chocolate icing better than the plain sugar glaze that I normally use?

4. Should I top the cinnamon rolls with mini chocolate chips or not?
5. Is it OK to freeze the cinnamon rolls? I experimented with this previously in the Second Annual Cinnamon Roll DOE and learned that freezing the rolls appeared to hurt customer satisfaction. My students seemed very surprised by the result. In fact, most said they didn’t even realize that some of the rolls had been frozen! Therefore, I wanted confirmation of these initial findings. I realize that freezing the cinnamon rolls wouldn't necessarily make them taste better. I was just hoping that the test subjects couldn't tell a difference so that I could be able to make these rolls well in advance for my friends and co-workers.

As before, for those of you who just want to bypass the details and simply find the answers to these questions, simply scroll to the bottom of this post. For the rest of you . . .

Here is another way of listing the factors, or changes, that I wanted to try:

Factor:            Current Condition:     Change to Try:
A: Gender        Female                      Male
B: Dough          Plain                         Chocolate Chip
C: Icing             Glaze                       Chocolate
D: Topping        None                        Chocolate Chips
E: Frozen          Not Frozen               Frozen

I made 8 different batches of cinnamon rolls, each with differing combinations of above. 

Four of the batches were made fresh this morning (not frozen):

Batch 2: Plain dough with plain sugar glaze icing and no topping, then tasted by a male (This is actually the status quo condition, or the control group.)

Batch 4:  Chocolate chip dough with plain sugar glaze icing and chocolate chip topping, then tasted by a male

 

Batch 5: Plain dough with chocolate icing and chocolate chip topping, then tasted by a female

  
Batch 7: Chocolate chip dough with chocolate icing and no topping, then tasted by a female

The other four batches were made a couple of weeks ago and frozen. I took them out of the freezer to thaw last night:

Batch 1: Plain dough with plain sugar glaze icing and chocolate chip topping, then tasted by a female

 
Batch 3: Chocolate chip dough with plain sugar glaze icing and no topping, then tasted by a female 

Batch 6: Plain dough with chocolate icing and no topping, then tasted by a male

Batch 8: Chocolate chip dough with chocolate icing and chocolate chip topping, then tasted by a male 

As before, I asked each student/co-worker who wanted to participate in this experiment to taste a particular batch (or batches) of cinnamon rolls. (A big thanks to all who participated!) Then, each was asked to provide a "likeability" score (using a 1-5 scale) by completing the following survey:

This time, I collected 10 surveys for each batch, for a total of 80 observations. After analyzing the results, I learned that freezing the cinnamon rolls, adding chocolate chips to the dough, and using chocolate icing significantly affected the likeability score. This is depicted in the chart below by the very long bars.

Even though most of the responses were very favorable, I learned that freezing the cinnamon rolls (Factor E) had the most significant negative effect on the likeability score. Darn. That’s twice now. One of my students suggested that next time, perhaps I should freeze the rolls before baking them instead of after baking them. So, I’ll be anxious to try that in the next round of experimentation. (See last graph below.)

Using chocolate chip dough (Factor B) also had a negative effect on the likeability score. That is, the average likeability score from those batches where I used the chocolate chip dough was statistically significantly lower than the average likeability score from those batches that I used the plain dough. (See second graph above). Hindsight, I think this makes sense. I noticed that the chocolate chip dough didn’t rise nearly as much. I guess the chocolate chips were just enough to weigh it down.

One surprising result for me was the negative effect due to the chocolate icing (Factor C in the third graph above). Really? Gosh, I loved it! It did, however, reinforce a key teaching point that I’ve been making for years:  expect to be surprised! (This is why businesses should be making data driven decisions instead of strictly relying on gut . . . but I admit I'm pretty biased here.)

Interestingly, there was essentially no difference in the likeability score among the females and the males (Factor A). Also, there wasn’t a significant difference between having chocolate chip toppings or none at all (Factor D). See the first and the fourth graphs above.

For this experiment, I found a clue where possibly the dough and freezing method interact; that is, what happens when you combine these two variables? The data suggests that if I use the chocolate chip dough combined with freezing the batch, the likeability score plummeted. Using the plain dough, however, there wasn’t much of a difference between the likeability score of a frozen batch or a fresh batch. So for me, that means hope! Perhaps if I just leave the dough alone, then freezing may be OK after all.

Another potential interaction involved gender and icing. Now this one did not surprise me. It appears that males preferred the plain sugar glaze icing whereas the females preferred the chocolate icing. Since I loved the chocolate icing myself, perhaps I can make this variation for, say, my all-ladies book club!

Bottom line: These results were more like the very first cinnamon roll DOE. For most of the factors, it looks like I should simply "leave well enough alone." That is, stick with the plain dough, plain sugar glaze icing (unless I’m sharing with a bunch of ladies only!), and don’t bother with the chocolate chip topping.

I'm still hopeful that freezing the rolls will not hurt the likeability score as long as I don’t modify the dough. So once again, that means more testing. I’m already looking forward to the next round of experimentation!