When
I teach experimental design concepts, I could discuss varying machine settings
such as machine speed, temperature, pressure, product mixes, etc. to see how
each one affects some important output measure such as production rates or
off-quality. Or, I could show the students how these concepts
come alive by varying the basic recipe of my cinnamon rolls. No brainer, right?
So, for three years in a row, I’ve been doing just that. Today marks the Third
Annual Cinnamon Roll DOE (Design of Experiments).
Here
are the new questions I wanted this experiment to answer:
1. Do females have different preferences re:
cinnamon rolls than males? Here, I was speculating that females preferred
chocolate variations whereas males would not.
2.
Is a chocolate chip dough preferred to the basic plain dough that I
normally use?
3. Is a chocolate icing better than the plain
sugar glaze that I normally use?
4.
Should I top the cinnamon rolls with mini chocolate chips or not?
5. Is it OK to freeze the cinnamon rolls? I
experimented with this previously in the Second Annual Cinnamon Roll DOE and learned
that freezing the rolls appeared to hurt customer satisfaction. My students
seemed very surprised by the result. In fact, most said they didn’t even
realize that some of the rolls had been frozen! Therefore, I wanted
confirmation of these initial findings. I realize that freezing the cinnamon
rolls wouldn't necessarily make them taste better. I was just hoping that the test
subjects couldn't tell a difference so that I could be able to make these
rolls well in advance for my friends and co-workers.
As
before, for those of you who just want to bypass the details and simply find
the answers to these questions, simply scroll to the bottom of this post. For
the rest of you . . .
Here is another way of listing the factors, or
changes, that I wanted to try:
Factor: Current
Condition: Change to Try:
A: Gender Female Male
B: Dough Plain
Chocolate
Chip
C: Icing
Glaze Chocolate
D: Topping
None Chocolate
Chips
E: Frozen
Not Frozen
Frozen
I made 8 different batches of cinnamon
rolls, each with differing combinations of above.
Four of the batches were made fresh this
morning (not frozen):
Batch 2: Plain dough with plain sugar glaze
icing and no topping, then tasted by a male (This is actually the status quo condition, or the control group.)
Batch 4: Chocolate chip dough with plain
sugar glaze icing and chocolate chip topping, then tasted by a male
Batch
5: Plain dough with chocolate icing and chocolate chip topping, then tasted by
a female
Batch
7: Chocolate chip dough with chocolate icing and no topping, then tasted by a female
The other four batches were made a couple of
weeks ago and frozen. I took them out of the freezer to thaw last night:
Batch 1: Plain dough with plain sugar glaze
icing and chocolate chip topping, then tasted by a female
Batch 3: Chocolate chip dough with plain sugar
glaze icing and no topping, then tasted by a female
Batch 6: Plain dough with chocolate icing and no
topping, then tasted by a male
Batch
8: Chocolate chip dough with chocolate icing and chocolate chip topping, then
tasted by a male
As before, I asked each student/co-worker who
wanted to participate in this experiment to taste a particular batch (or
batches) of cinnamon rolls. (A big thanks to all who participated!) Then, each
was asked to provide a "likeability" score (using a 1-5 scale) by
completing the following survey:
This time, I collected
10 surveys for each batch, for a total of 80 observations. After analyzing the
results, I learned that freezing the cinnamon rolls, adding chocolate chips to
the dough, and using chocolate icing significantly affected the likeability score.
This is depicted in the chart below by the very long bars.
Even though most of the responses were very favorable, I learned that freezing
the cinnamon rolls (Factor E) had the most significant negative effect on the
likeability score. Darn. That’s twice now. One of my students suggested that
next time, perhaps I should freeze the rolls before baking them instead of
after baking them. So, I’ll be anxious to try that in the next round of
experimentation. (See last graph below.)
Using chocolate chip
dough (Factor B) also had a negative effect on the likeability score. That
is, the average likeability score from those batches where I used the chocolate
chip dough was statistically significantly lower than the
average likeability score from those batches that I used the plain dough.
(See second graph above). Hindsight, I think this makes sense. I noticed
that the chocolate chip dough didn’t rise nearly as much. I guess the chocolate
chips were just enough to weigh it down.
One surprising result
for me was the negative effect due to the chocolate icing (Factor C in the third graph above). Really?
Gosh, I loved it! It did, however, reinforce a key teaching point that I’ve
been making for years: expect to be
surprised! (This is why businesses should be making data driven decisions instead of strictly relying on gut . . . but I admit I'm pretty biased here.)
Interestingly, there was essentially no difference in the likeability score
among the females and the males (Factor A). Also, there wasn’t a
significant difference between having chocolate chip toppings or none at all
(Factor D). See the first and the fourth graphs above.
For this experiment, I found a clue where possibly the dough and freezing
method interact; that is, what happens when you combine these two variables?
The data suggests that if I use the chocolate chip dough combined with freezing
the batch, the likeability score plummeted. Using the plain dough, however, there
wasn’t much of a difference between the likeability score of a frozen batch or a
fresh batch. So for me, that means hope! Perhaps if I just leave the dough
alone, then freezing may be OK after all.
Another potential
interaction involved gender and icing. Now this one did not surprise me. It appears that males preferred the plain sugar
glaze icing whereas the females preferred the chocolate icing. Since I loved
the chocolate icing myself, perhaps I can make this variation for, say, my
all-ladies book club!
Bottom line: These results were more like the very first cinnamon roll DOE. For
most of the factors, it looks like I should simply "leave well enough
alone." That is, stick with the plain dough, plain sugar glaze icing
(unless I’m sharing with a bunch of ladies only!), and don’t bother with the
chocolate chip topping.
I'm still hopeful that
freezing the rolls will not hurt the likeability score as long as I don’t
modify the dough. So once again, that means more testing. I’m already looking
forward to the next round of experimentation!
