Fourth Annual Cinnamon Roll DOE

This week, I taught another class on experimental design concepts. I really enjoy what has now become an annual tradition of demonstrating this Lean Six Sigma tool by experimenting in class with my basic cinnamon roll recipe. (Pinch me! I get paid to do this!) :D

Each year, I try to come up with a different theme. The first class I tried this simulation in enjoyed red velvet variations. Two years ago, the class got to try different orange variations. Last year's theme was chocolate. This week? I wanted to experiment with healthy substitutions. Yeah, it's pretty ironic to see "cinnamon rolls" and "healthy" in the same sentence. The challenge was on!

Here are the new questions I wanted this experiment to answer:

1. Would a healthier substitute for the sugar that I use in the dough make a difference? What if I used natural honey instead of refined white sugar?

2. Would a healthier substitute for the oil that I use in the dough make a difference? Lately, I've been using coconut oil for so many things. Maybe it would work for the cinnamon rolls, too?

3. Would a gluten-free cinnamon roll be preferred? Doubt it. But, would it at least be tolerable? One blogger swears that tapioca flour mixed with coconut flour produces a great dough. (And, if it's on the internet, it must be true. Right?)

4. Would a healthier substitute for the brown sugar that I use in the filling make a difference? What about using coconut palm sugar instead? It even looks like brown sugar!
5. Would a healthier substitute for the butter I use in the filling make a difference? I first learned about ghee last year when I completed a Whole 30, and I've been cooking with it ever since. Ghee is clarified butter that is pure butterfat without the milk solids and water. It is better tolerated by those with dairy sensitivities, and it's even found to lower levels of unhealthy cholesterol. Now, wouldn't it be nice for a doctor to say, "Since you need to lower your cholesterol, eat cinnamon rolls!"  (Keep dreaming . . .)

6. Would a healthier substitute for the really sweet glaze icing make a difference? What if I used natural honey here, too?

7. Would placing a label on the package (either gluten-full or gluten-free) make a difference? Here, I was speculating that some people may look at a gluten-free label and already have some preconceived biases about the taste before even taking a bite.

As before, for those of you who just want to bypass the details and simply find the answers to these questions, simply scroll to the bottom of this post. For the rest of you . . .

Here is another way of listing the factors, or changes, that I wanted to try:

Factor:                   Current Condition:     Change to Try:
A:Dough Sugar        Regular sugar            Honey
B: Dough Oil             Canola oil                   Coconut oil
C: Dough Flour         Gluten-full                   Gluten-free
D: Filling Sugar         Brown sugar               Coconut palm sugar
E: Filling Butter          Regular butter            Ghee

F: Icing                       Regular glaze             Honey

G: Label                     No label                       Label

I made 8 different batches of cinnamon rolls, each with differing combinations of above. 

The first four batches were made with the regular bread flour (gluten-full):

Batch 1: Dough made with regular sugar, canola oil, and bread flour. Filling made with coconut palm sugar and ghee. Honey icing and no label.

Batch 2: Dough made with honey, canola oil, and bread flour. Filling made with brown sugar and real butter. Honey icing and a gluten-full label.

Batch 3: Dough made with sugar, coconut oil, and bread flour. Filling made with brown sugar and ghee. Regular glaze icing and a gluten-full label.

Batch 4: Dough made with honey, coconut oil, and bread flour. Filling made with coconut palm sugar and real butter. Regular glaze icing and no label.

The last four batches were made with the tapioca and coconut flour (gluten-free):

Batch 5: Dough made with sugar, canola oil, and gluten-free flour. Filling made with coconut palm sugar and real butter. Regular glaze icing and a gluten-free label.

Batch 6: Dough made with honey, canola oil, and gluten-free flour. Filling made with brown sugar and ghee. Regular glaze icing and no label.

Batch 7: Dough made with sugar, coconut oil, and gluten-free flour. Filling made with brown sugar and real butter. Honey icing and no label.

Batch 8: Dough made with honey, coconut oil, and gluten-free flour. Filling made with coconut palm sugar and ghee. Honey icing and a gluten-free label.

As before, I asked each student/co-worker who wanted to participate in this experiment to taste a particular batch (or batches) of cinnamon rolls. (A big thanks to all who participated!) Then, each was asked to provide a "likeability" score (using a 1-5 scale) by completing the following survey:

As before, I collected 10 surveys for each batch, for a total of 80 observations. After analyzing the results, I learned that only one thing made a difference: changing the flour significantly affected the likeability score. This is depicted in the chart below by the very long bar.

This was the boldest factor I've tested in the various cinnamon roll DOEs. I wasn't surprised at all by the results. When I picked up the dough to knead it into rolls, the gluten-free dough felt like dried crumbly Play-doh. It was awful! It seemed to take forever just to get it to stick together. I'm certain that if I had collected data on some visual assessment in addition to the likeability score, it would have bombed here, too. They were a hot mess: little tiny blobs that never did rise. (See graph below.)

Like working with dry, crumbly Play-doh

What a mess!

Not only did the gluten-free dough significantly decrease the likeability score on average, but it also significantly increased variation. You'll notice below that survey responses spanned the entire range of the rating scale from 1 to 5. In other words, opinions were all over the place!

Interestingly, there was essentially no difference in the likeability score when changing any of the other factors. The more I started thinking about the analysis, though, the more I was afraid that I may have missed something. It was very apparent when observing the different taste testers that there were two distinct groups of individuals: those who cared about healthy alternatives and those who couldn't care less. I was afraid that the extreme effect of the dough change may have overshadowed other important ingredient changes. Hindsight, I probably should have done a separate DOE for those two customer segments. 

It wasn't practical for me to start the baking process over again. Instead, I created a subset of my data, focusing on only those recipes that used the regular bread dough (gluten-full) and continued to explore the ingredient changes. 

It turns out that a couple of other factors were indeed significant after all! 

Changing the original sweet regular glaze icing to honey appears to hurt the likeability score when you're only focusing on the nice, gluten-full cinnamon rolls. 

On the other hand, it appears that using coconut oil instead of the canola oil appears to significantly improve the likeability score. We found a winner!

Bottom line: People preferred the nice, gluten-full cinnamon rolls over the gluten-free alternative. Not even close. When using the original bread flour (gluten-full), it appears that changing the icing to honey hurt, so keep using the regular glaze icing. If you want to reduce some of your sugar intake, though, it would be OK to substitute honey for the sugar in the dough (not icing) and substitute coconut palm sugar for the regular brown sugar in the filling. You could also substitute ghee for regular butter. Those three substitutions didn't hurt the likeability score. I'm also getting a clue that I should be changing the canola oil to coconut oil. I'd like to test that one again just to be sure. As before, I’m already looking forward to the next round of experimentation!