Movement versus outcome: Where is your focus during shaping?
I recently spent a week in Germany at the Scientific Symposium of The BHV, a German dog training association. I was privileged to get to give a lecture about errorless learning at the symposium. Then, after the symposium, Dr. Jesús Rosales-Ruiz and I gave a two day workshop about shaping to a wonderful group of about two dozen trainers.
During part of one of his lectures at the workshop, Dr. Rosales-Ruiz discussed the concept of shaping movements versus shaping outcomes. This is not a topic often discussed by animal trainers, but it is a really important one. I know that when I first heard about it, it changed the way I think about shaping and clicker training.
I’m sorry I can’t share the entire conference with you, as we had just a delightful time and many wonderful conversations about training. But, I would like to share some of the conference with you. In this blog post I’ll discuss some of what Dr. Rosales-Ruiz presented on the topic of reinforcing movements and outcomes, as well as some of my own thoughts about how understanding this topic can improve your shaping and training skills.
First, some definitions.
For the purposes of this discussion, we will refer to movement and outcome as follows:
Movement. These are the actual physical elements of the response, that is, how the animal moves while performing the behavior. While observing movement, you would watch specific body parts and how muscles change position, the sequence of muscle actions involved, and how the skeleton changes shape.
Outcome. When considering outcome, on the other hand, the focus is on the result. You would look for the impact or change that is produced by the response. That is, you would focus on what the behavior does or produces, rather than the physical movements.
For example, imagine that you want to go to the fridge to get a glass of water. You could walk to the fridge. Or, you could skip, or hop, or walk backwards. You could even crawl or do somersaults. All of these involve very different types of movement. (And notice, walking forward and walking backward are both “walking,” but are actually very different movements because they involve different types and sequences of muscle movements.) Yet for all of these, the outcome is the same — you eventually arrive in front of the fridge.
So, how does this relate to shaping?
Have you ever heard the analogy that the click is like a camera? It is commonly used when describing how clicker training works.
It goes something like this…. Think of the clicker as a camera.When you click, you are taking a picture of that exact moment in time. You are capturing that moment and the desired behavior in the animal’s mind and making it more likely that the animal will repeat that picture.
This analogy is often used to emphasize the importance of needing to have great timing during clicker training. However, it also paints the clicker as a tool that reinforces static images, or outcomes. This metaphor can lead trainers to focus on static positions, rather than looking at and focusing on the animal’s movements.
Animal trainers, especially beginner trainers, often focus on reinforcing outcomes. For example, consider teaching an animal to touch a target. Many trainers start by holding the target close to the animal to greatly increase the chance of success. The trainer then waits until the animal makes contact with the target. This would be an example of reinforcing an outcome. The outcome is the animal’s nose touching the target.
Alternatively, the trainer could begin by reinforcing any time the animal’s nose moved in the direction of the target, even if the animal did not make contact with the target at first. This would be an example of reinforcing movements.
For some simple behaviors that are learned quickly, the movement versus outcome question can seem like splitting hairs, particularly if the trainer does not need the behavior to be done with a huge amount of precision. However for other behaviors, focusing on movement rather than outcome can increase the rate of reinforcement during training and make a giant difference in how the animal performs the final behavior.Many people find shaping difficult precisely because they have been taught to focus on outcomes. Click To Tweet
Many people find shaping difficult precisely because they have been taught to focus on outcomes. They confuse the outcome with the behavior and have not been taught how to shape movement. For example, consider teaching a dog to sit. Most training books teach you to click and reinforce when the dog’s bottom touches the floor.
But, think of all of the ways the dog can get from the standing position to the sitting position!
He can keep his front feet mostly stationary and tuck his back feet in and under. Or, he can drop his back end in place, while moving his front feet back slightly. Or, he can sit down crookedly, so that he ends up on a hip with his legs out to the side.
Or….there are plenty of other possibilities!
Now, if you are training your pet dog to sit and wait while you put his food bowl on the floor, you probably don’t care, as long as he stays sitting until you tell him that it is time to eat. However, if you and your dog compete in obedience or other dog sports, you’ll likely want him to move in a precise way so that he ends up sitting straight and in just the right position next to you. A sloppy sit with the wrong set of muscle movements could cost you a ribbon.Here’s one really interesting example of movement versus outcome for you to consider. You probably know the story about Pavlov. He conditioned dogs to salivate when he rang a bell.
Most people discuss the saliva as if it were the behavior being conditioned. However, the production of saliva is an outcome. The actual behavior was the movement of the muscles and glands that produced the saliva and moved it through the body. This is an interesting example because the behavior is inside the animal and largely invisible, at least with the technology that was available to Pavlov
What can happen if you focus only on outcomes during training?
Focusing on outcomes isn’t always a bad thing. Sometimes it really is only the outcome that is important. However, if you are mostly (or only) reinforcing outcomes, you can end up focusing most of your attention on a narrow slice of the behavior. This means that you are less aware of what the animal is doing at other times during the training and you are less aware of how the animal is actually moving. This can lead to a variety of issues. If you are not careful, you may end up reinforcing:
• The wrong behavior: For some behaviors, it’s not just the outcome that is important. The movements that lead to that outcome need to be executed in a precise way. The correct movement may help achieve superior performance or may prevent injury or fatigue. During teaching, the animal may do the behavior in a way that ultimately leads to basically the correct outcome, but that really is the wrong sequence of muscle movements. If this wrong sequence of movements is repeatedly reinforced, it may be very hard to change later on.
• Different versions of the behavior: If you wait for the outcome before you click, you can end up reinforcing many different versions of a behavior. You lose precision because the “behavior” you are trying to train will actually end up being a handful of different but similar behaviors, that all result in basically the same outcome. This may also be confusing to the animal, as the animal may feel like you are clicking for a different behavior each time. Often, trainers do need the animal to do a precise version of the behavior, whether for competition, or to keep the animal in a certain balance or posture for health reasons, or so that the behavior can be incorporated into a larger sequence of behaviors. All of this can be difficult to achieve if you initially reinforce several different versions of the behavior.
• Unwanted behavior chains: If you mainly focus on reinforcing outcomes, you can end up creating chains of behavior that include both the behavior you want and other unwanted behaviors. For example, in teaching dogs not to jump up, trainers often recommend clicking when the dog has all four feet on the floor. This often works, but I have also met dogs subjected to this training protocol that have inadvertently learned to jump once or twice, then stand patiently waiting for their treat. The whole sequence becomes: dog approaches a person –> dog jumps up briefly –> dog then stands and waits for his treat. The whole sequence ends up getting reinforced. It is important to remember that the clicker doesn’t just capture what is happening at the moment of the click. It can, and often does, reinforce the whole sequence of behaviors that precede the click.
• “Messy” behaviors: I often see trainers focus on outcome initially, with the idea that they can further refine the behavior later on. So, the trainer reinforces the horse for taking six steps back, whether his head is up or down, whether he is well balanced or not, whether his ears are forward or back, whether he backs in a straight line or curves some to the side. However in the horse’s mind, all of these responses are now appropriate ways to walk backward. It can be difficult to clean up the behavior later on and narrow it down so that the horse is only backing in a straight line, in good balance, with his head at the right height and with his ears forward.
Part of the difficulty with trying to clean up or refine a “messy” behavior is that every time you click for a correct part of the behavior, you often end up also reinforcing incorrect parts of the behavior. If you clicked for ears forward while the horse walked backward, but the horse was out of balance, you’ve now reinforced ears forward, but also the incorrect balance. (How to clean up a messy behavior? This would need to be a whole separate post. However, rather than trying to clean it up, the easier solution is often to just reteach a more precise version of the behavior by reinforcing the correct movements from the beginning, and then give the new behavior a new cue.)
Behavior cycles: Where does the behavior begin? Where does the behavior end?
We’ve covered a lot in this post, and perhaps you already have enough to think about. But, I will leave you with one final concept to help further clarify this question of movement versus outcome and to help you as you decide what to reinforce during shaping.
Each response has a beginning, a middle, and an end. But how do we know when one behavior ends and the next repetition begins?
One thing that can help is to think of behavior as a cycle.
The behavior really isn’t over until the animal is in a position to do the behavior again. So, the animal is in the same position at the beginning and end of the behavior.Think of behavior as a cycle. The behavior isn't really over until the animal is in a position to do it again. Click To Tweet
This is more intuitive for some behaviors. Imagine a dog or horse going over a small jump. Although the trainer sometimes focuses on clicking when the animal is in midair, we usually think of the behavior as the entire sequence of movements: the animal approaching the jump –> beginning to take off –> clearing the jump –> beginning to descend –> successfully landing after the jump –> and beginning to move away. The animal is now back on the ground and in a position to go over another jump. The behavior cycle begins and ends when the animal has all four feet on the ground.
However, this notion of behavior cycles is less intuitive for other behaviors. Think of a dog sitting. We usually think of sitting as the following sequence: dog is standing –> dog begins to lower his bottom –> dog ends up in the sit position with his bottom on the floor. However, if the dog is sitting, he is not in a position to sit again until he is standing back up. So, the behavior cycle of sitting really includes the dog moving from standing to the sit position and then back to standing again. Similarly, for a behavior such as head lowering, the full cycle would include the animal lowering his head, but then raising it back up again.
Trainers often focus their clicks on the middle of the cycle, as this is often the outcome the trainer is looking for. Clicking in the middle of the cycle would mean clicking when the dog’s bottom touches the floor during a sit or when the horse’s nose almost touches the ground during head lowering.
However, the behavior cycle is made up of a sequence of movements. And thinking of behavior as a cycle will help you to start to see all of the movements.
The trainer can move her click earlier (or later) in the cycle to achieve different results. If you need a very precise version of the behavior (rather than a collection of responses that result in a similar outcome), you will want to begin by clicking very early in the cycle and reinforcing the precise opening movement and correct muscle patterns that will lead the animal to the version of the movement that you desire.
Where is your focus?
In many cases it is practical and easy to just focus on outcomes and click for results of behaviors. I do this myself and it can be sufficient for some training situations.
However, if you find that you are having trouble shaping a particular behavior or that your animal is offering lots of extra or unwanted behaviors, stop for a moment and consider what you are currently reinforcing and also what you focused on reinforcing when you initially trained this behavior. Were you focused on reinforcing physical movements or were you focused on reinforcing outcomes and results?
Leave a comment below and let us know what you think about the topic of movement versus outcome!
And, if you found this blog post interesting, I encourage you to join us for the 8th annual Art and Science of Animal Training Conference. We’ll be spending the first day of the conference talking all about shaping. I know in particular that Kay Laurence’s talk about micro-shaping will be very much focused on how we can reinforce movements during shaping. You can find more information about the conference here.
Excellent article on shaping. I have trained using both methods, depending on the situation.
I’m really looking forward to the Art & Science days in Feb. 2016.
Thanks so much!
Thank you for this informative article! I’ll be sure to share it with the other trainers at work.
This captures the reason one needs great observation skills and why video is helpful. It can be quite valuable to carefully watch an animal engaged in a sequence of movements which result in a specific outcome. For example, I may watch many repetitions of horses laying down naturally in order to determine which version I desire. This would drive which movement I start R+ and how we proceed. Does the head drop first or the back feet step under first? Watching an uninterrupted expression will tell me.
Thanks for the thoughtful comment.
That’s a really great point to start by watching your animal or video of that animal (or even slow motion video of the animal!) doing the behavior naturally.
I think one reason why so many trainers aren’t good at reinforcing movements is that we often haven’t watched the movement enough to actually understand how it happens.
I am rather a novice at this, and am confused about 1 concept. In behavior cycles, for example the dog goes from stand to sit to stand again to complete the cycle. Are you saying that you shape the sit, and then shape the stand from the sit; or shape both separately and then put them together; or that the click ends the behavior and releases the dog to stand for reinforcement?
Good question, Nancy.
Yes, the full cycle is stand –> sit –> stand. People often think of this as two separate behaviors. But, following the logic of the cycle, it can also be thought of as one.
So, if you are following the cycle, you can start by training the first half (stand to sit) and then after that train the second half (sit to stand). While working on the first half, you can use reinforcer delivery (such as tossing a treat to the side) to get the dog to stand back up.
After the behavior is trained, the cycle can always be expanded, such as adding duration to the sit.
Hope this helps answer your question!
My comments or questions are primarily about extinction/fading of micro-movement clicks and the rewards, clarification on outcomes, and the animal’s emotional state/frustration with the process or the trainer’s timing.
Extinction/fading of clicks/rewards.
I agree and appreciate shaping micro-movements helps the animal to understand what he/she did is what I’m looking and to do more of “that” movement.
recap: micro-movements are all the behaviors from the start to the finish –> that lead to the precise outcome we are seeking, such as in teaching a dog to Sit (without the use of placement –> only shape, lure or capture is allowed).
However, at some point, the animal offers the micro-movements and demonstrates he/she is closer to the outcome I’m looking for so I eventually need to stop rewarding the micro-movements because they are learned and so I need to move on to the next micro-movement or the chain of movements to reach the outcome. Is that a good strategy? Did I understand the definitions?
Here’s a training goal I often hear/see from pet dog parents/caregivers:
I want the dog to ring the PoochieBell (the terminal behavior) which is hanging from this door knob (an exterior door to a patio or backyard) so that I’m aware she wants to go outside. I take her outside or give her access to exit and hopefully she eliminates while she’s outside.
What is the outcome in this scenario? Is the outcome essentially what I want the dog to do (ring the bell) or what the dog wants me to do (open the door)?
In this case would the outcome be the dog’s signal (ringing of the bell), or is it the dog’s comprehension or awareness of his/her actions that communicate to the owner/trainer/caregiver that he/she needs door opening services?
About the animal’s emotional state/frustration with the shaping process or the trainer’s timing.
When a dog is “clicker charged” –> he/she understands each click delivers a reward, and they can get anxious/frustrated because the trainer is waiting to click for outcomes vs. click for micro-movements, could this be the reason why they get frustrated?
The dog starts to guess or offer things he/she has done in the past to produce a click (followed by a reward), but not necessary any of the new micro-movements I’m looking for and when I don’t respond with a click, he/she gives up or barks in frustration.
Cindy, these are awesome questions. Here are a few thoughts about each of them.
Question 1 (Extinction/fading of clicks/rewards):
Yes, that is a good strategy. Start by reinforcing the micro-movements at the beginning of the movement. Then, gradually expand so that you are reinforcing a larger piece of behavior.
The nice thing about reinforcing micro-movements is that it actually usually requires less extinction. Because you are reinforcing the right movement (rather than looking at outcome) the animal is usually ready to offer or even already offering the next part of the movement, and you can then move your click forward.
Question 2 (Outcomes):
This is a really great question.
For the purposes of movement vs. outcome, we are just looking at the behavior. So, the dog’s nose making contact with the bell would be the outcome.
In addition to outcomes, we probably need another word such as “accomplishments” (or consequences) that describes the end result outside of the behavior / behavior cycle. So, in the example you give, as a result of ringing the bell, the accomplishment is that the dog gets to go outside to potty.
Another example would be a human going to a gas station and paying money to buy a lotto ticket. That would all be part of the behavior. Receiving the ticket would be the outcome. Occasionally (on a not very probable schedule) this behavior also accomplishes the winning of a large sum of money.
Question 3 (Emotions/frustration):
The frustration often comes because of a low rate of reinforcement. If you are working on an improbable micro-movement, one strategy that you can use is to alternate between the new behavior you are working on and a different, easy behavior that the animal already knows well. So, do several repetitions of the new behavior you are shaping, then several repetitions of an easy behavior, then several more repetitions of the new behavior.
This can increase the rate of reinforcement and also reduce frustration. For this to work, however, you do need some sort of cue or change in context so that the animal knows when you are switching from the easy behavior to the new, harder behavior. For example, horse trainer Alexandra Kurland will alternate between targeting while standing up, and shaping the new behavior while slouching against the wall.
However, if the dog is continuing to get anxious or frustrated, this can also indicate that the trainer might need to make adjustments to the training environment so that it is easier for the dog to figure out the behavior.
I LOVE this article. It would also love to translate it into Polish – I run a free website about dog training and I translated so far few articles from other trainers. I think this blog post is just amazing. Please let me now if you agree.
Emilia, we would love to have you translate our article into Polish!
I will send you an email right now and we can talk further.
Hi Mary. As a remedial Horse Trainer I have to concentrate on “calm and relaxed” throughout all stages of training.I have a new horse in yesterday with haltering issues….ear fear per say and fear of the haltering process. This means that catching is an issue too. She lives in flight mode! With everything I start to do with her shaping and teaching her that she can now participate, I must be aware of her emotions throughout.The danger of horse like this is that you can soon shape behaviours but unwittingly be reinforcing the fear and tension within the shaping process.
I agree completely!
When shaping any behavior, we need to be concerned with both the physical movements and with the animal’s emotional state. Understanding the movement cycle can help with this, because you should be able to see the animal’s emotions from the beginning of the cycle.
This can let you know early on when you need to do something differently. But, the challenge is figuring out how to begin the cycle without the undesired emotion.
Great article, and I also enjoyed the questions and answers following.
I think there are times where outcome reinforcement works, but far more circumstances where movement reinforcement is a much better choice. One of my earliest training jobs involved marine mammals that were collected in the wild. There was no way you could just reinforce outcome. We would reinforce movement to slowly get the animals to come close enough to present a target, and then reinforce movement toward the target and eventually get a target touch. That is only one example,there are many more. I think it is a more positive experience for the animal and it helps prevent frustration by setting them up better to succeed.
Thanks again, Debbie
Thanks for stopping by and leaving a comment. Great example of your work with marine mammals, thanks for sharing it. I agree that reinforcing movement often helps reduce frustration!
I feel gobsmacked with an aha moment!
Sometimes when I do shaping, it goes well and rapidly.
Other times, it seems like I’m having a lot of trouble even with things that seem like they should be simple.
I just realized the difference in the two is whether I’m focused on movement or outcome.
With less complex behaviors, I’m often focusing on outcome.
With complex behaviors, it’s like there’s no choice but to focus on movement.
So teaching a horse to stand on a pedestal was easy for me.
Teaching “easier” things, has sometimes been harder.
This article just helped me diagnose the problem.
I’m glad you enjoyed the article! This concept was also an “aha” moment for me when I first learned about it. 🙂
A clever insight and wonderful suggestions you have on your site. You’ve got obviously spent lots of time on this. Well done!