Puppy Forum and Dog Forums banner

1 - 20 of 57 Posts

·
Registered
Joined
·
1,719 Posts
Discussion Starter #1
So can you all tell me more about the concept of a no-reward marker? I mean, I know what it is - just a marker that means "that's not what I want so you don't get a treat," right? But...isn't that really just a very mild aversive? Isn't that a very fine line? Maybe this has been discussed here before, but I'm curious!

I've been thinking about this because we did a free-shaping session last night at agility class (so much fun!) and in our "homework" instructions it says not to say "no" or "eh eh" (i.e. don't use a NRM) because it can be discouraging to the dog. Maybe this is only true for dogs who are just learning how free-shaping works?

Don't get me wrong - I'm not saying there is anything wrong with an NRM even if it actually is an aversive. I see the usefulness, just like I see the usefulness of having a way to tell my dog to stop doing [name her chosen undesirable activity]. Just wondering what y'all think about this. Thanks!
 

·
Registered
Joined
·
339 Posts
I think in a free-shapping, trick training, and all the good stuff, NRMs should be avoided. You're not trying to manage behaviors or anything, so why use them? It's all about having fun and learning new things. I personally don't think NRM are bad, they just have situations where they would be most appropriate to use. I use "no" and "eh eh" as a reminder for my dog to not do what she's about to do. The NRM is quick and short, and it gets the message across clear. I like using NRM for example on a walk, if she's wandering too far away from me I use "hey" or "eh eh" to let her know that's as far as she can go. I prefer this over pulling on her leash or restricting her in a physical way. When I'm trying to train a new behavior or work on a current one, I never use NRMs.
 

·
Registered
Joined
·
11,145 Posts
I think it depends on how much training your dog has had, too. By that I don't mean how "well-trained" they are but how experienced they are at learning new things and how much the two of you have worked together, if that makes sense.

I have done the most and the most varied training with Squash out of all my dogs, he's been in various classes pretty much continuously since his puppy class and he's a year and a half old now. He knows what NRMs mean so well that I don't think he finds them discouraging or sees them as an aversive at all at this point. Would I have used them in his very first tricks class? No. Do I use them with him using training new tricks or skills now? Sometimes yes, sometimes no. But it's not something I automatically don't use, because he knows the game so well.


ETA: I didn't really properly address totally free shaping or just capturing behaviors... I don't see tons of use to NRMs in that context for a novice dog and trainer, at least in the early stages. Part of the point is teaching the dog to offer behaviors, as much or more as teaching any specific behavior. Once they kind of get the basics of what you want I think they're useful to refine something specific, though. And again, I think for an experienced dog who is used to NRMs and comfortable with offering behaviors, it can be kind of a shorthand or shortcut to wade through some of the chaff they offer.
 

·
Registered
Joined
·
5,406 Posts
I think it is a very fine line between a NRM and a mild aversive. I would be more likely to use one for something I really don't want the dog doing (counter surfing, or getting ready for a squat on the carpet for instance). But then it becomes an interrupter instead of just information. I might still use it in such a situation, though I do think by then it has become a mild punisher. I would not use one in free-shaping, as I think the more latitude you give the dog in that situation, the better. And I don't want to do anything that discourages the dog from trying new things. So, when free shaping, I just try to keep my mouth shut.
 

·
Registered
Joined
·
3,511 Posts
I wouldn't use a NRM when trying to do free-shaping because the whole idea is to get the dog to offer new behaviors and I wouldn't want to discourage that. The only training I've used NRM in was when I had to redefine what sit, down, stand meant to Jubel. He's still not 100% on stand either, I didn't teach him sit or down he knew them when I adopted him. In his mind all those commands meant come close to me, usually in front of me, and sit/down/stand. I wanted him to sit/down/stand where ever he was when I gave the command, to teach that I needed the NRM as well as a small barrier (I used a small cardboard box) placed in front of him.

I can't really think of any other training I'd do that would need a NRM. For me the NRM is used mostly as a verbal correction/interrupter for unwanted behaviors often followed by sit/drop it/leave it/off.
 

·
Registered
Joined
·
3,342 Posts
Going against the grain here ... I think the use of an no-reward marker IS most useful when free shaping. In fact, it's probably about the only time when I would even consider using one.

ie: if I'm patiently waiting out a slight movement of the RIGHT paw, and the dog keeps offering LEFT paw after LEFT paw after LEFT paw ... I *might* use an NRM in an attempt to provide useable information and prevent the dog from shutting down completely. In other words, in the above scenario at least, it might actually be more deterimental if one were to idly stand by, merely withholding the click while the dog went through multiple 'incorrect' repetitions.
The fundamental premise of an NRM is to ultimately "keep" the dog in the game, and lessen the likelihood of him bailing out in the face of no forthcoming reinforcement.


You can find a bit more info on their use / non-use, here ..
http://www.clickertraining.com/node/179

and here ...
http://www.clickertraining.com/node/2848
 

·
Registered
Joined
·
1,719 Posts
Discussion Starter #8
I wouldn't use a NRM when trying to do free-shaping because the whole idea is to get the dog to offer new behaviors and I wouldn't want to discourage that. The only training I've used NRM in was when I had to redefine what sit, down, stand meant to Jubel. He's still not 100% on stand either, I didn't teach him sit or down he knew them when I adopted him. In his mind all those commands meant come close to me, usually in front of me, and sit/down/stand. I wanted him to sit/down/stand where ever he was when I gave the command, to teach that I needed the NRM as well as a small barrier (I used a small cardboard box) placed in front of him. I can't really think of any other training I'd do that would need a NRM. For me the NRM is used mostly as a verbal correction/interrupter for unwanted behaviors often followed by sit/drop it/leave it/off.
Funny you mention sitting/downing at a distance -- that's our homework for agility! Biscuit thinks "down" means "come to me and lie down." We've been teaching it with one person holding the leash and other person giving the command from a distance and clicking/tossing treats, but the NRM has definitely crossed my mind.
 

·
Registered
Joined
·
1,719 Posts
Discussion Starter #9
Going against the grain here ... I think the use of an no-reward marker IS most useful when free shaping. In fact, it's probably about the only time when I would even consider using one.

ie: if I'm patiently waiting out a slight movement of the RIGHT paw, and the dog keeps offering LEFT paw after LEFT paw after LEFT paw ... I *might* use an NRM in an attempt to provide useable information and prevent the dog from shutting down completely. In other words, in the above scenario at least, it might actually be more deterimental if one were to idly stand by, merely withholding the click while the dog went through multiple 'incorrect' repetitions.
The fundamental premise of an NRM is to ultimately "keep" the dog in the game, and lessen the likelihood of him bailing out in the face of no forthcoming reinforcement.


You can find a bit more info on their use / non-use, here ..
http://www.clickertraining.com/node/179

and here ...
http://www.clickertraining.com/node/2848
I see what you're saying, but I think that would be very situation-depending. If Biscuit is offering behaviors while free-shaping, but not doing what I want, I feel like that would mean I was asking too much from her, I should take it back a step and make it easier. If she was shutting down and wandering away, I might give her a hint to keep her in the game, but that's a separate thing from NRM. But, she doesn't know the game that well yet.

I use an NRM in daily life, as some of you have described, to interrupt and derail undesirable activities - I see Biscuit starting to counter surf or some such, I say "Eh Eh," she cuts it out and goes to do something else, hopefully something more productive. It doesn't freak her out. But I can see how if I was trying to get her to offer behaviors as she thinks them up, the NRM would be discouraging.

I'm glad I'm thinking about this in a sensible way. Thanks guys!
 

·
Registered
Joined
·
5,406 Posts
Going against the grain here ... I think the use of an no-reward marker IS most useful when free shaping. In fact, it's probably about the only time when I would even consider using one.

ie: if I'm patiently waiting out a slight movement of the RIGHT paw, and the dog keeps offering LEFT paw after LEFT paw after LEFT paw ... I *might* use an NRM in an attempt to provide useable information and prevent the dog from shutting down completely. In other words, in the above scenario at least, it might actually be more deterimental if one were to idly stand by, merely withholding the click while the dog went through multiple 'incorrect' repetitions.
The fundamental premise of an NRM is to ultimately "keep" the dog in the game, and lessen the likelihood of him bailing out in the face of no forthcoming reinforcement.
If I'm truly "free shaping" I am allowing the dog to be creative, and often what I get is better (and different) than what I may have originally had in mind. So, unless he's peeing on my leg, the dog can do no wrong. This is subtly different from just shaping an intended behavior. If I am doing directed shaping, I may make things easier for the dog to make the step I want by positioning myself differently, splitting the behavior into tinier pieces, etc. If my dog is really clicker savvy, I don't need to use my voice. Absence of a click means "try something else".
 

·
Registered
Joined
·
3,342 Posts
I use an NRM in daily life, as some of you have described, to interrupt and derail undesirable activities - I see Biscuit starting to counter surf or some such, I say "Eh Eh," she cuts it out and goes to do something else
The way these words (eh eh, no! etc) are being described / used in this thread sounds more like they're verbal corrections, ... by my own definition anyways.

Subtle differences ... one being, NRM's "keep the dog IN the game". ("oops" and "TRY AGAIN" are typical marker words that are employed.)
 

·
Registered
Joined
·
3,342 Posts
If I'm truly "free shaping" I am allowing the dog to be creative, and often what I get is better (and different) than what I may have originally had in mind. So, unless he's peeing on my leg, the dog can do no wrong. This is subtly different from just shaping an intended behavior. If I am doing directed shaping, I may make things easier for the dog to make the step I want by positioning myself differently, splitting the behavior into tinier pieces, etc. If my dog is really clicker savvy, I don't need to use my voice. Absence of a click means "try something else".
Ah, I think I see what you're saying now.

(*brain fart, on my part re: terminology*)

My bad. Oops. Try again. LOL
 

·
Registered
Joined
·
7,262 Posts
The way these words (eh eh, no! etc) are being described / used in this thread sounds more like they're verbal corrections, ... by my own definition anyways.

Subtle differences ... one being, NRM's "keep the dog IN the game". ("oops" and "TRY AGAIN" are typical marker words that are employed.)
I would classify those words (no! hey! etc) in a similar way: interrupters or verbal corrections. My understanding of NRM is I ask for a "sit," dog does something else (stares at me, lies down, etc), I use a NRM (e.g., "sorry" or "nope") to say, "that's not what I asked for; no reinforcement for you." The NRM would be used only after the dog has learned the cue fairly solidly, not while first teaching the cue. Perhaps I'm misunderstanding the terminology.
 

·
Registered
Joined
·
179 Posts
Occasionally while shaping my dogs will get into a rut and do the same behavior repeatedly. I usually use a NRM to get them out of that. I think this probably stems from my use of extinction bursts to get more of certain behaviors out of them. They think that maybe if they do something for long enough then it'll be enough and I'll click for it. I say "oops" in a happy tone and if they're interacting with something, I remove it for a second and then replace it. The NRM is just a way of communicating that whatever they're doing is a waste of their time and isn't what I'm looking for. Basically, telling them to try something else.

I also use it for weaves, because SiSi does them independently and gets no marker for doing them correctly. She just gets no treat for doing them incorrectly, so I started saying "oops!" when she does it wrong so she knows there will be no treat and doesn't spend ages looking for one.

Neither of my dogs have seemed discouraged by a NRM. On the contrary, I often use it to stop them from getting frustrated during shaping. I wouldn't call it aversive, although I guess it is a punisher because it's decreasing the frequency of a behaviour.
 

·
Registered
Joined
·
9,391 Posts
I think we're talking about secondary subtleties.
1. Consider the game Hot/Cold - you can use a NRM as Cold, and a Keep Going Signal (KGS) as Hot ... You might use this for searching, nosework, fetching a ball with an inadequate retrieve and drop.
2. If you're trying to perfect a behavior, changing a sloppy Sit into an obedience Sit, you may need a NRM to help fine tune, if you haven't been incrementally increasing the requirement for reward.
3. Although not all NRM are aversive - some are instructive; Not all aversives are bad. If you drop food on the ground and tell the dog Leave It!, that is an aversive for most dogs. When you are training your dog and the time to stop has come, getting no more treats is an aversive. We do this all the time.
4. My dog paws me. When he was a pup, I taught him to shake and gave him a treat. I was a little slow one day, so he pawed me with the other paw.... I recognized that and was able to teach him Right and Left.... I glance away or shake my head a bit as an NRM, and he offers an alternate behavior (as opposed to a completely different behavior) I also do the same when asking him to count by barking.

I believe that if you give your dog lots of training and appropriate freedom, he'll learn independence, especially Retrievers and Herders. You have to put limits on the freedom, and I think it's good to have a signal ... to reduce freedom in some situations.

I imagine that everyone re-directs and corrects their dog, most without punishment.... dogs want more food, more walks, to play all day and/or sleep all day in the bed. So they learn limits. ... My dog follows me into the kitchen, hoping for a treat. I accidentally drop something on the floor. He doesn't move, but he watches it intently. I pick it up and toss it in the trash... a kind of silent correction. However, later, I'll drop a piece of kibble, intentionally, and he'll snatch it in mid-air. If he miscalculates, he'll pick it up off the floor. He knows the difference with no additional prompting from me... based on previous training of Leave It!

In early days of computer assisted instruction (CAI) (now called computer based training CBT), researchers found that purely positive methods, praise or no feedback, would frustrate advanced learners. The advanced learner would get into a loop where they believed that an incorrect behavior or rule was correct. No positive approach could pull them back. So they needed a correction for the incorrect behavior... which relieved the frustration of going around in circles. ....it works with dogs, too...
 

·
Registered
Joined
·
44 Posts
Not really an answer to your question, but Emily Larlham just wrote about errorless learning vs no reward markers.
The article defines an NRM as a punisher, which is not correct. Melissa Alexander's dfinition (NRM = extinction marker) is also not correct.

A no-reinforcement marker is a neutral signal that is a secondary (i.e., conditioned) no-consequence. Neutral means not aversive and not appetitive. It only becomes a punisher if it reduces the frequency of a specific behavior.

NRMs are most useful to keep things moving in shaping with a dog that has experience with shaping and also has built up some duration with some behaviors (e.g.. long down). It is definitely not necessary to use an NRM, and probably not beneficial for most training programs. As many of the comments here suggest, what people are calling NRM is really a secondary punisher, or what starts out as NRM becomes a punisher.

"Errorless" training is setting up training scenarios to maximize the frequency of reinforcement, and thus minimizing the number of trials required to attain fluency. It is the most efficient way to acquire behavior. However it does not exercise the problem-solving skills of the dog.


Although not all NRM are aversive - some are instructive; Not all aversives are bad. If you drop food on the ground and tell the dog Leave It!, that is an aversive for most dogs.
No proper NRM is ever aversive, by definition. If it has become aversive then it has become a secondary punisher and has ceased to be an NRM. Whether "leave it" is aversive or not depends on how the behavior was trained. If it was trained using positive reinforcement, then the cue predicts an opportunity for reinforcement and is not aversive. This is also an example of a non-aversive punisher (cue for incompatible behavior). Of course if "leave it" was trained through aversive methods then the cue is a secondary punisher, and thus aversive.

When you are training your dog and the time to stop has come, getting no more treats is an aversive.
That is not correct. Dog wanting more treats = appetitive. The risk here is inadvertent negative punishment (treats are removed from the environment), which is the reason for the training 101 rules: (a) always end on a good note and (b) finish up with a game.
 

·
Registered
Joined
·
869 Posts
The article defines an NRM as a punisher, which is not correct. Melissa Alexander's dfinition (NRM = extinction marker) is also not correct.

A no-reinforcement marker is a neutral signal that is a secondary (i.e., conditioned) no-consequence. Neutral means not aversive and not appetitive. It only becomes a punisher if it reduces the frequency of a specific behavior.


If it is a neutral signal, then what purpose does it serve? If it's neutral, it doesn't do anything.
 

·
Registered
Joined
·
952 Posts
Wether a NRM like "Oops" issued in a bright cherry tone of voice is a punishment or not depends on your dog and the condition that you issue it in. My dog is absolutely crazed for certain agility equipment, the teeter, the A frame and the dog walk. So in that instance, I would not worry about a NRM being a punishment. Because it will Not diminish her drive for the agility equipment. at all. And usually that's what we are always afraid of in agility, decreasing drive. Because most people spend so much time trying to build drive and positive association with the equipment. Me, I need to decrease drive and build focus. So a NRM in agility in theory would just tell her wrong answer try again, but in reality she's totally blowing me off and gunning for the equipment and so she'd never hear the NRM in the first place.

Now in free shaping, in my house no agility equipment near by so she's totally focused and calm... I'd probably not use a NRM.... unless like previous posters mentioned... the dog was getting frustrated after offering the same behavior over and over again with no feedback. If I "oops-ed" her then she might realize that she needs to try a new tactic. OR she might just get frustrated with the whole game and want to leave. I am still a newbie to free shaping and don't do it often.
 

·
Registered
Joined
·
1,719 Posts
Discussion Starter #19
Thanks all. This is a really interesting discussion! It seems like the NRM sits right on the line between punishment and not, and it depends on the dog, the activity, the tone of voice used, the specific word used, and which way the wind is blowing that day!

Last night Bisc and I were doing some free shaping and for kicks I said "Oops!" in a happy voice when she tried to chew on the plastic bin we were using as a shaping object. It did set things back - I had to go back to rewarding her for any interaction with the bin, three or four times, before we got back to where we had been before I said it. On the plus side, my dog now knows the very useful skill of putting her front feet in a plastic bin. :)
 

·
Registered
Joined
·
44 Posts
If it is a neutral signal, then what purpose does it serve? If it's neutral, it doesn't do anything.
It signals the end of a trial that has no consequence.

It seems like the NRM sits right on the line between punishment and not
If it is +P, it is not NRM. If it is NRM, it is not +P.

NRM has to be conditioned, just like any other marker. A dog that has not been conditioned for "oops" as no consequence, hearing it for the first time, can interpret it in any number of ways.

Here is an example of NRM that most are probably familiar with.
Context: quiz show where contestants attempt to be first to press a button in order to answer a question
Antecendent: question presented
Behavior: quickly press button and offer answer
Consequences:
if correct answer, bell rings and contestant point total incremented (+R)
if incorrect answer, buzzer sounds

The buzzer sounding signals no consequence and the trial is over. This will not by itself reduce the frequency of the contestant pressing the button to answer future questions. The buzzer is a NRM.

If instead the incorrect answer gets a buzzer and point total decremented, this is punishment. It reduces the frequency of the behavior. The buzzer is a conditioned punisher.
 
1 - 20 of 57 Posts
Top