[Ice222]

I don't know how else to title this, and I'll probably get a lot of people thinking that I mean collar jerking or hitting, but what I really want is ideas on how to get a dog to know when what he's doing is not acceptable? eg. deliberately ignoring a command they know in favor of doing something else..

Right now, walking away or removing attention or giving him a timeout is pretty much the main form of punishment I currently use and it is effective in situations where I can use it, but what other options do I have? I have heard of redirecting behavior, but I have tried it (maybe I'm doing something wrong), and it simply does not make sense to me. For example, in the case of a dog that was playing with a toy starts biting on some wires instead, if I redirect by giving him a toy to chew on instead then won't I be doing several things that are counter productive to training?

a) Won't I be rewarding his biting of wires by giving him attention?
b) Also giving him an actual reward - the toy
c) Attempting to trade the lower value toy for the higher value wire (since he was already playing with the toy, grew bored, and then decided the wire's more novel and fascinating)

I've tried just acting very displeased and giving him a firm "no" before redirecting, but my displeasure really seems to mean little to him, so that doesn't really work as a punishment. Plus there are misbehavior that can't be redirected, eg. when he's ignoring a recall. I can't ignore it, since he will probably go off and find something to amuse himself with, and thus reward himself for disobeying . I can't put him in time out, since I'd have to catch him first, and I think he will soon figure out that he could run away to avoid being caught. And I can't really redirect it, since if he's running off, then clearly what I have to offer is not nearly as worthwhile to him as whatever has got his attention.

My dog Sherlock is much like his namesake: clever, independent, stubborn and endlessly curious and driven. It's very hard to keep his attention, and makes it even harder when he feels that he can deliberately ignore me if he doesn't feel like doing what I say. Of course I do try and reward and strengthen everything that he already knows, but it just doesn't feel like rewarding alone is enough, so I am looking for other options for training better discipline/compliance.

 

[swellmomma]

I am brand new to dog ownership so I may be doing things wrong but with my puppy I Give a very firm no with a snap of my fingers followed by a finger point. WHen it came to things like biting wires in addition I would pull her away, after all that is a safety issue. Now at 14 weeks all it takes is "no" *finger snap* and she 8/10 will stop and lay down. It is not 100% yet, I still have to actively move her away from something when it really has her interest.

When it comes to ignoring recall I would use a long training lead outside for sure, and inside if needed. That way no matter when you say "come" dog can be brought to you if they ignore.

1 thing I did read was never call dog to you to discipline, always go to dog (of course when teaching come that doesn't follow that guideline). Anyway, if you call the dog to come to you for discipline it will learn that resonding to your call is a negative thing. I always 100% make responding to come a happy thing. WIth lots of love and praise everytime. If she does not respond the first time I call her I keep calling her and when she finally listens I heap on the love and praise. SHe usually comes running when I call her because she knows nothing but good stuff happens when she does. I do not discipline for disobedience in this regard. It is all about the positive reinforcement.

Generally speaking I think of training my dog just like teaching/disciplining my children. Discipline does not mean punishment, it means teaching.

Another thought with the dog that abandons the toy for a wire. (trust me we went through several chewed wires in the training process), is that he is bored, so find a way to engage with him and do some training. If he is the type that would rather explore for a while, hide treats/chews that he can have and find on his own. Generally I keep chew treats in my puppy's toy box, but occasionally I will randomly place them around for her to find.

 

[Amaryllis]

I don't discipline. Never did. Management, redirection and training are key. With them, you don't need discipline. (If they're doing something dangerous, I use recall.)

First of all, dogs do not think like people. People think in terms of cause and effect. I bite the wire, cause, I get attention, effect. Dogs DO NOT think in terms of cause and effect. Dogs associate. In the example with the wire, the puppy bites the wire, you give them a toy, they bite that, they get praised. The puppy comes to associate chewing on toys with praise and chews on the toy. (Also, they sell wire covers, I'd recommend buying them.)

Puppies don't chew on things to piss you off, or because they lack a middle finger with which to flip you the bird. They chew because they do. You need to manage that behavior by removing from their reach things you don't want them to chew and redirecting by providing them with things to chew and training by praising chewing toys greatly to encourage it.

If you want, I'll tell you what happens to an overdisciplined dog. Oh, and it's the dog that decides he's overdisciplined.

 

[hanksimon]

To echo Amaryllis - when the pup bites the wire, he wants to chew something. Give him a toy to chew. If he doesn't like that toy (after you offer it... don't read his mind ), then have a special toy available as a distraction. Jiggle it. Also, you are allowed to pick up the puppy and remove it from dangerous situations, then provide the toy. You can also block access - puppy proof the house, so that he can't chew inappropriate things.

Pups have a very short attention span, so he wouldn't associate the new chew toy as a 'reward' for chewing wires.

He is not ignoring you... he is distracted by something of more interest. So you have to incrementally make yourself even more interesting with training and treats and attention beforehand.... It's an ongoing process.

BTW, after I taught my dog Bite Inhibition by yelping at him, I was able to yelp at him to communicate "Please don't do that." It won't work with all dogs... and yelping doesn't mean much until you train it...

 

[chubby]

I don't discipline either,

Check out some Positive Training books and blogs - some good places to start: Dog Star Daily.com (Ian Dunbar), and the book "The Other End of the Leash" by Patrician McConnell, and "Kikopup" on Youtube.

Positive Training uses distraction, redirection, and positive reinforcement instead of punishment. Much more effective, and in the end, your dog listens because he finds joy in having a harmonious relationship with you, not because he's fearing you.

Also, keep in mind that your dog will work better with you the more time you spend training with them. You really get out of it how much you put in. Incorporate training into your lifestyle, and always reinforce good behaviour with attention, pets, treats, and play time. Bad behaviour that's ignored will eventually dissipate unless they're self reinforcing in themselves (like peeing inside the house), in which case, you need to incorporate management as well (keeping forbidden objects out of reach, supervision, prevention).

 

[hargyle]

I taught Loki the "leave it" command. Every time he would get something he was not suppose to have "leave it" and if he stopped he got a treat, Or If I had to take it and re direct him he got a treat. I started though by watching the videos on the training forum stickies ... "its your choice" ... I think thats what it was called.

Redirecting is your best bet from what I have learned... remove from the situation and stimulate him in another, more rewarding way - - be that play or training session. Maybe invest in a bottle of Bitter Apple to spray on the wires, to he learns that these are not tasty things to chew on.

 

[Amaryllis]

Also, keep in mind that when you discipline, you're teaching your dog what you don't want him to do. That's nice, but the problem is, there are about 5,000,000 things you don't want your dog doing, and only a few you do want him doing. It's a much more efficient use of time to teach him what you do want him doing instead. Dogs are simple creatures. They do what gets them what they want. They want attention, play and food. If you give them those things for doing something (i.e., chewing a toy), they'll chew a toy. If you discipline instead, you may end up having to discipline for chewing the wire, then the remote, then the couch, then your shoes . . . you'll spend all your time disciplining!

 

[doxiemommy]

Just to add, management, in my opinion is super important. If you can PREVENT the "bad behavior" before it even starts, then you are one step ahead. So, get the wire covers, OR just don't let him have access to the wires. That may mean rearranging furniture, OR, just watching him very closely, so that you can stop him BEFORE he gets to the wires.

For intance, puppy walks over in the direction of the wires, you distract him BEFORE he even gets close to them. Give him a toy, play a quick game with him, ask for a command. What you've done is distract, redirect, and really take the option of chewing wires out of his list of cool things to do.

If you PREVENT him from even getting to the wires, pretty soon he doesn't even consider that as a choice of something to do.

 

[Ice222]

First of all, dogs do not think like people. People think in terms of cause and effect. I bite the wire, cause, I get attention, effect. Dogs DO NOT think in terms of cause and effect. Dogs associate. In the example with the wire, the puppy bites the wire, you give them a toy, they bite that, they get praised. The puppy comes to associate chewing on toys with praise and chews on the toy. (Also, they sell wire covers, I'd recommend buying them.)

You call it association, but 'a' is followed by 'b' is the most basic learning concept and a more generalized version of understanding causation. If you don't think that dogs can understand that at all then you are underestimating how smart dogs are, in fact that would put them below even the intelligence of pigeons and rats, because even they can understand 'press a button and food will follow'. If what you mean is dog can't learn 'a' is followed by 'b' then 'c', and that they will only learn the 'b' to 'c' association (ie. that biting the correct toy is good, and not that biting wire will get you toy/attention) then I can somewhat agree, since most dogs will only associate the most recent behavior to the reward. But for others, attention in itself, is already a reward, and can learn 2 associations rather than one: 'biting wire' is followed by attention, 'biting toy' is followed by 'praise/treat'. I just want to have some way to say to him 'a' is bad, to break that chain before following it with 'b' gets you 'c', whether it's yelping like what hanksimondoes or saying 'leave it' like hargyle.

Those are the kind of options i mean in terms of discipline and trying to get them to understand when behaviors are bad, not hitting or yelling, so whatever you have to say about over-disciplined dogs have little to do with me. Besides I have watched almost all those vids, I already incorporate training into his daily life, I already use various method to try and improve important training like his recall: doing a 'jackpot' of treats, playing recall games where I toss treats leading from one recall to another, letting him back off to play so that he doesn't associate it to leaving or getting crated every time. But no matter what I do, and how much I train him in recall, I still think he will ignore me in favor of a fleeing cat simply because we both know that I can't trump that in terms of how interesting I can be.

I never said that he's doing things to piss me off or give me the finger, I said he's doing things because he only understands what's good, not what's bad. You might think it's fine to ignore a whole side of learning: the negative reinforcement and positive punishments (note this does not mean 'good' punishment), but I think it depends who or what the pup associates punishment with. Eg. if they bite a bee and get stung, it is associated with the bee, not you, so there's no breakdown of your relationship.

Here's another one using the wire as an example, if you attach a can with coins inside by string to a unplugged wire, when he bites or tugs, it dislodges the can sending it clattering to the floor, the pup would associate it with the environment rather than to you punishing him. Or using bitter apple spray to make things yuck to bite. Dogs learn things from their environment all the time if they are allowed to make their own mistakes, so why can't/shouldn't we set them up to learn from small mistakes to prevent them from making big ones (eg. actually getting electrocuted)?

I am looking at options in training to better communicate what I want and don't want to him. Especially with things that are essential, such as recall. He should be trained to come the first time he's called, not wait till he decides after the 10th time I call to come wondering in. No, he doesn't get punished, I've never punished him before, but I just think there must be more ways to constructively communicate what I want than simply treating what's good and ignoring the bad.

 

[ohbehave]

Here are my thoughts on the puppies and punishment:

First, I am not opposed to punishment in general. It is something that requires a good deal of thought and great timing. (Reinforcement also requires timing as well... and badly timed reinforcement can lead to nasty problems, too)

Second: For a puppy, I am going to try using super-care in preventing problems and redirecting the pup to things that I allow. If I can not supervise the dog, it is crated or contained somehow. I give it serious exercise (not hard to do on a small puppy) and then don't feel guilty about the crating.

The reason I am avoiding punishment is that I want MILD forms of punishment to be effective once the pup is an adult. If I expose the pup to too many "NO!" or other versions of punishments, it could become desensitized to those things. Then, when I need an emergency and resounding punishment effect, my "NO!" will fail to have it's effect. That being the case, I would have to resort to more dramatic or intense forms of punishment to produce the same suppressive effect.

I have never owned a pup before, but I am familiar with behavioral principles. I have always rescued large dogs in the past. Even though I sometimes regret getting this puppy, I am resolving to obsessively supervise and create as much of an error-free, early development phase for the puppy.

I find it interesting, though, that my adult dogs required no help. They seemed rather easy-to-be-with shortly after taking them home. No training... none of this fatiguing puppy chasing. So how did they become so wonderful with, clearly, no formal home training?

I'm honestly a bit worn with this puppy. Keeping after the pup's waking moments AND supervising my girlfriend's interactions with the puppy (to prevent her from reinforcing obnoxious behavior) is sort of tense for me.

But I want my pup to remain somewhat easily affected by mild punishment stimuli... so I'm trying to avoid the use of it for now.

For instance, when she barks or whines, I do nothing. This, I hope, is simple extinction. If she is seeking attention or some sort of reaction from others, she gets none of it if she barks. Eventually, if the barking continues incessantly, I may come up with some simple, sharp, bark-ending reaction.

 

[cshellenberger]

You want to teach her to leave things alone, hten you need to teach her a command, it's called "Leave It" and it's taught via Doggy Zen, one of the easiest things to teach, you just have to be patient. In the mean time, PUPPY PROOF, anything you don't want that curious little pup to chew, should be put OUT OF HER REACH. Use baby gates to keep her out of certain rooms, keep cords hidden from her and pick up anything you don't want her to destroy. Get on her level and look for things that are hazards JUST as you would with an infant or toddler.

 

[Roloni]

I dont punish for bad behavior , or give treats for good behavior..
Teach Recall , Once you have this rock solid...everything else falls into place.

 

[xxxxdogdragoness]

I use ignore for one dog but I use loss of privileges for another, for ajosefins. Of listening (like deliberately ignoring a command) means a time out & a bit of being ignored & not paid any mind at all, bc I learned that she as acting out to gain attention when we got buddy (when she was the only dog for a while she was like the perfect dog) so I have been using time outs for her. For buddy he loves attention so to renforce him being around me from day one he got here I had him on leash (other place is obliviously not safe for a dog to even be off leash in my presssence) but here on this ranch its a lot more remote & very safe (premeiter fence is coyote proof due to the fact that this is a breeding farm & small babies are vulnerable to attacks) so when he chooses to stay around me I make sure to give him lotsa attention & reinforcement for his choice in behavior, when he goes away from me (remember they can NOT leave the property into someone else's property) I ignore him, now he is never gone longer then 10 minutes & never very far (he isn't a very active guy, he plays wih the other dogs for a bit but only usually leaves for bsthroom breaks).

 

[Amaryllis]

Well, Ice222, if you already know the answer to the question, why did you ask it?

You've got numerous forum members, all with well behaved, well trained dogs, telling you they don't discipline. And no, cause and effect and association are not the same thing. At all.

For example, you punish your dog for chewing on wires. Maybe he learns that the act of chewing on wires is bad. Maybe he learns that wires are around when owner goes nuts and makes him scared. Maybe he learns that chewing makes owner nuts. In the first scenario, you've taught him not to chew on wires. Nice, but you still haven't taught him want you do want him to chew. In the second scenario, you've taught him to fear wires, which could end very badly. In the third scenario, you've taught him not to chew, which is very, very, very bad.

Dogs DO NOT THINK LIKE HUMANS. Full stop. Not kinda, not sort, not but, they don't think like us. That's what makes punishment such a bad idea when dealing with a dog. You can't quite be sure what lesson you are teaching. It might very well be that you are scary and cannot be trusted.

 

[doxiemommy]

It is true, a dog's cognitive process is NOT like that of humans. ANYTIME you hear anyone on the forum suggest or discuss any form of punishment, you will find that they also talk about timing being KEY. For instance, if you are going to punish, you have to do it when the dog is in the middle of the "bad" activity. Many people don't have the timing down, and that is one of the reasons punishment can be ineffective in some situations.

This also shows that dogs need that IMMEDIATE correction (if you choose to use corrections/punishment), or they won't make the correct associations, as far as WHY they were punished. The reason for this is, once they are done with the "bad" activity, they are done. It's over, they've moved on. They are not going to remember, and be able to reason out that they may have been punished for something they did 2 minutes ago.

This is why, for corrections AND for proper rewarding, it is always recommended to get the timing right. Even for praise/reward, it's recommended that you praise (or at least mark) the good behavior within seconds, for the dog to understand EXACTLY what behavior was right. If you wait, that connection is lost. It's the same with punishment. If you wait, that connection is lost.

I always say, it's better to PREVENT before the bad behavior even starts, so that you take away that "bad" option. We've seen it with our own dogs, anytime they went NEAR the wicker chair to chew on it, we interrupted them. We called them to us, we played a game, we gave a toy, etc. We NEVER let them get near it. Ever. When we were gone, we blocked their access to it. Now, months later, they have no interest in it at all, because we took that option away, didn't even give them the chance to put that on their list of things to do.

 

[Amaryllis]

I'll give you the speech on the over disciplined dog.

When I first got Kabota, he wouldn't chew on anything. He wouldn't even try. He did have teeth removed, but I took him to my vet 2 weeks after that, and he said the other vet had done a great job and the extractions had healed up nicely, try stuffed toys for the softness. I bought stuffed toys, Kabota had no interest. He did pick out a stuffed sheep at the groomer, but he would just carry it around, and if he saw me looking at him with it in his mouth, he would drop the sheep and go pure appeasement: curled up, tail touching his belly, head down.

During this time, on day my husband was baking croutons. He opened the oven to get them out and realized he didn't have pot holders. So he turns to grab the potholders and turns back to see Kabota just about to stick his head in a 450F oven, so my husband instinctively shouts "NO!" and Kabota squeals, pees and runs away. He was still hiding under the coffee table when I got home from work an hour later.

My guess from all of this is that Kabota was taught, through punishment, that all chewing is bad. He probably went to town on some shoes and learned the lesson of a lifetime: chewing makes people attack me, chewing is scary, I won't chew. Dogs need to chew. Dogs need to play. These are basic dog needs. Kabota was punished into not being a dog. This may have been well-intentioned punishment, but the effect was terrible.

It took me two months to convince Kabota that I love it when dogs chew. Then I had to ignore him chewing on socks, slippers, blankets, pillows and anything else the same texture of a stuffy for two weeks. Why? I feared that the slightest hint of correction would push him back to not chewing and not playing. I'd rather replace every soft textured thing I own. After that, it didn't much training and redirection to get him to understand that toys are for chewing- I love it when you chew toys!- and other things are not for chewing.

You don't have to hit every operant conditioning box to train a dog. Just because positive punishment is an option doesn't mean you have to use it. It would be very easy for you to create another Kabota with punishment, but impossible to do so with management, training and redirection.

 

[Greater Swiss]

Wow, everybody here has FAR more intelligent and informed things to say than what I've got (and its all great stuff!), but I thought I'd give my two cents anyway...

With Caeda, for nipping for instance before she got a time out we would use "no" as a warning and "bad" as the cue for "You're getting a time out". At the time, she didn't care one bit about our displeasure, just about the time out. I think two things happened though. First off, the sounds became a "positive interruptor", despite being negative words sometimes she stopped because she heard something and would look at us....lots of praise. Then the sounds of the words No or Bad became conditioned punishers....the punishment of course being a time out (no hitting or anything, but it is still a punishment).

Move on about 8-9 months and she actually did start caring about our displeasure, or so it seems, she certainly has become more of a velcro dog (well...not velcro, more orbiting us). The word "no" quite often, depending on her level of drive, seems to get her to stop what she is doing. Usually "leave it" or "carry on" does the job, but No actually helps our scattered human brains that might not bring up the right term for her at that second. Sometimes if I say Caeda and sound disappointed (say she is doing something not dangerous, but a real pain in the butt) she usually stops too...I think she has picked up on our tones of voices and body language to a point, and even looking back I don't think we ever did anything consistently enough for her to attach that tone to her doing something wrong. Perhaps she does care a bit now! Either way, she does behave more depending on our attitude towards what she is doing. If we are excited she does tend to misbehave a little more lol...

Prevention works the best, but there will be things a dog will get into, to me the next step is a redirection/incompatible behaviour. But in our case we seem to have stumbled into a conditioned punisher (I didn't have a clue of that at the time....reading later on I kind of figured it out). The punishment had no violence, and is actually a punishment that is often recommended for nipping....the walking away for a few minutes. Yeah, she probably "fears" the punishment when she hears it, but IMO, that isn't the worst thing in the world. She has also started connecting "Eh Eh" with "you are about to hit the end of the leash and not get to go further", most of the time she turns back and stays within the leash range, sort of a conditioned punisher.

I have heard of individuals who believe that negative words like "no" and such are bad to use because of the negativity, and that it can hurt your relationship with your dog.....when it comes down to it, IMO, its a sound to a dog, and we attach the meaning. The word "Apples" could be used too and have the same effect or you could use profanity to mean "sit". Words seem to be sounds to dogs, and they learn the meanings as they go along.

Anyway, that's my relatively uninformed two cents....just the fruits of a fair bit of pondering Caeda's behaviour when it comes to the issue of punishment. (If I'm way off base with my "theories" please let me know! I'm learning, this is just what I've figured from reading and observing Caeda)

 

[Ice222]

Well, Ice222, if you already know the answer to the question, why did you ask it?

You've got numerous forum members, all with well behaved, well trained dogs, telling you they don't discipline. And no, cause and effect and association are not the same thing. At all.

Dogs DO NOT THINK LIKE HUMANS. Full stop. Not kinda, not sort, not but, they don't think like us. That's what makes punishment such a bad idea when dealing with a dog. You can't quite be sure what lesson you are teaching. It might very well be that you are scary and cannot be trusted.

I don't already know the answer, but I'm not allowed to question yours if it doesn't make logical sense just because you're more experienced with dogs? In that case should I take Caeser Millains methods on blind faith too? Heck he has years of experience and has well behaved dogs. I've said many times that I don't mean hitting or yelling, yet you still assume that is what I meant.

I may not be an expert, but from my understanding of behavioral psychology, I agree with the premise that all animals learn in a similar way, including humans. The main difference is that humans have speech so we can communicate an attach meaning, and we are able to project and plan further than the immediate moment so it's possible to delay human punishment/rewards, but not so for dogs. If you're going to dispute the fact that dogs learn completely different to humans, you'll need a better argument, or a better explanation of what you mean by association because right now it doesn't make sense. If you give human a reward for doing something, they are more likely to repeat that action, if dogs don't learn like humans AT ALL, by that logic, dogs don't repeat a behavior more often if you reward them so why give treats and rewards at all? I'm not saying this is what I believe, but it's an example of why it DOESN'T MAKE SENSE.

I have considered your opinion, I have told you why I question you opinion. I am open to hear new and different stand points and take in what I believe to be useful, logical, and constructive, and I agree that the people of this forum have a lot to offer, but if you seem to fixated on what you THINK I'm talking about when I talk about discipline/punishment which is not the same as what I've been telling you I mean. On the other hand almost everyone else here had something to contribute towards what I was actually asking and have given me great insight into what THEY do in terms of managing bad behavior. Whether it's only using timeout, or the word "no" eventually had a meaning since it got associated with a timeout, yelping or whatever else.

 

[cshellenberger]

The word 'no' is overused and has no meaning for a dog because of that. Train a command, as linked 'leave it' is very effective. If I want a dog to get off the couch I train them to the command 'off' if I want my dog to wait while I open the door, I teach a sit, then train them to 'wait' (different than 'stay' I which they remain in place while I walk away). By training specific commands you make what you want clear to the dog. That's not saying I don't use interrupters, I do, but they are just that, a way to interrupt the unwanted behavior, direct attention to me so I can redirect the dog to something appropriate.

 

[doxiemommy]

I may not be an expert, but from my understanding of behavioral psychology, I agree with the premise that all animals learn in a similar way, including humans.

This is not true. Dogs do not have the same cognitive process as humans. I mean, really, cats don't even think the way dogs think, much less humans, or apes, or lions, or tigers, or bears (oh my!).

The main difference is that humans have speech so we can communicate an attach meaning, and we are able to project and plan further than the immediate moment so it's possible to delay human punishment/rewards, but not so for dogs.

This is ONE of the differences, not the only one, and not the main one. But, it is a big factor in determining how we communciate with dogs and how we need to express ourselves best so that they can understand what we expect. For instance, we "see" and interpret the world mainly with our eyes first, while dogs "see" and interpret with their noses first. This means the sense of smell is so much more important in how they view things, and in what choices they make.

If you're going to dispute the fact that dogs learn completely different to humans, you'll need a better argument, or a better explanation of what you mean by association because right now it doesn't make sense. If you give human a reward for doing something, they are more likely to repeat that action, if dogs don't learn like humans AT ALL, by that logic, dogs don't repeat a behavior more often if you reward them so why give treats and rewards at all? I'm not saying this is what I believe, but it's an example of why it DOESN'T MAKE SENSE.

You can reward or punish a human at any point, and they can understand the praise or criticism. It doesn't have to be immediate. But, for dogs, it does need to be immediate. Once a dog has stopped a certain behavior, you can't make any connections to that behavior, it's over, the opportunity is lost.

I have considered your opinion, I have told you why I question you opinion. I am open to hear new and different stand points and take in what I believe to be useful, logical, and constructive, and I agree that the people of this forum have a lot to offer, but if you seem to fixated on what you THINK I'm talking about when I talk about discipline/punishment which is not the same as what I've been telling you I mean. On the other hand almost everyone else here had something to contribute towards what I was actually asking and have given me great insight into what THEY do in terms of managing bad behavior. Whether it's only using timeout, or the word "no" eventually had a meaning since it got associated with a timeout, yelping or whatever else.

My responses in bold.