I'll answer this question from the perspective of a dad and a former martial arts instructor.
Your kid is like a lot of other kids it sounds like. It's normal at this age (3 years old) for kids to not be able to concentrate and do what they're told for extended periods of time. A good teacher has to be able to get the kids interested in what he/she is doing. They need to make it fun. They can't spend too much time on any one thing, or else the kids will get bored. And they can't expect kids to pay attention perfectly. They have to go into it knowing that kids at this age are just trying to have fun, and if they can get one or two kids to be able to do what they're being shown every now and then, that's a success. Their expectations must be pretty low.
That being said, it sounds to me like your kid's karate instructor did the right thing. Actually, he was probably a little light if all he did was remove your kid's belt. The best thing he could have done would have been to take your child off the floor and have him sit it out on the side - with you, of course, not alone. My guess is he was being nice and reserving that for the future.
The other thing to note is that the instructor did this right away in the first class. If he avoided it until several classes had gone by, your child would have been confused, because this was inconsistent with all of the other times your child was not disciplined. And that would have made it harder for your child to learn. The fact that the instructor did this right away is a point in his favor.
Your child had a typical reaction, which was to be upset and cry. From this, he'll learn that if he disobeys the instructor again, the same thing will happen, or worse.
Kids at this age don't understand the boundaries and what they can and can not do. So they test. They goof off. You can tell them not to do stuff, but you can't reason with them at this age. That's not going to be very effective. Instead, it takes some kind of disciplinary action to get them to realize there are consequences to bad behavior. Actions are a lot more effective than words.
We do a disservice to children by not giving them proper boundaries. That goes for parents and for teachers. In most martial arts classes teaching children, you're going to see some really bad behavior going on with the kids. They're going to be goofing off, playing with other kids, talking while the instructor is talking, running around the class, etc.
What are the teachers doing? If you watch any of these kinds of classes, you'll commonly see the same thing. The teachers are ignoring the kids that aren't paying attention, and they only care about the kids that are giving them attention. They tell the misbehaving kids to not misbehave, but those kids just laugh and keep misbehaving. And the cycle continues.
This is generally how teachers cope with it. They know parents don't want their kids to be "disciplined". Parents want their kids to have a good time. They don't want it to be "too serious". But let me ask you, if your kid is one of the ones misbehaving, aren't you upset that the teacher isn't giving them any attention? Aren't you upset they're not improving? You should be.
And by "improving", I don't mean at karate. I think expecting a 3 year old to be improving at karate is misguided. Instead, improving in this context means being able to follow along in class with the rest of the students and to keep his attention on the teacher. That's going to pay off in all aspects of his life, especially academics later on.
So you need to be on-board with the teacher disciplining your child when it is necessary. Otherwise don't pretend it's anything but a babysitting class. Because, they're getting nothing out of it otherwise. They're not improving.
This idea extends into parenting as well. In fact, it starts and ends at home with the parents. Parents that do not tell their kids "No" a lot and frequently are doing a disservice to their kids. They need to set those boundaries, and the way to do that isn't by talking and hoping their kids will eventually get better. The way to do it is by taking their children out of the situation forcefully if they're not obeying right away. They need to make sure their kids know that if they don't obey without delay, there will be consequences.
Let me give you an example from my life as a dad. My 4 year old daughter and I drove to a nearby park to play. I parked the car and told her to put on her coat, because it was too cold outside to be without it. She refused. I told her that she had to, because it was too cold. She said no again. Rather than argue with her about it, as I did in the past without success, I said, "Okay then. No park today." And I quickly drove back home.
In the car, she said "okay", she would put her coat on. But it was too late. It wasn't about the coat, but about her refusing to do what she was told. So I kept driving home. And she cried some more. It hurt me, too, because I love her to death. But this is what a real parent has to do. Actions speak louder than words. Knowing there are consequences to bad behavior is really important for young children ages 2 and up. It establishes boundaries and guides them towards making better choices in life.
The other thing I do when my daughter misbehaves is I tell her what I saw her do from my perspective and how it made me feel, and how it made others feel. We have a conversation about it when some time has passed and she's not too upset anymore. Being able to see things from someone else's perspective is a huge part of their development. It helps them figure out what went wrong. Otherwise it can seem pretty arbitrary and trivial to them, which just makes them angry and resentful.
And keep loving your child. Changing your parenting style does not change how often you hug and kiss your kid and tell him how much you love him. Keep doing that. He's going to hate you and cry after you discipline him, but it will be short lived. And you should always tell him you love him, even as you're disciplining.
And by the way, I don't recommend spanking or hitting your child, either. That's not what I mean by disciplinary action. Spanking just breaks his trust of you and creates hatred and resentment. It drives him away from you. That's not what you want. You want open lines of communication. You want him to go to you and tell you things without fear. You want him to feel comfortable enough around you to express his feelings. Spanking will poison all of that.
Anyway, getting back to the incident with the karate instructor, once again I think what he did was the right thing to do. The fact that he did this immediately on day one helps, also. Without proper boundaries, your kid probably will not improve. And that goes for parenting and for regular academic classes as well.
Hope that helps.