It's a lot like playing back your game in go or chess. Incredibly useful, incredibly powerful, but easy to read too much into or not see what is blazingly obvious to someone with a little more experience. In go our expression is that you are 4 stones stronger when reviewing, but services like the Go Teaching Ladder exist, in part, because sometimes 4 stones is not enough to actually see what is going on.
We used to do this with some frequency when I was rapier fighting. Except that when we played back the tapes, we would actually do it on a "pizza night" and get a large group of us together to comment. That way you got the benefit of more experienced input and of being able to watch those who are weaker than you, both of which in turn can help you analyze your own technique. That gives you the best of all worlds (so long as you have thick skin, they can identify a lot wrong in a group setting like that).
In many ways it is similar to looking at another student of about your level and seeing what they are doing differently from either what you are doing or what you think you should be doing. Last night I was doing this with the other black belt student in the class: Trying to see what she did that looked different from my mental image or from what I was doing, and asking for clarification about which way the technique was supposed to look. Similarly, by looking at yourself in a video you can see minor variations from how you may think it is supposed to look, and it gives you an opportunity to ask questions and gain clarifications. It is similarly beneficial to look at others in the same light, since it helps you with identifying your own gaps in technique.
I do think it is easy to walk away with a mistaken impression of what you are doing wrong, but I don't think that removes the fundamental value of the exercise. It just means that when you are doing it, it is worth remembering that your analysis is not perfect, and part of what you are doing is training yourself to analyze students for when you teach in the future. In that light, I've found it useful to pretend that the person I am analyzing is not me: it is someone else who I am trying to help or who I am trying to understand, giving a level of depersonalization to the exercise.
The other side of this is that the problem with looking exclusively at people much better or much worse than you is that what they are doing is going to be subtly different. Or in some cases what they are doing is not quite the same technique, or perhaps they come from a slightly different school. By looking at others your own level while analyzing yourself, you can see a little more of how the technique is taught at your school to people of your level.
So long story less long: absolutely use it as an exercise, but with a few caveats/pieces of advice:
- Don't do this in an echo chamber: Get the feedback of others.
- Practice reading others as well as yourself.
- Depersonalize it.
That's what has helped me the most with this sort of exercise.