I think it depends on how far off they're getting, and what they're doing in that tangent.
If they are working with beginners and not working the techniques you/the instructor is showing then they are doing that student a dis-service (and themselves, but they are responsible for their own training).
Even as a mid-level student (lots of time in, but I'm slow at physical things) it is a great frustration for me to go to a seminar or work in a different dojo and work with someone that isn't on the same scroll as the instructor.
Eventually I just figured out that I'd learn what I could from that guy, then try to catch up when we switched partners again.
We (IMO) do not learn from the guy at the front of the class (generally) we learn from each other. We learn from doing, and having done to us (it is often more valuable to be Uke than Tori).
It is absolutely critical to learning that we be the best training partner we can be, both for ourselves and for our fellow students.
That said, sometimes something catches your mind, or you see something in a junior that really needs to be corrected right there. If you (the instructor for that session) didn't catch what happened right before, then you might not understand why the divergence took place.
You should talk about your concerns (privately) with the Senior Instructor in your school and ask him (a) what he thinks of this, and (b) how you should address it in your classes.