In a recent interview, I learned some things about the job that hadn't been in the job description, and those things were deal-breakers for me. (I had not known about these things until the interview, so I wasn't just wasting everyone's time just getting some interview practice.) There had already been other things that made the position less than ideal (pay, location, etc), and the new information told me definitively that it wasn't for me.
If something is brought up during the interview and your honest reaction is, "This position isn't going to pay me enough to put up with that," is it a good idea to just tell them right there that you are no longer interested if that task is a regular part of the job? Or should I just wait until I send them a thank-you note to let them know?
Go through the rest of the interview and stay friendly and engaged, and then let them know afterwards that the position isn't quite what you're looking for.
Why should you bother, when you've already determined the job isn't for you? Because these people may have a job opening you like better in the future, and if you impress them now, that'll give you a leg up then. And also because if you impress them now, they might refer you to another company that's a better fit. And because the world is small and people talk, and you don't want to be "the guy who abruptly announced that he'd never take a job that involved ___ and got up and left."
Think of it as networking and/or interview practice, so that you don't feel like you're wasting your time.
And when you contact them later to let you know that the job isn't the right fit for you, you might consider letting them know why. For all you know, they'll come back with, "Oh, if you hate doing ___, that's no problem -- we can easily structure the position around just X, Y, and Z instead." And even if they don't, it's still useful for them to know your thought process, so they know whether the next opening that comes up is one that would interest you or not. If they don't know why you withdrew, they won't know if it was their entire company culture that you disliked or something specific to just that position.
But you've got to be pleasant when you do it; don't word it like you did here! For example:
Good: "I'm actually looking for something that doesn't involve significant customer contact; my heart is more in behind-the-scenes work, and it sounds like you really want someone to work with customers."
Bad: "I won't take a job that involves that much customer contact."
As a side note, last year we addressed this same issue from the other side of the equation -- whether the interviewer should cut an interview short if she realizes the candidate isn't right. It raised different, but related, issues and it's here if you want to check it out. (The comments are especially interesting.)