Is it rude to talk to a deaf person?

I don't mean talking while signing. That I already know that is rude because you are not signing in ASL if you are also talking. "If you are talking while signing you are not signing in ASL" (Mitch Noll, ASL 4 professor).