Yes, I do the exact same thing with making contact and then looking away. Not only is it creepy but also, taking my felonious background into consideration, it makes me look like the pos criminal I am. Unfortunately, it has become instinctual now so it will take some effort to change.
But at least we are aware of this, right? That means there's some hope.