Perhaps you may look at the act in a more significant manner. Meaning... looking at anyone, regardless of the sex or sexual preference, is not haraam. A man looking at a man, a woman looking at a man, a boy looking at a man ... these acts of looking are not haraam in any way.
It is the MANNER in which one looks, that can become haraam. So, we can conclude, through the beautiful Guidance of Allah Most Mysterious, that intention is a key factor in this act of "looking". And it is haraam even if the transgresor is the only one who knows of the wrong intention of his / her looking.
As for holding of hands, it is clear that physical contact between any two people, is reserved for those who have the right, privilege or relation to do so. As also, is the Guidance of our Most Divine Allah, Most Wise. Again, this can be logically accepted as a most worthy Guide. As it is physical contact which leads to intimacy. And intimacy can evolve into seductive thoughts. Which can, in turn, transfer to sexual intentions and acts.
So, although the holding of hands is not "wrong" at all, it is an act of intimacy that needs to be respected in order to avoid unclean thoughts. And rather than place risk upon those less acquainted, rather than invoke intimate thoughts by onlookers who may continue into unclean intentions, rather than risk inviting the other person to have unclean thoughts (for which he will be answerable), it is the wisest (note I say wisest and not most understandable) choice to avoid holding hands....can you see where this goes? It is best to refrain from holding hands unless it is with a those that are permitted by Allah Most Insightful.
Perhaps this will help you draw your own clear, balanced and wise decision. I can only add and remind, that Allah All Knowing, declared that the greatest Gift he bestows upon man is the intelligence, Alghamdu Lilaah. It is up to us, to expand our thinking to what is the wisest, and not what makes the easiest sense.
Say to believing women that they should lower their gaze and remain
chaste and not to reveal their adornments save what is normally
apparent thereof, and they should fold their shawls over their bosoms.
They can only reveal their adornments to their husbands or their
fathers or the husbands fathers, or their sons or their husbands sons
or their brothers or their brothers sons or their sisters sons or
maidservants or those whom their right hand possess or their make
attendance or their male attendants who have no sexual desire or
children who still have no carnal knowledge of women.
Al Quraan - LIGHT 24:31 -
I answer the question with absolute respect and in bowing humility. As I acknowledge there is no complete answer. As in order to remain filled with desire for true enlightenment and driven for the Pleasure of Allah All Powerful, we must respond when someone seeks a positive influence. I ask apology for any offence my comment may draw from the most respected Umah.