Lower your gaze does not mean in every occasion ,
In Islamic terminology it refers to :
Refraining from looking at people’s ‘awrahs, which includes the beauty of a non-mahram woman.
Shaykh al-Islam Ibn Taymiyah (may Allaah have mercy on him) said in Majmoo’ al-Fataawa (15/414):
Allaah, may He be glorified and exalted, has enjoined us in His Book to lower the gaze, which is of two types: refraining from looking at ‘awrahs and refraining from looking at the site of desire.
The former refers to a man refraining from looking at the ‘awrah of another person.
And yes, i believe it is too much to lower gaze in social situation. How can you talk to a woman without seeing their eyes? How can you do that at the office while at work?
Lowering gaze is required if you see her awrah accidentally.