Some findings, as stated by @umH in comments you may find the hadith in imam al-Bukhari's book al-Adab al-Mufrad -see here-:
Ibn 'Umar said, "Modesty and belief are together. If one of them is removed, the other is removed."
with the Arabic wording:
إِنَّ الْحَيَاءَ وَالإِيمَانَ قُرِنَا
were Iman and Haya' have been joined or put together so the word that refers to both in this version is the verb قرن which means they have been connected to each other. The hadith was compiled on the authority of ibn 'Omar and qualified as sahih by al-Alabni.
The other version which was compiled by al-Hakim from Nishapur in his al-Mustadrak and ibn abi Shaybah in his al-Mussanaf is all on the authority of ibn 'Omar ():
" إن الحياء والإيمان قرناء جميعا ، فإذا رفع أحدهما رفع الآخر " .
Where the wording actually differs in one single word: in this case Iman and Haya' have been joined using a noun: قرناء which means they are mates and means joined and put together and one cannot imagine one of them without the other. so far non of my findings adds the "description" by using the fingers.
Imam al-Hakim said it is sahih on the conditions of both al-Bukhari and Muslim -see here in Arabic- and see also hadith #12 in this page from Mussanaf ibn abi Shaybah. You may find this hadith also in al-Bayhaqi's Sho'ab al-Iman (as he was a student of al-Hakim)
Imam at-Tabarani narrated a slightly different version saying:
الحياء والإيمان مقرونان لا يفترقان إلا جميعا
This version appears in his al-Mo'jam as-Saghir المعجم الصغير -see here in Arabic- and al-Awsat المعجم الأوسط -see here in Arabic- on the authority of abu Musa al-Ash'ari the teacher 'Abdullah ibn Muhammad ibn 'Obaydah عبد الله بن محمد بن عبيدة (accepted, ibn Makula considered him as da'if) from whom at-Tabarani heard the hadith and his father (unknown) are the weakest parts of the narrator chain. Imam at-Tabarani pointed at some uniqueness in this hadith.
One could translate it as follows:
All translations in the following are of my own unless declared otherwise! Take them carefully
Shyness/Modesty and belief are connected don't leave each or break off unless both (together).
Note that even if the wording may change mostly the meaning is the same.
Some other versions and their translation:
الحياء والإيمان في قرن فإذا سلب أحدهما اتبعه الآخر
Shyness/modesty are connected if any of them was taken the other will follow.
this should appear in al-Mo'jam al-Awsat of at-Tabarani on the authority of ibn 'Abbas ()! I couldn't find it so far. Imam al-Bayhaqi has compiled a similar narration saying in his sho'ab al-Iman:
الْحَيَاءَ وَالْإِيمَانَ فِي قَرَنٍ، فَإِذَا سُلِبَ أَحَدُهُمَا تَبِعَهُ الْآخَرُ
الحياء والإيمان أوتوهما ومنعتموهما (Source)
Shyness and belief were given to them and taken from you.
This is a part of a longer hadith in al-MO'jam al-Kabir of imam at-Tabarani on the authority of 'Oyayna ibn Hisn ().
One Source of my research was this Arabic thread.