No (not every word like this is feminine) and No (the word Allah has no feminine).

The letter "ت" or "ة" is an indication for female names, but not exclusively

Not all words that end with the letter "ت" or "ة" refer to a feminine word in Arabic, not even if this word is a "noun" for example: If we call a person genius we say Nabighah to him (like the Jahily poet al-Nabigha al-Dubyani النابغة الذبياني :

النابغة

If a scholar shows deep knowledge in many topics in Arabic we don't call him عالم 'Alim but 'Allamah:

العلامة

and both certainly are masculine words.

So yes if a word ends with the letters "ت" or "ة" this might be an indication that this word -if it was a noun- is referring to a female, but there are many counter examples. Therefore this indication must be taken carefully.

Al-Lat اللات

Actually seems to refer to a female deity or idol of the Kufar which was brought to the Ka'aba in Mecca by the tribe Thaqif. But the sunnah tells us:

Narrated Ibn 'Abbas:
(regarding His Statement about the Lat and the 'Uzza: Lat was originally a man who used to mix Sawiq for the pilgrim. (Sahih al-Bukhari)

And ibn Kathir quoted in his tafsir:

(Have you then considered Al-Lat,)
Al-Lat was a white stone with inscriptions on. There was a house built around Al-Lat in At-Ta'if with curtains, servants and a sacred courtyard around it. The people of At-Ta'if, the tribe of Thaqif and their allies, worshipped Al-Lat. They would boast to Arabs, except the Quraysh, that they had Al-Lat.
Ibn Jarir said, "They derived Al-Lat's name from Allah's Name, and made it feminine. Allah is far removed from what they ascribe to Him. It was reported that Al-Lat is pronounced Al-Lat because, according to 'Abdullah bin 'Abbas, Mujahid, and Ar-Rabi' bin Anas, Al-Lat was a man who used to mix Sawiq (a kind of barley mash) with water for the pilgrims during the time of Jahiliyyah. When he died, they remained next to his grave and worshipped him."
Al-Bukhari recorded that Ibn 'Abbas said about Allah's statement, (Al-Lat, and Al-`Uzza.)
"Al-Lat was a man who used to mix Sawiq for the pilgrims."
Ibn Jarir said, "They also derived the name for their idol Al-'Uzza from Allah's Name Al-'Aziz. Al-'Uzza was a tree on which the idolators placed a monument and curtains, in the area of Nakhlah, between Makkah and At-Ta'if. (Source: qtafsir)

As for the origin of the name Arabic Wikipedia (See and compare with the content of Wikipedia in English) offers a couple of possible explanations:
_{In the following I'm translating from Arabic as these translations are of my own take them with the necessary care!}

• قيل: اللاتُ: يجوز أن يكون من لاتَه يَلِيتُه إذا صرفه عن الشيء كأنهم يريدون أنه يصرف عنهم الشرّ.قيل: هي مشتقة من لويت الشيء إذا أقمت عليه.
  It was said that al-Lat comes from the expression Laatahu yaleetuhu لاتَه يَلِيتُه if one hinders (someone) from (doing) something: As if they wanted to hinder "the bad" to reach them. It was further said that it was derived from lawytu a-Shaya'a لويت الشيء which means I remained doing (something).

• قيل: أصلها لَوْهة فعلة من لاهَ السراب يلوهُ إذا لمع وبَرق وقُلبت الواو ألفًا لسكونها وانفتاح ماقبلها وحذفوا الهاء لكثرة الاستعمال واستثقال الجمع بين هاءين.
  It was said: the origin was Lawha لَوْهة derived from Laat as-Sarab لاهَ السراب yaluhu يلوهُ if something flashes or shines but the letter waw "و" was transformed into an alif "ا" as it was silent while the letter before it had a fatha diacritic and they simply left out the letter ha' "ه" as it was often used and due to the fact that it sounds heavy to put two succesive ha''s together.

• قيل:اللات هي مؤنث اسم (الله)، وقد كانت تعرف في عهد هيرودوت باسم: أليتا، ويذكر هيرودوت بأنها كانت الزهرة السماوية.
  It was said that al-Lat is the feminine of the name Allah, and it was known at the time of Herodotus by the name Alitah, and he also said that she was the (planet) Venus.

However the same article also adds:

• و اللات هو وثن كانت تعبده ثقيف فورد أنها صخرة كان يجلس عليها رجل كان يبيع السمن واللبن للحجاج في الزمن الأول، وقيل إن اللات كان يَلتّ له السويقَ للحج على صخرة معروفة تسمى صخرة اللاتّ وكان اللاتّ رجلا من ثقيف، فلما مات قال لهم عمرو بن لحيّ: لم يمت ولكن دخل في الصخرة.
  Al-Lat was an idol which was worshiped by Thaqif, it was said that it was a rock on which a man who sold butter fat and milk to the pilgrims in the first age. It was said that he used to mix Sawiq for the pilgrims on a known rock which was called al-Lat and this man was from the tribe of banu Thaqif , when he died 'Amr ibn Luhay عمرو بن لحي pretended that he didn't die but entered in the rock.

• ويعتقد البعض أنها آلهة أنثى (كما يتضح من اسمها) وأنها كانت ربة السماء، التي عبدتها الشعوب السورية نتيجة ترحل العرب الأنباط[؟] الذي أخذوا معهم أربابهم إلى المواطن التي حلوا فيها.
  Some believe that it was a female deity -as it seems from the name- and that she was the Godess of the sky which was worshipped by the Syrian nations due to the migration of the nabatean Arabs who carried their deities with them where ever they settled down.

• They quoted also the hadith from Sahih al-Bukhari and explained the word Sawiq as a paste. So the meaning of the hadith is that this was a good man who used to make paste for the pilgrims and feed them (honoring them). So when he died the people honored him and stood at his tomb and then declared him a God. It was said that the name of this man was Sirma ibn Ghanam "صرمة بن غنم".

• وروى ابن أبي حاتم عن ابن عباس: (كان يلت السويق على الحجر فلا يشرب منه أحد إلا سمن فعبدوه)
  Ibn abi Hathim narrated from ibn 'Abbas (about this man that): He was mixing Sawiq on the rock all the people who drunk (consumed) it became fat so they worshiped him (later).

• وروى سعيد بن منصور والفاكهي عن مجاهد: (كان اللات رجلاً في الجاهلية وكان له غنم فكان يسلو من رسلها ويأخذ من زبيب الطائف والاقط فيجعل منه حيساً ويطعم من يمر من الناس فلما مات عبدوه وقالوا هو اللات. وكان يقرأ اللات مشددة).[
  Sa'id ibn Mansur and al-Fakihi nattated from Mujahid: He was a man of the Jahilyah and he had some sheep and used to contemplate in its slowness and he was picking from the raisins from at-Taaif and from the cheese and made (of them) Hais (a sweet dish) and he used to feed all the people that pass by and when he died they startedt worshiping him and said he is al-Lat and he (Mujahid) used to recited al-Lat with a shidah: اللّات