The Jews and atheists are disbelievers, as they commit Kufr. The Jews disbelieve in Allah's revelations (Injeel, Quran) and prophets (Zechariah, John, Jesus and Muhammad ﷺ) as well as in many other creeds regarding Allah's person (such as claiming that He rested on the Sabbath). While the atheists disbelieve in Allah completely.
There are numerous verses in the Quran which say or mean that the disbelievers will not be forgiven by Allah:
فلن يغفر الله لهم ذلك بأنهم كفروا بالله ورسوله
Never will Allah forgive them. That is because they disbelieved in Allah and His Messenger.
— Quran 9:80
إن الذين كفروا وماتوا وهم كفار أولئك عليهم لعنة الله والملائكة والناس أجمعين خالدين فيها لا يخفف عنهم العذاب ولا هم ينظرون
Indeed, those who disbelieve and die while they are disbelievers - upon them will be the curse of Allah and of the angels and the people, all together. Abiding eternally therein. The punishment will not be lightened for them, nor will they be reprieved.
—Quran 2:161-162
And Jews and atheists come under the meaning of these verses, as they have disbelieved.
As for the verse:
إن الله لا يغفر أن يشرك به ويغفر ما دون ذلك لمن يشاء
Indeed, Allah does not forgive association with Him, but He forgives what is less than that for whom He wills.
— Quran 4:48 and Quran 4:116
It does not say 'polytheism' but rather it says أن يشرك به i.e. shirk. While shirk is often translated simply as polytheism, it is not limited to that.
Rather, as a term shirk is often used as a synonym for kufr in Islamic texts, as mentioned by various scholars:
الشرك والكفر قد يطلقان بمعنى واحد وهو الكفر بالله
Kufr and Shirk may carry the same meaning, which is disbelief in Allah
— Sharah Sahih Muslim - Nawawi
كل كفر شرك، وكل شرك كفر
Every Kufr is Shirk, and every Shirk is Kufr
— Al-Muhalla bi’l-Aathaar
كل كافر مشرك بالله لأن الكافر إذا كفر بنبي فقد زعم أن الآيات التي أتى بها ليست من عند الله، فيجعل ما لا يكون إلا لله لغير الله
Every Kafir is a Mushrik, for one who disbelieves in a prophet holds that the verses (signs) given to him are not from Allah, hence he ascribes an exclusive act of Allah to other than Allah
— Ma‘aani al-Qur’an
Also see Qamoos al-Muheet, Misbaah al-Muneer , Taaj al-‘Aroos etc. Some have also explained that kufr is shirk because it is essentially taking one's own whims as a god besides Allah (see 25:43).
There are examples within the Quran and Ahadith where shirk and mushrik are used for Kufr rather than self-admitted polytheism:
Consider the following verse:
ولا تأكلوا مما لم يذكر اسم الله عليه وإنه لفسق وإن الشياطين ليوحون إلى أوليائهم ليجادلوكم وإن أطعتموهم إنكم لمشركون
And do not eat of that upon which the name of Allah has not been mentioned, for indeed, it is grave disobedience. And indeed do the devils inspire their allies [among men] to dispute with you. And if you were to obey them, indeed, you would be associators [of others with Him].
— Quran 6:121
This verse was revealed when some of the Quraysh argued with the Muslims saying that it was lawful to eat dead meat (carrion). They argued that the Muslims eat what they kill themselves and what their own hunting dogs and hawks kill - so they should also eat what Allah Himself kills (i.e. animals which die naturally). This verse says that if we were to obey them (i.e. consider carrion to be lawful) then we would become Mushrikeen (i.e. would commit shirk). And it is obvious that considering carrion to be lawful is not polytheism but rather it is Kufr. And it is called Shirk even though the person will not admit to belief in multiple deities.
Also consider the hadith:
بين الرجل وبين الشرك والكفر ترك الصلاة
Between a man and Shirk and Kufr there stands his giving up the Salat
— Muslim
It is obvious that abandoning salah (i.e. not considering it obligatory) is Kufr, and it is called Shirk even though the person does not express belief in multiple deities.
And there are examples within the Quran and Ahadith where mushrik is used for Jews. For example consider the verse 2:135 - it is apparent that here Mushrikeen means all disbelievers and especially the intent is to say that Abraham was not a Jew or Christian. And it is also understood in this, this and this hadith that Jews are being referred to by calling them Mushrikeen. And scholars too will often in their writings refer to the Jews as Mushrikeen such as here.
Hence, 4:48 means that Allah will not forgive any Kufr: be it denial of the existence of Allah, or denial of any belief regarding Allah such as Tawheed, or denial of His prophets and revelations etc.