First we should define what exactly interest is:
There are two types of riba discussed by Islamic jurists: an increase in capital without any services provided and risktaking, which is prohibited by the Qur'an, and commodity exchanges in unequal quantities, also prohibited in the Qur'an.
And for practising riba, which was forbidden, and for consuming the people's money illicitly. We have prepared for the disbelievers among them painful retribution. (Quran 4:161)
These excerpts show that the reason for the prohibition of riba is because you are basically cheating people out of their money, you charge them excessively for no extra services provided.
paying interest to yourself
Some do not realize in a 401k plan that the money you have to pay back to you account or "interest" is taxed, and this may be the reason to regard it as haram.
Second, on the interest component, it’s more complicated. Here, the “interest” you’re paying on a retirement plan loan is new money you are putting into your account. You’re using after-tax dollars to make these interest payments/contributions, as opposed to the pre-tax dollars that you might normally put into the plan. When the money comes out in retirement, however, you owe income tax on this “interest” at the same rate as the other pre-tax amounts in the plan. So, in this sense, you “pay tax twice” on the interest.
To summarize it, you are taxed once when you pay the interest, unlike the usual pre-tax payment to the bank from your paycheck. Then you are taxed a second time when you pull the money out of your bank account.
This means that you are being taxed twice on money that should only be taxed once, hence calling it "interest" the government makes a gain off of money you are "giving yourself" they are basically cheating you out of your money, you shouldn’t have to pay taxes on the same money twice.
Jabir said that Muhammad cursed the accepter of usury and its payer, and one who records it, and the two witnesses, and he said: They are all equal
The bank and the government probably do this knowingly, to make people belive they pay money to themselves, when some of the money goes elsewhere. This would make it haram, they all play a part in it including the person paying the money, and this is what the prophet prohibited.
So in conclusion, not all the interest goes to yourself, you're being taxed on the interest so part of it goes elsewhere, most likely the government, and in rotation, eventually the government returns the money to the bank when it gives loans, turning "a payment to yourself" back to a regular old interest payment to the bank.
sources: wikipedia-Riba, 401k blog