As we know there is a certain rule when Zakat becomes Farz. But I suppose in case of Hajj, it cannot be that strict because the cost to travel to the Holy Makka differs from place to place.
So what is the rule regarding Hajj becoming obligatory?
Conditions which when met, make Hajj fardh for you:
Click here to see the full fatwa.
Just in case that link stops working, please see the edits of this answer for the full fatwa.
I'll assume that you meet most of the standard criteria, so the real major thing to consider is being able to financially support it.
Hajj is obligatory on a person who has the financial means to travel to Makkah and is able to maintain himself there and his dependants (if any) at home. As for other debts, if a person is heavily indebted and his liabilities exceed his assets, Hajj is not compulsory upon that person. If he fulfils his monthly commitments and his creditors do not have any arrears claim against him, he will not be infringing on the rights of his creditors by performing Hajj on condition that he is sincere in fulfilling the remaining amount according to the commitments. If the creditors have arrears claims against him, he will be infringing on their rights by performing Hajj. Hence, he cannot perform Hajj unless he receives the prior consent of his creditors claiming arrears from him. (Source)
Most people will never be free from debt, so it's difficult to consider the ruling on whether they can afford it. Some things are required to live, such as a house, maybe a business/job, which can get in the way of performing hajj. Scholars have conflicting opinions on whether someone who saves money to get married does not need to perform Hajj first.
But generally, it's agreed that if you have manageable debts, if your creditors allow you to do so, and if don't need to work in that period of time to survive, it becomes compulsory.