No, remember, there are always molecules coming into the balloon from the outside as well as molecules going out from the inside. What will happen is that eventually the number of molecules on the inside will decrease to a point where, even though the individual molecules are finding their way out faster than individual molecules outside are finding their way in, there will be few enough molecules on the inside so that the total number of molecules going out will equal the total number of molecules coming in per second. At this point, there will be fewer molecules per unit volume inside than there will be outside. This will decrease the density of the balloon and it will lift off.