Should we panic now or later?

Absolutely Not. The issue of zero or low rate loans will take some considerable time to resolve. The ATO has commenced this process by issuing the two IDs.  

What is most significant about the IDs is that the ATO has not, as yet, indicated its assessing policy in respect of zero interest loans. It may very well be that the ATO will take a reasonable attitude of allowing a grace period in which LRBA arrangements, to which the two IDs apply, be restructured and also provide some safe harbour guidelines as to minimum acceptable interest rates, loan terms and LVRs.