Is not using all credit cards a good idea?

They can cancel the cards, but that is ok. Six cards is more than you need for a good credit score. However, you should use the oldest cards, so that the companies do not cancel those cards. If you want to bury some and take the risk that they might cancel those cards, then bury the newest cards.