Does WhatsApp approve messages during the "support window"?

No. During a chat window session, which lasts 24 hours from the moment your customer sends their last message, you can send messages that are not message templates – and therefore, those messages do not need to be pre-approved by WhatsApp.