I gave it another shot tonight and now I'm stuck at the same rooms you are. I figured out every other secret on the floor:
Copper Pickaxe - Left from Black & Green Slimes
You can push one of the walls out of the way.
Blue Pickaxe - North of Candle Room
Use up your copper pickaxe.
Red Ring - Whisps & Ghosts
The bubbles travel along preset paths - there are three walls they pass through. Break those with the pickaxe.
Bow - Left of 4 Trees
Need the Red Ring to pass through the bubbles. Break the wall behind the red statue and push it.
Potion - Shadow Room
There's a room filled with shadowy enemies; using the candle makes them and the room's walls vanish.
Defeat them all without using the Candle (the bow really helps).
Magic Key - Ghoma's Room
Kill Ghoma with the bow.
Heart Piece - First Room
Shoot the rightmost center statue with the bow.
That room with the 4 trees I'm
positive you gotta burn all 4 bushes. But the candle only has three charges. Probably need the wand + book I'm guessing? The empty room I'm pretty sure is the boss. I think it'll appear once you collect something else on the floor.
The grey panels, my first instinct was that you need to put something there, but there's absolutely nothing to put aside from Link, who can't occupy the space above it. I thought maybe I had to make the room symmetrical or maybe there was a secret button somewhere but no dice on any account. I should mention I didn't think to use the potion anywhere. I figured it was one use and was meant for the boss. It would be pretty silly to make you use it on something else. My best hunch is it must have something to do with the pickaxe, or you need to use the wand on them both.
The wand room being related to time completely baffled me. I tried walking clockwise around the room, I tried holding still and waiting for three minutes, and I also tried beelining for the room like you did. But then I thought, maybe it's related to the blue switch - pressing it probably makes a timed bridge over the lava.
Therefore, I think there's something pretty obvious we're meant to do with that blue button. It's a big ol' button, obviously you gotta press it, but I tried everything I could think of: I even used the boomerang to get one of the wizards onto it, in case it was pressure sensitive. I tried breaking some walls around it and even tried poking at some walls in that room, but unless the button is a giant red herring, it is *very clearly* the thing. I don't know if we need a hammer or something but that doesn't seem likely cuz the inventory is getting pretty full. Short of a secret room through a secret wall I've got no idea.