The lights in the mirror do always pass a small amount of current - enough to light LEDs dimly (not enough to light the original bulbs, but the current is still there). The fuse is just a protection device, smaller is better, but it won't make any difference to battery drain.
The issue may well be to do with how the wind deflector is wired. Power should be from an "accessory" source, only on when the car is on. You obviously have it powered from a battery source (always on). As such it can fool the battery drain circuit in the car to not turn off the supplies, and so drain your battery.