@Bonsie, Yep, pretty much exactly that situation
Except I decided to get up there in the middle of the rainstorm and throw a tarp over the problem area, because I didn't want my computer and stereo to short out. It was just sort of dribbling down the wall lazily - I didn't get that cinematic dripping into a pot thing this time. I threw some lumber onto the tarp thinking that that ought to hold it in place, however, wind was up to 50km/h. In the morning, still drizzling outside, I noticed a new dribble appear and sure enough the wind had blown the tarp out of place! So it got lashed down next time.
Waiting for a dry day to arrive, I managed to squirt some silicone under the screws and re-tighten them. Some screws were replaced but people are of varying opinion on whether or not you need to replace the screws if you're doing silicone. Let me tell you, the fumes from that stuff will do a number on you after a while - my typing speed increased dramatically