Loss of fuel pressure after period of low RPM

Motor: 1996 Force 120.

Problem: I am thinking fuel pump, but just want to make sure. After about 10-15 minutes of low rpm's the engine will stall. I go check the bulb and it has no pressure. Prime it back up and it starts right back up and runs fine. It will usually do this once either while comming in/out of Bayou Chico or when trolling. Could the diaphram in the fuel pump just be dirty and the low RPM's just are not keeping enough pressure? Anything else I should check? It runs like a champ otherwise. I just usually have to remind myself to check the primer bulb halfway coming in or going out when I idle through Bayou Chico.

Not too long ago, I took some folks out from work. Two of them had never been out in a boat before. Wierd huh. Anyway the boat stalled out and I played dumb and they were about to flip out when I told them they were going to have to paddle with their hands back to the ramp. That was funny to see their faces.