The whales would dive, but whales have to breathe air so come back up again. My knowledge of this is based on Moby Dick, so take it with the pinch of salt necessary. It describes the experienced sailors being able to estimate how long the whale would be under water for, and look-outs on the larger ship could see where the whale broke the surface and head there to attack it in the smaller boats. Then they'd spear the whale with a long rope attached to the spear and the whale would dive under again with the spear stuck in it. As it dived, the pressure of the water would cause it more pain in the wound and it would have to resurface. Repeat, with a grisly finale. But yeah, it does seem pretty insane (and cruel) that anyone would attempt this.