Similar calc is done on Naruto Forums, but this is slightly different.
Beaters can hit bludgers head-on to send them back at the same speed in the opposite direction.
The Firebolt is 150 miles per hour. All the other broomsticks are somewhat slightly less than that speed, even cheap broomsticks like Cleansweeps and Comets can somewhat keep up with Harry's Firebolt.
150 mph = 67.056 m/s
Bludgers are 10 inches in diametre. Iron has a density of 7860 kg/m^3.
7860*(4/3)*pi*(0.254/2)^3 = 67.4 kg
67.4*67.056^2*0.5 = 151 532 Joules, Wall level
Consistent with the durability of wizards. Not to mention Neville can harm Nagini, a snake which is at least Wall level, with a sword.