It's one thing to be "used to hiking" and another thing to walk the Camino. I was "used to hiking" when I did the Camino this fall, but I was not used to walking 15 to 20 miles, every day, for a month. Unless one trains by walking similar distances, every day, the Camino may come as a bit of a shock to the body.
As far as fatigue goes, although it may be true that walking with poles uses more energy than walking without, this misses the point, which is that walking with poles spreads out the load and allows you to take some of the stress off the back, hips and knees, and do some of the work with your upper body, rather than just your legs. I am definitely in better shape after a long day of hiking with poles than after a long day of hiking without.
Not everyone needs poles, I suppose, but I can't imagine doing the Camino without them.
Karl