From this post, I got the idea to remove and replace the wheels altogether using a hacksaw.

Here’s what it looks like right now. My problem now is that I can’t remove the leftover metal shaft from the caster wheel.

I’ve tried using some lubricant and hammering it with a screwdriver but it didn’t budge.