Spider A Url

I want to Spider a URL, to learn how a site is built. This is for educational purposes, and I would like to know how I can use the method myself, on a similar URL