You can bypass robots and no-follow links by using wget -e robots=off, the user agent argument itself is not sufficient to bypass robots.txt.