Web Spider

Discuss the many weaknesses of browser security and ways to mitigate the threat

Web Spider

Post by Calian on Thu Aug 12, 2010 5:20 pm
([msg=43638]see Web Spider[/msg])

Just wondering if any recommendations for a freeware/shareware webspider? have done a bit of searching, just wanted 2nd opinion.

Ty
Calian
New User
New User
 
Posts: 48
Joined: Thu Mar 04, 2010 1:53 pm
Blog: View Blog (0)


Re: Web Spider

Post by tgoe on Sat Aug 14, 2010 10:46 am
([msg=43714]see Re: Web Spider[/msg])

This looks good: http://metha-sys.org/. pavuk and wget might work too.
What's your ultimate goal though?
User avatar
tgoe
Contributor
Contributor
 
Posts: 658
Joined: Sun Sep 28, 2008 2:33 pm
Location: q3dm7
Blog: View Blog (0)


Re: Web Spider

Post by Calian on Sat Aug 14, 2010 2:58 pm
([msg=43734]see Re: Web Spider[/msg])

Ultimate goal is to learn to be honest, and from what I've been reading and seeing all over the place, gaining all the information I can before looking at an attack is where I should be starting.

Figured a web spider would be a good tool for pulling up info on any projects. If there a better way go ahead and tell me, just wanted something more customisable than google.

Thanks will take a look at what you've mentioned here.
Calian
New User
New User
 
Posts: 48
Joined: Thu Mar 04, 2010 1:53 pm
Blog: View Blog (0)


Re: Web Spider

Post by Goatboy on Sat Aug 14, 2010 3:13 pm
([msg=43736]see Re: Web Spider[/msg])

You could use something like Paros or Webscarab.

Basically it's a proxy that runs on localhost, intercepting all of your web traffic. It can automatically follow links, building a virtual map of the website. It also does some basic vulnerability testing, though I haven't had much luck with it. A lot of false positives, too. Worth a shot though. Definitely beats following links manually. I'd recommend Paros.
Assume that everything I say is or could be a lie.
1UHQ15HqBRZFykqx7mKHpYroxanLjJcUk
User avatar
Goatboy
Expert
Expert
 
Posts: 2823
Joined: Mon Jul 07, 2008 9:35 pm
Blog: View Blog (0)


Re: Web Spider

Post by Calian on Sat Aug 14, 2010 3:59 pm
([msg=43740]see Re: Web Spider[/msg])

Thanks for the new ideas, have a few machines set-up will drop them all down and have a tinker, see which I think will do what I need, now and in the long run.

Thanks again.
Calian
New User
New User
 
Posts: 48
Joined: Thu Mar 04, 2010 1:53 pm
Blog: View Blog (0)



Return to Web

Who is online

Users browsing this forum: No registered users and 0 guests