10

Given a domain, is it possible for an attacker to discover one or many of the pages/resources that exist under that domain? And what could an attacker do/use to discover resources in a domain?

I have never seen the issue addressed in any security material (because it's a solved problem?) so I'm interested in ideas, theories, best-guesses, in addition to practices; anything an attacker could use in a "black box" manor to discover resources.


Some of the things that I've come up with are:

  • Google -- if google can find it, an attacker can.
  • A brute force dictionary attack -- Iterate common words and word combinations (Login, Error, Index, Default, etc.) As well, the dictionary could be narrowed if the resource extension was known (xml, asp, html, php.) which is fairly discoverable.
  • Monitor traffic via a Sniffer -- Watch for a listing of pages that users go to. This assumes some type of network access, in which case URL discovery is likely small peanuts given the fact the attacker has network access.

Edit: Obviously directory listings permissions are turned off.

3
  • All this assumes that you have directory listing permissions turned off of course. Apr 1, 2009 at 22:31
  • Yes it does - but worth adding to the question! Apr 1, 2009 at 22:32
  • The bounty on this question kind of scares me :)
    – annakata
    Apr 6, 2009 at 15:44

12 Answers 12

14
+250

The list on this is pretty long; there are a lot of techniques that can be used to do this; note that some of these are highly illegal:

  • See what Google, archive.org, and other web crawlers have indexed for the site.
  • Crawl through public documents on the site (including PDF, JavaScript, and Word documents) looking for private links.
  • Scan the site from different IP addresses to see if any location-based filtering is being done.
  • Compromise a computer on the site owner's network and scan from there.
  • Attack an exploit in the site's web server software and look at the data directly.
  • Go dumpster diving for auth credentials and log into the website using a password on a post-it (this happens way more often than you might think).
  • Look at common files (like robots.txt) to see if they 'protect' sensitive information.
  • Try common URLs (/secret, /corp, etc.) to see if they give a 302 (unauthorized) or 404 (page not found).
  • Get a low-level job at the company in question and attack from the inside; or, use that as an opportunity to steal credentials from legitimate users via keyboard sniffers, etc.
  • Steal a salesperson's or executive's laptop -- many don't use filesystem encryption.
  • Set up a coffee/hot dog stand offering a free WiFi hotspot near the company, proxy the traffic, and use that to get credentials.
  • Look at the company's public wiki for passwords.

And so on... you're much better off attacking the human side of the security problem than trying to come in over the network, unless you find some obvious exploits right off the bat. Office workers are much less likely to report a vulnerability, and are often incredibly sloppy in their security habits -- passwords get put into wikis and written down on post-it notes stuck to the monitor, road warriors don't encrypt their laptop hard drives, and so on.

5
  • I LOL'd at the dumpster diving. I thought that was just good for getting free donuts! :P Apr 6, 2009 at 15:30
  • Hehe, funny ideas! I like the one with the hot-dog-stand! "Hey, we have IT security problems ... mhhh it can't be related to the new hot-dog-hootie in front of our door, they make so delicious hot dogs"
    – Martin K.
    Apr 6, 2009 at 15:32
  • It's good suggestion too... my resume has hot dog stand experience. Apr 6, 2009 at 15:33
  • dumpster diving for list for directories / files is a lot effort, isn't it :)
    – dr. evil
    Apr 6, 2009 at 15:35
  • @LFSR Consulting: I've got the onion-mincing and mustard-sqooging skills. Let's do it! :)
    – Don Werve
    Apr 6, 2009 at 21:32
7

Most typical attack vector would be trying to find well known application, like for example /webstats/ or /phpMyAdmin/, look for some typical files that unexperienced user might left in production env (eg. phpinfo.php). And most dangerous: text editor backup files. Many text editors leave copy of original file with '~' appended or perpended. So imagine you have whatever.php~ or whatever.apsx~. As these are not executed, attacker might get access to source code.

0
4
  • Brute Forcing (Use something like OWASP Dirbuster , ships with a great dictionary - also it will parse responses therefore can map the application quite quickly and then find resources even in quite deeply structured apps)
  • Yahoo, Google and other search engines as you stated
  • Robots.txt
  • sitemap.xml (quite common nowadays, and got lots of stuff in it)
  • Web Stats applications (if any installed in the server and public accessible such as /webstats/ )

Brute forcing for files and directories generally referred as "Forced Browsing", might help you google searches.

1
  • I like the OWASP link. I wasn't aware that the project existed. Apr 6, 2009 at 15:08
2

The path to resource files like CSS, JavaScript, images, video, audio, etc can also reveal directories if they are used in public pages. CSS and JavaScript could contain telling URLs in their code as well.

If you use a CMS, some CMS's put a meta tag into the head of each page that indicates the page was generated by the CMS. If your CMS is insecure, it could be an attack vector.

2

It is usually a good idea to set your defenses up in a way that assumes an attacker can list all the files served unless protected by HTTP AUTH (aspx auth isn't strong enough for this purpose).

EDIT: more generally, you are supposed to assume the attacker can identify all publicly accessible persistent resources. If the resource doesn't have an auth check, assume an attacker can read it.

2
  • I agree. Don't assume that your site is secure based on the fact that the attacker shouldn't know about a specific URL.
    – Kibbee
    Apr 6, 2009 at 15:09
  • 1
    I don't think that he's worried about that.
    – dr. evil
    Apr 6, 2009 at 15:26
1

The "robots.txt" file can give you (if it exists, of course) some information about what files\directories are there (Exmaple).

0
1
  1. Can you get the whole machine? Use common / well known scanner & exploids.
  2. Try social engineering. You'll wonder about how efficient it is.
  3. Bruteforce sessions (JSessionid etc.) maybe with a fuzzer.
  4. Try common used path signatures (/admin/ /adm/ .... in the domain)
  5. Have a look for data inserts for further processing with XSS / SQL Injection / vulnerability testing
  6. Exploid weak known applications within the domain
  7. Use fishing hacks (XSS/XRF/HTML-META >> IFrame) to forward the user to your fake page (and the domain name stays).
  8. Blackbox reengineering - What programming language is used? Are there bugs in the VM/Interpreter version? Try service fingerprinting. How whould you write a page like the page you want wo attack. What are the security issues the developer of the page may have missed?

a) Try to think like a dumb developer ;)

b) Hope that the developer of the domain is dumb.

1

Are you talking about ethical hacking?

You can download the site with SurfOffline tools, and have a pretty idea of the folders, architecture, etc.

Best Regards!

1
  • The answers will fall under the ethical hacking category, but it's mostly curiosity. Apr 6, 2009 at 17:57
1

When attaching a new box onto "teh interwebs", I always run (ze)nmap. (I know the site looks sinister - that's a sign of quality in this context I guess...)

It's pretty much push-button and gives you a detailed explanation of how vulnerable the target (read:"your server") is.

1
  • +1 Nice tool - security stuff like this can really boil down to the tools in the tool box. Apr 10, 2009 at 17:37
0

If you use mod_rewrite on your server you could something like that:

All request that does not fit the patterns can be redirected to special page. There the IP or whatever will be tracked. You you have a certain number of "attacks" you can ban this user / ip. The most efficient way you be automatically add a special rewrite condition on you mod_rewrite.

0
0

A really good first step is to try a domain transfer against their DNS servers. Many are misconfigured, and will give you the complete list of hosts.

The fierce domain scanner does just that: http://ha.ckers.org/fierce/

It also guesses common host names from a dictionary, as well as, upon finding a live host, checking numerically close IP addresses.

0

To protect a site against attacks, call the upper management for a security meeting and tell them to never use the work password anywhere else. Most suits will carelessly use the same password everywhere: Work, home, pr0n sites, gambling, public forums, wikipedia. They are simply unaware of the fact that not all sites care not to look at the users passwords (especially when the sites offer "free" stuff).

2
  • I think you missed the point of the question... Apr 9, 2009 at 16:39
  • I don't like telling crackers how to do their job; instead I answer questions like this how to protect. Apr 10, 2009 at 8:08

Not the answer you're looking for? Browse other questions tagged or ask your own question.