From 27C3 public wiki

Jump to: navigation, search

Freerainbowtables

  • Developer? Coder? Check this
  • Wonder what those tables do? Check down here at "Would you like to know more?"

Contents

freerainbowtables.com

... or FRT for short is a (small) project which generates ... rainbowtables. Huh - captain obvious at work here - but let's see what's so special about it:

  • table generation by BOINC, CLI and GUI clients for Linux/Windows/OSX
  • table generation by CPUs or/and GPUs (CUDA-based)
  • multi-CPU, multi-GPU so you can show dem MD5s who is the boss around here
  • free (hah, didn't see that coming ... oh well, you did. Sorry mate)
  • BOINC allows you to freely decide how much power you contribute
    • ... and manage your bandwidth just as YOU like it
  • tables can be read on windows/linux alike
  • twitter and facebook page
    • website/forums for the thirtysomethings

Recruiting How to help

FRT it is a rainbowtable generation community, if you like. And it can always use help. Your help. In several areas:

  • Computing power - the BOINC-section is just for you
  • People who seed torrents after downloading
  • People who provide "conventional mirrors"

FRT.com is a small project in comparison with the data it generates. Additional mirrors would take some load from the "official one" and provide some failover capacity.

Looking for a project related to cryptography and with challenging math? FRT.com can always use additional developers as it develops new rainbowtable formats (.rti2). This means work on the distributed generator and the software to use the tables as well. Here is some info right away to get you started

BOINC: The infrastructure framework

The job to distribute the work, in the form of "work units" or WUs, is done by BOINC, the Berkely Open Infrastructure for Network Computing). BOINC was used to free up valuable development time, as it is already finished, including the client GUI a wealth of management options which would've been a pain to code. You use BOINC (GUI or CLI) to connect to the project, and control what your system is doing when and how much of it. It is also "fair" as it allows you to support other projects, when FRT has no jobs (WUs) available or something is amiss (failing server HDDs).

Hints for BOINC people

If you're already "BOINCing" you should know that there are some steps when computation of a specific table set is finished to create the final result. Those steps are not manageable by BOINC, just as the final analysis of a scientific project. Since distrrtgen generates a lot of data, it takes a while to create the tables for downloading and then some time to get them into sensible locations for distribution.

So if there are no WUs available, just run another project you like.

distrrtgen

The program/project run by BOINC. This is the actual program generating the tables; you attach your BOINC client to this project. The client logs show what is happening and if there are any problems/strange messages - check the forums, and post if there is no answer already.

Computing with CPUs

Old Pentium 4 CPUs will work sluggishly despite boasting 2.8 GHz or more. Those and even older CPUs (PIII, AMD K5) will be outclassed by the Intel Atom in your netbook. Tests showed that an atom in an MSI Wind netbook was just about 15% slower than a Dell D610 with a Pentium M750.

So, no country for old CPUs. Unless you have a bunch of old multi-cpu servers just run BOINC on a current medium- or low-end CPU. You will be happier.

CUDA: Generating tables with graphic cards

BOINC will check if you have a CUDA capable card after installation/start. You'll need the CUDA framework (comes with the driver) and of course a recent version of BOINC.

  • see here
  • no need to update your drivers when your system computes GPU work units
  • windows users: check the nvidia website
  • ubuntu users: 10.04 LTS? The non-free nvidia driver is enough
    • but you'll need boinc from their website
  • set the Options to your liking
    • <ignore_cuda_dev>N</ignore_cuda_dev>, <ignore_ati_dev>N</ignore_ati_dev>
      • this allows you to disable adapters, for example the one attached to your screen
    • <exclusive_gpu_app>important.exe</exclusive_gpu_app>
      • WOW-raid coming up? Use this to disable GPU-computing when you run a certain program

Things to consider when doing GPU-computing

First, distrrtgen doesn't use OpenCL so there is no support for ATI-cards. However, if there are any volunteers ... it might become coded in! Just kidding. OpenCL is still in a too early stage for many developers to consider working with it. Secondly, low end cards such as the nvidia ION or old 8600 will have less computing power than a current mid-range CPU. No need to run computation on those, just use your CPU. Thirdly, a current mid-range card or old high-end (9800 series) will complete a WU every 5 minutes or so, and start to upload 4.77 MBs. Change your bandwidth/network settings in BOINC if this isn't to your liking. No one forces you to run GPU-WUs 24/7 and upload all the time.

  • Any BOINC project is happier with you finishing a few WUs than you getting angry.

New to GPU-computing?

If you compute with your graphics card, it will run at a high load for a long time - longer than you can play or watch movies since you will fall asleep sooner or later. If you haven't done that on your system yet (ran CUDA-applications, not sleeping) remember that:

  • the fan of your card will run at maximum speed most of the time
    • plan your cooling, as your system will (hopefully) shut down if it overheats
  • the card will draw power at its maximum, so watch your electricity bill or set up shop in russia :)
  • Systems with several cards do all of the above, just faster and hotter.
    • but hey, free hairdryer!

Just let your system complete some WUs to see if it is a good idea or not to use your card. As with other projects, you don't need to run GPU work units all the time if there's any trouble. That includes the fan noise bothering you.

Air cooling (adding fans to your case until there is space for no more or simply opening it is still the cheapest way to cool your GPUs. Avoid stacking cards which only push air "downwards" - those will just heat up themselves and the card next to them.

rcracki_mt

Once you have a set of tables, you can use rcracki_mt to read them.

For linux, you'll just need the openssl-development libraries. Then it should compile on most distributions.

Usage

Formatting your hash-lists

You need to format your hashes in advance:

  • LM/NTLM:
    • pwdump-format: [1]
      • account:number:LM-Hash-PartALM-Hash-partB:NTLM-Hash:::
    • CAIN .lst-files
    • or hash-list, one hash per line
    • LM-hashlists: remember to split up LM-pairs
  • All other formats:
    • one hash per line

LM-NTLM-pairs (or better, triplets) are special case. If you use LM-tables against a pwdump-style formatted file, it will use a recovered plaintext from an LM-entry (which will be all uppercase) to bruteforce the NTLM-plaintext (Upper- and lowercase). If you use the big "LM-FRT"-table, it will also do the necessary special-character bruteforcing for you, for example if the plaintext contains german umlauts and so on. The mapping is explained in detail in this thread.

Running rcracki_mt

If called with no options, rcracki_mt will tell you the syntax 

usage: rcracki_mt -h hash rainbow_table_pathname
       rcracki_mt -l hash_list_file rainbow_table_pathname
       rcracki_mt -f pwdump_file rainbow_table_pathname
       rcracki_mt -c lst_file rainbow_table_pathname

-h hash:                use raw hash as input
-l hash_list_file:      use hash list file as input, each hash in a line
-f pwdump_file:         use pwdump file as input, handles lanmanager hash only
-c lst_file:            use .lst (cain format) file as input
-r [-s session_name]:   resume from previous session, optional session name
rainbow_table_pathname: pathname(s) of the rainbow table(s)

Extra options:    -t [nr] use this amount of threads/cores, default is 1
                  -o [output_file] write (temporary) results to this file
                  -s [session_name] write session data with this name
                  -k keep precalculation on disk
                  -m [megabytes] limit memory usage
                  -v show debug information

example: rcracki_mt -h 5d41402abc4b2a76b9719d911017c592 -t 2 [path]/MD5
         rcracki_mt -l hash.txt [path_to_specific_table]/*
         rcracki_mt -f hash.txt -t 4 -o results.txt *.rti
  • Example:
    • your rainbowtables reside in "/tools/RT"
    • you have a pwdump-file "192.168.1.20.pwdump"
    • you have two cores you want to use
      • rcracki_mt -t 2 -o rt.result -k -session mysession -f 192.168.1.20.pwdump /tools/RT/

This will run rcracki_mt on two cores, looking for tables in /tools/RT/. If for some reason you interrupt the process, you can restart it with -r mysession instead of -session mysession.

In general, the last argument will always be the path to your tables. rcracki_mt will check sundirectories on its own devides - and go through all matching tables it findes.

Trouble?

If rcracki_mt complains about a file called "charset.txt" just copy it over from the directory where you called "make".

Getting the rainbowtables

Torrents

Web-Mirrors

Would you like to know more?

What are rainbowtables, anyway?

Data used to "uncover" plaintext the plaintext of a hashed password. So it is a data structure used in password cracking. They're used for the following reason:

Using hashes to avoid storing plaintext passwords

  1. Someone chooses "admin" as his password, his OS uses MD5 to store the password
    • this value will be stored 456b7016a916a4b178dd72b947c152b7
    • if he logs in again, and types "admin", again the MD5-hash will be computed
    • result is 456b7016a916a4b178dd72b947c152b7 again
    • OS compares the two values, sees they are identical and logs the user in

Awesome! We can log in without the OS having to store our plaintext password.

Password guessing

However, you see the problem. If someone guesses our password correctly, he can just log in - he'll type "admin" and the OS will compute 456b7016a916a4b178dd72b947c152b7 and of course log you in. Our nice little system is vulnerable to password guessing! And that is what password cracking is all about.

  1. nefarious people get access to the stored MD5 hash 456b7016a916a4b178dd72b947c152b7
    • they can simply hash possible passwords until they create the same value
    • then they'll have guessed the passwords
  2. They can use a wordlist, and hash every entry with MD5.
  3. They can search a whole keyspace (with a bruteforcer)
    • They guess the password is 6 characters long
    • ... characters are from the latin alphabet
    • ... in lower and uppercase
    • ... and numbers
    • so they have to generate the keyspace 1-6, A-Z,a-z,0-9
    • Again, as soon as there is a match, the password was guessed correctly.

As you see, there is no "real" cracking involved. The MD5 hash is not "broken" to reveal the plaintext, but lots of candidate passwords are hashed until there is a match.

More efficient password cracking

Now think about someone who cracks password a lot, then you'll notice the problem:

  • every cracking (for example, 1-6, a-z,A-Z,0-9) run generates the same MD5 hashes
  • so they're doing exactly the same operation all over again
  • why not store the result in a hashlist and just compare the next time?

That idea is basically good. It seems the actual hashing is the only computational expensive step; so why not do it only once and be done with it? Several problems:

  • hashlist get really large, way too large for an interesting keyspace (too many terabytes)
  • this large file still has to be read from disk, and comparisons have to be done
  • result: more impractical than expected

rainbowtables

Rainbowtables are basically a solution to this problem; how to do the computational expensive stuff (hashing the candidates) only once and storing the result in a usable way. Creating the tables for the little example above, 1-6 characters long, consisting of a-z,A-Z,0-9 will work and result in the following

  • requires about 20-30 times the computation time of a single brute force run
  • tables will not include 100% of the keyspace, FRT.com does 99.9%
  • best result is "splitting" the tables into a lot of smaller files (single file smaller than 1 GB)
  • reading those tables will require a fraction of the power needed to bruteforce the same space.

However, this example is not really good, as any system (inlcuding your smartphone) has enough computation power to search that key space in a rather short time, so usually one would build tables for larger keyspaces which are inconvient to bruteforce for a single system.

Example

Target: MD5-hashes, password is supposed to consist of be a-z and a space and 1-9 characters long.

  • keyspace would be 7.919 billions to go through,
  • CPU might do 50 million per sec
  • this CPU would take about almost 2 days to go through the whole keyspace

A rainbowtable from FRT:

  • md5_loweralpha-space#1-9_10000
  • 52 GB, one file 512 MB and 124 MB index
  • Same CPU takes 10 minutes to search for three hashes

Can't complain, huh? The numbers of this example are of course a bit streamlined, besides the time to look through the tables (i5 650, 64bit linux). Of course, this example assumes that the plaintext is not found, but the whole keyspace and/or tables have to be searched through. Another point is, that regarding MD5 hashes 50 million per second is slow, a current GPU might do a billion per second, and therefore search through the space in 2 hours and 10 minutes. But as you see, the speed advantage of the tables is still there, and they're not really large.

Background info

Check the following links (same as above):

Retrieved from "http://events.ccc.de/congress/2010/wiki/Freerainbowtables"
Archived page - Impressum/Datenschutz