Maintaining the Password Processors

Introduction

ECN maintains a user base of about 18,000 students, staff and faculty. Each user has one or more passwords associated with their account. The password is either the original one from when the account was created, or a new one chosen by the user and set using the passwd command. Rarely do users have more than one password, as the passwd command will change the password for all accounts and hosts maintained by the account maintenance system, a university wide resource.

Within that group of users are a fair number that have a password that is easily guessable. Easy-to-guess passwords are passwords like password! There is the document Hints On Choosing a New Password that describes passwords and what constitutes a good, secure password from an easy to guess password. The weak part of setting easy-to-guess passwords comes from the password changing command passwd. Unless the passwd command is substituted by a newer, more strict version, it will pretty much allow most any password. Current implementations of UNIX include some amount of password regulations, such as a minimum length, but most all are useless from keeping out easily guessable passwords.

Currently, the account maintenance system at ECN includes a very strict password regulation based on the use of the software library CrackLib version 2.7. Users changing their passwords at any ECN maintained Solaris 8 workstation or server will need to provide a good password before it will allow the password change to propagate to the user's accounts. Unfortunately, the user is able to change their passwords, even for all ECN hosts, from another account maintenance system host, such as those as ITaP. See the ITaP password changing web page. These alternate hosts may not have strict password regulations, so easily guessable passwords can still make it to ECN workstations and servers.

Most all university computers are connected to the Internet without firewall protection. And even when firewalls are installed, services such as ssh, imap and Windows shares are exposed, giving most Internet hackers ways of connecting successfully to user accounts because the user accounts are protected with weak passwords. There are constant streams of logon attempts made on user accounts using general, dictionary-based passwords.

Password Processors

In order to identify users with easily guessable passwords, a set of password processing servers have been installed at ECN. These processors' jobs are to take existing encrypted user passwords and try to brute force the plain text password. The brute force attack is by no means thorough, but those passwords identified are most likely the target of hackers and are the ones that will need changing the quickest. The brute force process is accomplished by a standard software package called Crack version 5.0.

This document describes how the password processors are configured and how to maintain the system in case of trouble.

System Configuration

The password processors are a set of three machines, each a Dell PC computer running FreeBSD 4.7. They are minimal machines, because all they need to do is operate a console, a little bit of disk space and memory, and compute DES hashes. Besides the fastest processor available, the minimum hardware requirements are:

• 128MB of RAM
• 500MB of hard disk space
• 10Mb/s Ethernet

The minimal FreeBSD operating system installation is needed. The only service exposed to the network is ssh. No backups are needed as all data is stored on a Solaris 8 server. Any failure on the part of the password processor system can be recovered by re-deploying a new machine and starting the cracking process from the beginning.

The current password processors are:

• splinter.ecn.purdue.edu
• fracture.ecn.purdue.edu
• sever.ecn.purdue.edu

The Solaris 8 server is harbor.ecn.purdue.edu. Harbor contains a few sets of shell scripts and is the master database server. The database will be backed up through the normal OmniBack backup procedure. If the database fails, restoring to the last known working state will be okay - it will mean that the passwords lost since the last run will have to be reprocessed by the password processors.

Account and Database Storage

All processing is handled by the account crack on both the master server Harbor and on all password processor hosts.

All password storage processing is handled by the database server, which is MySQL version 4.0.12. MySQL is configured so only Harbor and the password processor hosts are allowed to connect and update data. The database is stored in the account crack, which is located in /var/local/crack.

Harbor will run a script, once a week (Thursday at 7:30am) to populate the database with the latest list of hosts and sites. The hosts and sites come from the existing rdist configuration. Only hosts configured as a HOMESERVER are in the final list.

Once the hosts are populated in the database, the script will continue by collecting passwords from the stored list of hosts. The account crack is allowed to connect to HOMESERVERs as root for the purpose of collecting the password from the /etc/shadow file. The back end script is called /usr/local/etc/ecn/crkshdw.

Once passwords are collected, the information is downloaded into the MySQL database, marking new password entries as needing a pass through one of the password processors.

Finally, another script will run, once a week (Thursday at 8:15am) to mail out a report. The report will list users, by site, that have a known guessable password. The report will be sent to sites. It will be up to the site to decide password change policies and procedures for informing the susceptible users.

Host Communication

All communications between the password processors and the master server Harbor are through the MySQL database.

Each password processor will ask the database for a set of data to work on. When the password processor is completed checking all brute force attacks, the results are sent back to the master database.

Password Processors

The password processors have a cron job that fires off once an hour. If there is an existing brute force processor already running, the cron jobs exits with nothing more to do. If no brute force processor is running, a new one is started. This once an hour cron job will keep the machine busy in cases where new passwords appear in the database, or to restart the processing should the machine experience a power failure or operating system crash.

Each processor gets work by marking passwords in the database with its host ID. When a brute force processor starts, it will check the database to see if it already has work assigned. If no work is assigned, the processor will ask for a new block of work. Currently, the processor assigns work in blocks of 64 entries. If no work exists, the processor goes idle, and waits for the next hourly run.

When work is assigned, the brute force processor will collect the password data from the database and begin processing. When the processing is complete, it analyses the results. Any passwords that were guessed are marked in the database in the guessed section. Any passwords that make it without being cracked are placed in the processed section. Guessed passwords are also placed in the processed section.

Maintenance

In case of problems, there are some helpful maintenance commands available in the crack account on Harbor. You will need to be logged on as the crack account for this to work. The best way to log on is to put your ssh public key into the .ssh/authorized_keys2 file.

Status

A script called ./status.sh will query the database and the processors for their status.

$ ./status.sh
 -- Database status: --
Uptime: 73110  Threads: 1  Questions: 30218  Slow queries: 0  Opens: 11
Flush tables: 1  Open tables: 5  Queries per second avg: 0.413
 -- splinter.ecn.purdue.edu: --
Processer ID is 69286
At stage 328 out of 2313
 -- fracture.ecn.purdue.edu: --
Processer ID is 68828
At stage 334 out of 2313
 -- sever.ecn.purdue.edu: --
Processer ID is 68666
At stage 332 out of 2313
 -- Table status -- 
86      fracture.ecn.purdue.edu
76      sever.ecn.purdue.edu
69      splinter.ecn.purdue.edu

23768   processed shadows
2909    guessed shadows
504     unprocessed shadows
$ 

This shows that the MySQL is running. The processors are at 328, 334 and 332 out of 2313 stages. And that there are 86, 76, and 69 groups of passwords being brute force processed. The reason that they are not 64 entries is that there are some accounts that have the same password but are at different sites. The processors have completed looking at 23,768 passwords, and of those, 2909 passwords were guessed. Finally, there are 504 passwords left when one of the password processors complete its set and wants a new set.

Starting/Stopping MySQL

A script called ./server exists to stop and start the MySQL database server. This script is symbolically linked into the Harbor initialization scripts directory as /etc/init.d/mysql and /etc/rc2.d/S89mysql.