Our CA has a recently unveiled service which will scan our public IPS and report on detected certificates close to expiry which is handy, but we have a lot of servers on non-routable and / or firewalled addresses, so an internal scan is the only way to cover them all.

There seem to be at least a couple of other published approaches in the google including ssl certificate expiration check and the neat ssl-cert-check script at prefetch.net which will take a list of servers and express the expiry date. I wanted something slightly different (more minimalist): as we’re using the excellent zabbix for general system monitoring which especially likes system commands or scripts which take a single parameter and spit out a single return value.

In this case that parameter is a server name or IP address,  returning  either the number of days until SSL cert expiry, or the certificate issuer, depending on which version of the script is called. As I said, we’re using this with zabbix but this is just a command line script usable for quick on the spot checks or could easily be incorporated into another monitoring / alerting system.

A nice simple method I would probably use if I didn’t have zabbix would be to incorporate this into a quick and dirty loop script which periodically queries a list of ips or subnets and fires off an email if the ‘days to expiry’  is below a certain value (exactly what zabbix does now). It could be made a bit more elegant perhaps by using a bit of nmap to build a fast list of responding servers to query, but depends what you want, how much time you have, and how much of a stickler you are for efficiency =)

(Command line for all scripts is simply: ./script-name server.name.or.ip.address )

Script: sslcheck-expiry

#!/bin/bash
# Simple SSL cert days-till-expiry check script
# by Glen Scott, www.glenscott.net

openssl_output=$(echo "
GET / HTTP/1.0
EOT" \
 | openssl s_client -connect $1:443 2>&1);

if [[ "$openssl_output" = *"-----BEGIN CERTIFICATE-----"* ]]; then

        cert_expiry_date=$(echo "$openssl_output" \
         | sed -n '/-----BEGIN CERTIFICATE-----/,/-----END CERTIFICATE-----/p' \
         | openssl x509 -enddate \
         | awk -F= ' /notAfter/ { printf("%s\n",$NF); } ');

        seconds_until_expiry=$(echo "$(date --date="$cert_expiry_date" +%s) - $(date +%s)" |bc);
        days_until_expiry=$(echo "$seconds_until_expiry/(60*60*24)" |bc);

        if [[ $days_until_expiry -ge 0 ]]; then

                echo "$days_until_expiry";
                exit 0

        else

                echo "EXPIRED ($days_until_expiry days)";

                exit 0
        fi

else
    echo "NOT_FOUND";
exit 1

Checking the issuer as well as the expiry

Because we have a bunch of servers setup for dev, test or other purposes with self signed “snake oil” certs and I dont really care about the certs on those, I wanted a method to determine the issuer. (Zabbix then has some logic which only bothers to email us if a cert is about to expire AND is from a real CA.)

Script: sslcheck-issuer-o

Returns the “O” (Organisation) value from the issuer string.

#!/bin/bash
# Simple SSL cert get-issuer-O
# by Glen Scott, www.glenscott.net

openssl_output=$(echo "
GET / HTTP/1.0
EOT" \
 | openssl s_client -connect $1:443 2>&1);

if [[ "$openssl_output" = *"-----BEGIN CERTIFICATE-----"* ]]; then

        cert_issuer=$(echo "$openssl_output" \
         | sed -n '/-----BEGIN CERTIFICATE-----/,/-----END CERTIFICATE-----/p' \
         | openssl x509 -noout -issuer -nameopt sname \
         | tr '/' '\n' | grep O= | cut -c3- );

                echo "$cert_issuer";
                exit 0
else
    echo "NOT_FOUND";
exit 1
fi

Script: sslcheck-issuer-cn

If for some reason you want the CN value from the issuer string, use this instead.

 
#!/bin/bash
# Simple SSL cert get-issuer-CN
# by Glen Scott, www.glenscott.net

openssl_output=$(echo "
GET / HTTP/1.0
EOT" \
 | openssl s_client -connect $1:443 2>&1);

if [[ "$openssl_output" = *"-----BEGIN CERTIFICATE-----"* ]]; then

        cert_issuer=$(echo "$openssl_output" \
         | sed -n '/-----BEGIN CERTIFICATE-----/,/-----END CERTIFICATE-----/p' \
         | openssl x509 -noout -issuer -nameopt sname \
         | tr '/' '\n' | grep CN= | cut -c4- );

                echo "$cert_issuer";
                exit 0

else
    echo "NOT_FOUND";
exit 1
fi

Additional notes:

All scripts use the openssl s_client function to connect to the first string (assuming IP or server name) on default port 443. It echoes some HTTP GET requests and an EOT, otherwise openssl will sit there until timeout waiting for something to happen. If the remote connection sends a certificate down the pipe (identified by the presence of “”—–BEGIN CERTIFICATE—–”) it will process it, otherwise it returns the value ‘NOT FOUND’, which is a catch all for no certificate, a malformed certificate, a network timeout and so on.

A quick glance at these will reveal a lot of duplication: indeed the last two only differ by one line. This is on purpose; I actually started out building a do-everything script with a switch to determine behavior (like this script I subsequently found). This would be a lot more efficient in terms of network requests, you could retrieve the cert once and analyse it in multiple ways. Unfortunately it turns out zabbix really only likes dealing with the one parameter – a minor issue for which I will forgive it – so I’ve broken this out into three smaller scripts.

OpenSSL will only return the date in a format like “Sat Jan 1 17:15:00 WAST 2010″. I was starting to figure out how to chop it up into a usable format like DDMMYYYY using sed, awk, cut and the rest but discovered  to my surprise that the unix date utility understands the string just fine as is. The date is converted to seconds since epoch and the ‘bc’ utility does some math on it, returning a rounded value in days.

Instead of awk in the last two I’ve used a handy-dandy utility called, simply, ‘text replace’ (tr). I actually don’t use awk, sed and regular expressions much and am correspondingly unfamiliar with the syntax, thus was happy to discover and use this nifty shortcut. It’s not as powerful as sed/awk but is great for a simple character replace like this, especially when the process of trying to manipulate both kinds of slashes in the issuer string and replace them with newlines '\n' is frying my brain.

As always, your mileage may vary, particularly regarding any differences in the basic syntax of the various utilities across platforms (I’m using RHEL). ’date’ for example is very forgiving here, this might not be the same everywhere. I’ve also had issues in the past with the syntax of SED on OSX.

Hope someone finds this helpful, and I will write up a brief post on using zabbix to manage SSL cert expiry at some point as well.

Visit glenscott.net for more content. Some rights reserved: Except where specified otherwise, the content of this feed is licensed under a Creative Commons Attribution-Noncommercial-No Derivative Works 3.0 License.

If you want the gory details, check the thread over at bugs.launchpad.net. I’ll distil the useful bits below.

After the upgrade, my huawei 3G modem stopped being detected by NetworkManager. I’d see the fake ‘ZeroCD’ drive try to map itself and occasionally a gnome message box would be thrown up about a failed mount attempt, but no modem.

A look in the logs revealed /var/log/messages filling up with lines like this:

kernel: option 3-1:1.2: GSM modem (1-port) converter detected
kernel: usb 3-1: GSM modem (1-port) converter now attached to ttyUSB0
kernel: option 3-1:1.1: GSM modem (1-port) converter detected
kernel: usb 3-1: GSM modem (1-port) converter now attached to ttyUSB1
kernel: option 3-1:1.0: GSM modem (1-port) converter detected
kernel: usb 3-1: GSM modem (1-port) converter now attached to ttyUSB2
kernel: option1 ttyUSB2: GSM modem (1-port) converter now disconnected from ttyUSB2
kernel: option 3-1:1.0: device disconnected
kernel: option1 ttyUSB1: GSM modem (1-port) converter now disconnected from ttyUSB1
kernel: option 3-1:1.1: device disconnected
kernel: option1 ttyUSB0: GSM modem (1-port) converter now disconnected from ttyUSB0
kernel: option 3-1:1.2: device disconnected

So the modem was being disconnected and reconnected at least a couple of times a second for some reason, and the storage device was not appearing at all.

I tried the usb_modeswitch tool which is supposed to jolt misbehaving HUAWEI (and other brand) devices out of their stupor with some undocumented SCSI/USB commands, but no success this time.

After a bit of googling, it turns out this is (was) a known bug in the way the more recent linux kernel handles the combination USB Modem/Storage device hardware (and was allowed to remain in a major release of Ubuntu which is a bit unfortunate as it seems these types of USB modems are pretty common).

There are a couple of fixes pending an official update: either install a patched version of the kernel, or temporarily disable the USB Storage kernel module which looks pretty easy and apparently worked for a few people:

# rmmod usb-storage

Untested by me: Your mileage may vary.  Be warned that even if this works, by unloading the usb-storage kernel module you will lose support for any USB based storage devices, so this is strictly a temporary workaround. I thought I’d try the more permanent and possibly dangerous (?) kernel solution first, which worked.

Steps to upgrade your kernel to a compatible version:

1. Check your current version with the uname -a command. My post-9.10-Karmic upgrade version was: 2.6.31-14-generic #48-Ubuntu SMP  x86_64 GNU/Linux
2. Go to http://kernel.ubuntu.com/~kernel-ppa/mainline/ and download the .deb files for the kernel headers (“all”) and the kernel for your architecture (“amd64″ or “i386″). If you don’t have any kind of internet on the affected ubuntu box, grab them via another connected machine and copy them via removable media (windows or mac should be fine for just getting the files). You want the “linux-header” and “linux-image” files from within the folder with the latest (hopefully stable) version number. You can ignore the source file for now.
3. Go to a command prompt, change to the folder where the downloaded .deb files are located, and execute the following, substituting the .deb file names for the versions you have (make sure you install the headers first).

4. sudo dpkg -i ./linux-headers-2.6.32-020632rc8_2.6.32-020632rc8_all.deb

5. sudo dpkg -i ./linux-image-2.6.32-020632rc8-generic_2.6.32-020632rc8_amd64.deb

After this, provided everything worked, you’re a reboot away from your modem working again. After the boot, uname -a should reveal the newly installed kernel version. Mine is: 2.6.32-020632rc8-generic #020632rc8 SMP

Once plugged in, the modem worked instantly and my mobile broadband account connected fine. Hooray!

While lsusb output looked the same as before:

Bus 006 Device 004: ID 12d1:1001 Huawei Technologies Co., Ltd. E620 USB Modem

My /var/log/messages also looked a lot healthier:

kernel: USB Serial support registered for GSM modem (1-port)
kernel: option 6-2:1.0: GSM modem (1-port) converter detected
kernel: usb 6-2: GSM modem (1-port) converter now attached to ttyUSB0
kernel: option 6-2:1.1: GSM modem (1-port) converter detected
kernel: usb 6-2: GSM modem (1-port) converter now attached to ttyUSB1
kernel: option 6-2:1.2: GSM modem (1-port) converter detected
kernel: usb 6-2: GSM modem (1-port) converter now attached to ttyUSB2
kernel: usbcore: registered new interface driver option
kernel: option: v0.7.2:USB Driver for GSM modems
kernel: scsi 8:0:0:0: CD-ROM            HUAWEI   Mass Storage     2.31 PQ: 0 ANSI: 2
kernel: scsi 8:0:0:1: Direct-Access     HUAWEI   SD Storage       2.31 PQ: 0 ANSI: 2
kernel: sr0: scsi-1 drive
kernel: Uniform CD-ROM driver Revision: 3.20
kernel: sr 8:0:0:0: Attached scsi generic sg1 type 5
kernel: sd 8:0:0:1: Attached scsi generic sg2 type 0
kernel: sd 8:0:0:1: [sdb] Attached SCSI removable disk

Additionally, with the new kernel, both the pseudo cdrom, the 3G modem, and presumably the SD storage (though I don’t use it) are working at the same time. So, problem solved.

(Another improvement I noticed with the new version of ubuntu/kernel is I can disconnect the wireless broadband account via networkmanager without nasty gnome freeze-ups. Not sure what the culprit was for this: I worked around by disconnecting the hardware to avoid freezes, but looks like this too is now solved).

Visit glenscott.net for more content. Some rights reserved: Except where specified otherwise, the content of this feed is licensed under a Creative Commons Attribution-Noncommercial-No Derivative Works 3.0 License.

Note that the following explanations, definitions of features and so on are the product of my own observation and experimentation with various WM5 and WM6 mobile devices. I have found some documentation on their functions but the majority of information I have discovered through trial and error. If there is some official documentation somewhere which contradicts what I say here (and I wouldn’t be at all surprised) then so be it: what I can say for sure is mine works.

That said, Windows Mobile networking is in my experience notoriously flaky and even though the stuff here works for my device, your mileage may vary considerably.

Ok, lets get into it.

Golden rule: Anytime you change anything at all in the networking profiles, after you have saved the changes, disable and re-enable the wireless network/adapter. I have a control utility for this on my device – (HTC Hermes) – but this will vary between devices. Following this step  every time I change anything has reduced my frustrations considerably – not doing this means settings often just don’t take effect, and after doing this sometimes things just start working.

A quick explanation of terms I’ve used:

• “Config Profiles” refer to the named settings you can create and assign to different networks in “Network Management” (Start -> Settings -> Connections -> Connections -> Advanced -> Select Networks) – Some of the existing config profiles are ‘My ISP” and “My Workplace” (and you will have others automatically created for your ISP if you have mobile internet access on your SIM card via a 3G or GPRS network).

Explanation of how WM decides which network to use (And hence which attached config profile is used to decide how to connect)

Windows mobile networking is whack (but you knew that already, right?). Here’s how it breaks down: It decides how to handle a http network request based on whether there are any decimals (periods) in the dns name.

By its logic, anything with a decimal/period is ‘internet’ and anything without a decimal/period is ‘work’.

So:

1. “http://bogus.internal” is handled with the config profile attached to the“Internet” network
2. “http://bogus” is handled with the config profile attached to the“Private Network” network

You can create multiple different named config profiles and assign any of them to either “Internet” or “Private Network”.

An important thing to note is, a config cannot have a VPN server added to it (or use an already setup VPN) when applied to the ‘Internet’ network. If you want to use a VPN you’ll have to do it through the ‘Work’ network (see exceptions hint below).

Explanation of the ‘Exceptions’ settings.

Now – anything in the ‘Exceptions’ list goes through the “My Work” profile regardless of whether the dns name has decimals in it to not. The good news is you can use wildcards here to force a wide range of sites through the ‘My Work’ profile if you want – hint: http:/*.* and https://*.* . I didn’t end up using this for my solution, but you might find it useful.

I’m sure this flavor of networking makes sense to some software engineer in Microsoft land, but to me it just spells confusion. Once I worked out what was actually going on, I figured out some shortcuts/config hacks which can be used to railroad the networking into doing more or less what you tell it to.

So here’s what I’ve done to make mine work:

First, I access everything using its FQDN – no dotless machinename shortcuts. This makes sure everything is using the profile assigned to “Internet” (regardless of whether I’m on a work network or not).

Make sure the ‘Exceptions’ section has no entries.

Next, tell windows mobile that every wireless network you connect to is “The Internet”. Forget about the “Work” option . As far as my usage goes, that option is useless. All the wireless networks I connect to are set to “Internet”. If you have already added a wireless network and don’t know if its tagged to “Work” or “Internet, you can go into settings -> wireless networks, find existing networks, and change which network it connects to.

Next, create a couple of new custom network configs, as follows:

• ‘Direct Connection’ – this does as it says, and contains no settings for proxy or vpn.
• ‘Proxy Connection’ – this has my work proxy server entered

You do this via Settings –> connections (tab) –> connections (icon) –> Advanced (tab), Select Networks (button). Here you can edit existing or create new config profiles.

Incidentally, my workplace uses VPNs to grant authenticated access to the wireless network – so not allowing a VPN connection to a host on a “private network” just breaks everything.

Once you’ve done that and entered your proxy authentication credentials in the appropriate places, you’re ready to go. Whenever you want to change how you’re connecting to the net go to network settings, and change “internet” to one of your created profiles. Remember to start/stop the wireless to force the change, and your next network access should be using either direct, proxy, (or VPN – see below), whichever you’ve chosen.

By doing this you lose any pretense of windows Mobile networking transparently working from whichever location / network you are connected to, but it never worked properly for me anyway, and at least this way you have some control back.

Connecting to a VPN

The above covered getting web access only, either direct or via a proxy. To get a VPN connection active (eg for skype and the like) heres what you have to do instead:

1. Assign a config profile to the ‘work’ network
2. Add a VPN connection to the config profile you used. You can add VPN connections to a config profile by assigning it to to the “Internet” connection, hitting OK, going back to the ‘Tasks’ tab and clicking the ‘Add a new VPN server connection’.
3. Add the appropriate wildcard exceptions (to the ‘exceptions’ section) to trigger the VPN connection for every hostname.

Once I get a VPN up at my work from inside the wireless I can make direct connections to outside hosts, for example using PocketPutty. Be warned though that even if it does connect, Windows Mobile likes to shut down the VPN connection once it decides it is no longer in use, eg after you haven’t looked at web pages for a while, regardless of whatever else you are doing on the network, (say in a live SSH session). Parking pocket IE on a web page with an auto-refresh might possibly fool it into keeping the VPN alive, but I haven’t experimented with that yet.

Hopefully there is some useful info in here and it eases the pain of getting your mobile device networking in a saner fashion.

This is a fairly quick covering of networking with WM5/6 and its highly likely there are holes, inaccuracies and/or bits left out:  If anyone has queries, corrections or extra to add, go ahead and comment or hit up the contact form for direct email.

Visit glenscott.net for more content. Some rights reserved: Except where specified otherwise, the content of this feed is licensed under a Creative Commons Attribution-Noncommercial-No Derivative Works 3.0 License.

There seems to be a few different ways to get the SMB bit done but I ended up using smbfs: you’ll need this on your system for this script to work. If you don’t have it and you’re using a package manager it should be pretty simple to get, a bit of apt-get install smbfs should do the trick.

Note: I am aware of various security issues with running scripts as root, storing passwords in scripts, and this sort of thing. Since this is a super simple backup script, I’m doing it anyway : Complaints department is /dev/null ;)

Script 1: this is a super simple version. It tars and copies some folders to the remote share and thats it.


#!/bin/bash

#simple backup script
#by Glen Scott, glenscott.net

# set smb server and auth vars
sharename="//ourserver/ourshare"
username="ourdomain\ourbackupuser"
password="passwordgoeshere"

backuplocation="/backups/*"
savepath="/root/"
filename=$(hostname).backup.$(date +%a).tar
mountpoint="/mnt/smb"

#tar up the backup folder
tar -cf $savepath$filename $backuplocation

#connect to the share
mount.smbfs $sharename $mountpoint -o username=$username,password=$password

# move the tar
mv -f $savepath$filename $mountpoint

# disconnect the share
umount $mountpoint

#all done!

Script 2: this is the second version I made for another box. It needed a mysql database backed up as well so I added a few lines in for that. I also took the chance to add a quick working folders checker / creator, tidy it up a bit and comment everything.


#!/bin/bash

# simple backup script
# by Glen Scott, glenscott.net

# this is a simple script to tar.gz certain folder locations and copy them to a SMB share
# this script should be run periodically from crontab
# you will need smbfs installed on your system or modify the samba mount method

# set smb server and auth vars
sharename="//ourserver/ourshare"
username="ourdomain\ourbackupuser"
password="passwordgoeshere"

#set mysql details
mysqlhost="localhost"
mysqlusername="root"
mysqlpasswd="mysqlpasswordhere"

#set which folder locations we want to backup, inc trailing slashes
#add more here and append to the appropriate tar line further down the script if needed

location1="/var/"
location2="/backup/"

#set temp files and folders
backuptemp="/backuptmp/"
savepath="/root/backup/"
filename=$(hostname).backup.$(date +%a).tar.gz
mountpoint="/mnt/smb"

# make sure our working folders are present and accounted for

if [ ! -d "${backuptemp}" ]
then
mkdir $backuptemp
fi

if [ ! -d "${savepath}" ]
then
mkdir $savepath
fi

if [ ! -d "${mountpoint}" ]
then
mkdir $mountpoint
fi

# tar up the files we want into the backup temp
tar -cf ${backuptemp}files.tar $location1 $location2

#dump the local mysql db into the backup temp
mysqldump "-h${mysqlhost}" "-u${mysqluser}" "-p${mysqlpasswd}" --all-databases --lock-tables > ${backuptemp}mysqldump.sql

#tar up the backup temp folder
tar -czf $savepath$filename $backuptemp

#connect the smb share to our mount point
mount.smbfs $sharename $mountpoint -o username=$username,password=$password

# copy the tar (could also move it but whatever you like)
cp -f $savepath$filename $mountpoint

# disconnect the share
umount $mountpoint

#all done

As long as you have smbfs installed, the above should work fine.

A word on smbfs: without it the above script will fail. You can probably install smbfs quite easily on your system with the command apt-get install smbfs (or yum if you’re using redhat/fedora, or whatever your flavor of package manager happens to be). I use debian, so apt-get works just fine for me.

A word on Crontab: You’ll need to add the script to your local cron to get regular backups.

I won’t go into hideous details about how crontab works, theres plenty of that on the net already. To keep it simple, if your distro supports it (most should) you can put a symlink to the script in /etc/cron.daily or /etc/cron.weekly which will give you a simple schedule.

If you want something a bit more complicated, you’ll have to mess with the crontab. I’m aware there are commands to get this done but I’ve always just edited the system crontab directly. Mine runs twice a week, on wednesdays and fridays, so my crontab line looks like this:

# m h dom mon dow user command
0 2 * * 3,5 root /root/backupscript

Righto, thats it.

UPDATE: I notice a mutated version of this script has been posted in this forum thread over at linuxquestions.org – cool! Check it out over there if you want to see what someone else has done with it.

Visit glenscott.net for more content. Some rights reserved: Except where specified otherwise, the content of this feed is licensed under a Creative Commons Attribution-Noncommercial-No Derivative Works 3.0 License.

Preamble: Our new staff laptops are pretty fantastic. Faculty has an initiative subsidizing the cost of deploying tablet notebooks to all schools in Computing and Health Science. I already had one of the few X41s which were wasting away in storage when I arrived: nobody seemed to want to use them becuase they apparently underperformed. I dusted one off and moved from the T42 I was using to the X41, and found the reduced footprint/weight to more than make up for any loss of grunt.

So the new machines have landed, and they’re the very capable Lenovo X61. I’ve had one for a month or so running alongside the X41. I would have made the transition without delay, as the new hardware is better in many respects, and I’m not that biased against Vista that I would avoid upgrading for that reason alone (Vista is mandatory on these new machines to increase staff exposure to the O/S), but theres been a major sticking point, and thats support for my favorite XP encryption solution, Truecrypt with TCGINA.

I’ve had at least one laptop stolen in recent years, and while no important data was lost / exposed via the theft, I now have a heightened awareness of the danger of carrying unencrypted data around. TCGINA does a great job in XP of hooking into the login process and pre-mounting your encrypted storage even before the user profiles are loaded, which means that data can be stored there as well. This effectively means that your desktop folder, my documents, IM data, browser (IE or FF) favorites and history and the like are all stored safely. If the device is lost or stolen, without your password, that sensitive file you left on the desktop isn’t sitting there exposed on an unencrypted diak waiting to be harvested by a forensic undelete utility.

Truecrypt 4 with TCGINA is broken on Vista

Vista takes a different (non GINA) approach to handling the login process , so TCGINA no longer gets it done. This was a real headache: I needed to be using Vista for work but wasn’t comfortable with an unencrypted portable system. (This is before TrueCrypt5 was released – more on this shortly). The X61 has a TPM and thus (any big brother TPM/Vista backdoor conspiracy theories aside) bitlocker should have been an option, but because of the partitioning setup on my notebook, it isn’t. This is because bitlocker works by creating a separate primary partition (size ~1gb or so) on your drive to stick its bootloader and encryption software.

Problem was, I already had the maximum 4x primary partitions. I’m never satisfied, so after installing Vista I went in with a linux livecd and gparted (gnome partition GUI, like travelling first class compared to being jammed in the luggage bay using fdisk) and sliced it up so I could have three additional OS’s booting natively. These were Ubuntu, Backtrack 3 Beta, and WinXP SP2 – (the latter being a real adventure to get co-habitating peacefully with the others, due to an unfortunate tendency to blat whatever bootloader I was using with its own apon install).

Eventually it all worked, with GRUB on the boot partition loading up whichever of my 4 OS/s I wanted. On a side note, whenever I messed with the size of the Vista partition (gparted handles ntfs partition resizing fine) Vista would fail to load until I went in with the vista boot dvd and ran the very simple repair/rescue procedure. Did this each time, and then it was fine.

(I’m aware that alternative bootloaders such as XOSL can supposedly work some magic when it comes to maximum / types of partitions and the O/S’s loaded from them, but I have yet to try it. )

So having reached this stage things were mostly groovy, my 4 OS’s on one machine, but with no encryption whatsoever. At this point, I would have been happy to have it on the Primary O/S only (Vista) as the others were mainly for testing and would be unlikely to have any sensitive data on there, but even that seemed out of reach due to the TCGINA/Vista broken-ness.

So what to do?
Answer: Truecrypt 5 rocks and is not broken on Vista

Midway through pondering how I was going to find a solution, a rescue came along: TrueCrypt 5 was released with a major major feature added: the ability to encrypt entire system & boot partitions.

This was pretty much holy grail stuff to me at that point.

I wasted no time in firing up the new Truecrypt on Vista to see if the promises were true. Summary: some of them are. Its good, but not perfect. It didn’t work “out of the box” (but then bitlocker didn’t work at all): There were hiccups because I had GRUB as my primary bootloader (TC5 refuses to deal with anything but the windows bootloader) and I was unable to encrypt my entire disk, as I initially thought I’d be able to do, because my partition setup included logical partitions (this scenario throws an error after you try to process a whole disk which contains logical partitions).

So to get it working? First, I had to nuke GRUB from the primary partition, and set it up on the secondary so I could still access my linux installs. This was pretty simple: I booted into Ubuntu (which is where my grub config lives) and installed it on the second partition via some simple GRUB commands which I googled and now cant remember (partition 2 happens to be where XP is installed), then booted up with my vista dvd and let it replace the primary bootloader.

It is probably worth noting that I also had the Backtrack distro* installed on one of the logical partitions (along with a swap partition and an independently encrypted truecrypt partition), and GRUB could load Backtrack fine throughout this process, as its location didn’t change. (sda6).

After that it was as simple as running the truecrypt system/boot encryption wizard from Vista again, allowing it to create a recovery CD (backup of the volume headers in case of corruption or a changed, forgotten password) and waiting for a couple of hours while it processed my vista system partition, live.

Voila – it works. My Vista partition is now secure, and my other OS’s boot fine, albeit unencrypted. The next step is to reorganize everything so I can get rid of the logical partitions and hence do a proper whole-disk encryption to cover both my Windows and Linux installs. I’m sure theres a post in that.

* Backtrack is a Slackware based livecd distro loaded with a plethora of security tools. Since my primary laptop is usually of the subnotebook/ultraportable breed and hence doesn’t usually include a CD/DVD drive, I’ve previously installed backtrack to a USB key and booted off that, but its easier to have it integrated as a boot option, epecially with nice large hard drives making the 3GB or so loss of usable space hardly noticable.

UPDATE: I have since received a few queries about this article via email and clarified it a bit, so I’ll post the emails and responses below.

John: Hello,
i read your article Quad boot with Linux, XP and Encrypted Vista on the Lenovo
x61 Tablet, but there is not much details. My problem is that i have 2 primary
partitions, 1. is winxp, second is linux. I have grub loader. So my problem is
after i will encrypt my windows primary partition + use pre boot lock stuff from TC, how my grub loader will work? Because if i understand well TC boot lock will delete MBR a put own code overthere.
Thanks for your reply.

Hi John

I have not encrypted WinXP with Truecrypt yet, only Vista, however I suppose it is more or less the same.

To answer your question: When you encrypt your windows partition it will install the TC bootloader on the MBR, and yes it will overwrite GRUB.

What you need to do is install GRUB on your linux partition before running truecrypt in windows. You will need to boot into linux and run some “grub” commands (suggest you google them) to install it to another partition/drive. (It is ok to have the grub bootloader installed on two drives at once). Once you have encrypted your windows system partition, the Truecrypt bootloader will detect any other bootable drives on the system and give you the option of booting from them instead of your encrypted windows when you start up. (They will not be encrypted or otherwise protected by truecrypt, but they will be bootable)

John: Ok till that part its clean, u mean just install grub not into MBR but on the linux partition where the linux is. Dont understand what u mean by grub will be installed on two drives at once, u mean MBR + linux partition?

Yes, you install the grub bootloader onto your linux partition. After that grub will be *temporarily* installed two places at once, but only until you run fixboot+fixmbr, after that the Windows bootloader will be restored to the primary drive/partition.

If I recall correctly, truecrypt will not do full system encryption while you have GRUB on the primary MBR, so once you have installed GRUB on your linux parition/drive, you need to replace it on the primary with the default WinXP bootloader (easiest way is to go in with the WinXP boot cd, go to the recovery console and use the “fixboot” and “fixmbr” commands). Once you have done this, boot back into windows (should go straight on with no sign of grub) and TC should encrypt your windows system partition fine.

John: Here is a place where i completly got lost. What do u mean by primary MBR? Ok anyway why do i have to put grub to primary? Didnt u say that its enought to install grub on linux partition, and simply overwrite MBR by truecrypt? Why do i have to do fixmbr and stuf…

fixmbr and fixboot are the microsoft command line tools for restoring the default windows bootloader. You need to do this because truecrypt will not encrypt a windows partition which has grub installed as its primary bootloader. Truecrypt then replaces the windows bootloader with its own bootloader which will then launch windows (encrypted) and also any other bootable drives/partions (ie your linux one with GRUB installed) that it finds.

So a basic sequence of things you would do:

1. Boot into your linux install and install the grub bootloader onto the linux drive/partition
2. Boot into windows recovery console (winxp cd) and restore the default bootloader (fixboot/fixmbr)
3. Take cd out and boot up normally – grub should be gone and you will get into windows.
4. Run truecrypt and encrypt windows partition
5. Next time you boot up, TC bootloader is there and you can boot straight into windows or grub/linux.

Hope this answers your question!

John: Thanks a lot, –=John=–

Hope this helps anyone else as well  =) – Glen

Visit glenscott.net for more content. Some rights reserved: Except where specified otherwise, the content of this feed is licensed under a Creative Commons Attribution-Noncommercial-No Derivative Works 3.0 License.

It should be noted that most of this gripe is based on experiences with Pocket PC 2003 and its predecessors. WM5 and WM6 are recent additions to the fold for me, and a number of the mentioned issues seem to be, if not solved, at least partially smoothed over.

So far, so far the strongest argument I’ve yet encountered for blowing Windows Mobile away in favor of some flavor of embedded linux is the WM implementation of networking. A real shame because aside from that, WM more or less seems to get it right – decent information management, desktop / remote email sync (when you can get past the networking hurdles), and with third party tools, enough access to the internals to keep a techie happy. Except the networking interface.

Windows mobile networking has generally confused me. As a network admin, I’ve dealt with plenty of odd setups, but Windows Mobile truly does take the cake. After a few hours of mind games you’ll likely be begging for a simple ‘do what your told’ setup as opposed to the ‘second guess you because we know better’ philosophy that WM6 seems to adopt.

I have messed around with these devices for longer than I should admit. Many a time I’ve had everything working – for a while. Then it stops, develops amnesia, stumbles about disoriented. Losing wireless has the device utterly, and inexplicably confused, and too often for happenstance a hard reset will get things going again – with the exact same config.

Indeed, there seems to be a new definition of logic when it comes to how networking should function, and often a setting will seem to have no effect, or the result will be inconsistent. It will work for a while then stop. One application will work fine, but another will not. Changing a seemingly unrelated networking parameter has ramifications: things start working in an unexpected fashion or not at all.

The approach seems to be akin to a puzzle game with a random element as opposed to a tool designed to achieve an outcome. Sometimes it will work, sometimes will not. The same inputs to the black box will not always render the same output.

Now I’ve had a bit of a dubiously qualified rave, making vague accusations and pointing my finger about the place at indistinct phantoms, here are some actual specifics I have encountered.

Most, if not all of the headaches come from the implementation of multiple networking profiles – “My Work”, and “The Internet”. Now this multiple config setup could have been cool, if they hadn’t crippled them both in subtle and painful ways. Setting them up in seemingly logical configs does not work (ie you expect to connect to a network, access that network through a proxy if specified, access it directly if not).

After many many hours of trial and error I found some answers on the net which pretty much confirmed there wasn’t much to be done except half baked workarounds. I’ll outline the situation briefly; Its been a while since I struggled with them properly, but heres the gist:

• Options for the different networking areas are buried, entwined, and otherwise concealed within layers of subterfuge – idiosyncratic ways to get to oddly named tabs and mislabeled options, labels and check boxes. I can only assume this is to prevent joe businessman getting into the settings to mess them up, but they do equally well at confusing IT techs who expect some kind of consistency with other configuration standards. I’ve been hoping since the pocket PC days that they would throw all this out and start again, but sadly WM6 seems to have retained most of it.

• “My Work” traffic is defined by the device as any server accessed without a period-delimited dns entry. Whaa… So ‘ourmailserver’ would be accessed through whatever the ‘My Work” profile uses, but ‘ourmailserver.internaldomain’ won’t be. You don’t get an option to change this. Also, its not specified or appear to be documented anywhere obvious on the device.

• To get to the internet via a connection associated with the ‘My Work’ profile, you must have a proxy server entered. You do not get a choice. No proxy, no internet, regardless of whether you happen to have direct access or not.

• You can specify a list of addresses NOT to use the proxy/internet profile for. (Exceptions). This seemed to be a workaround to get access to the net via VPN from the wireless network on campus (see below).

• Activesyncing the device with a PC seems to arbitrarily replace the proxy settings on the device with the proxy settings of IE from the PC being synched. It took me a while to figure out this is why my old bosses settings would work for a while on his GPRS plan (which uses a proxy server on the ISP’s network), then die (after he docked his pda and the settings were replaced).

• VPN-ing only seemed to be allowed through an ‘internet’ connection. (this might have changed in WM6 – except… well see the next point). In WM5 The device assumes you will never be connecting to a VPN from the ‘My Work’ network. Wrong in our case, as we connect to a VPN internally when using wireless – 99% of how the PDA works. To get this working, the wildcard exceptions workaround needed to be used.

• VPN in WM6 – what VPN? It doesnt work. Sets up fine, then never offers to connect, and attempts to connect manually fail silently. Less than ideal. Fortuantely re-jigging the new devices to use our internal proxy seems to work for most functionality.

• Pocket IE is hardwired to obey the O/S proxy settings. Often I was unable to access web pages because of some internal device proxy confusion based in the proxy settings (third party tools would show clean pings and connections possible to the proxy server and / or the destination server). It is notable that I could often get pocket mozilla (minimo) and pocket opera to load pages when pocket IE would not.

• Pocket MSN seems very sensitive to proxy settings. I have only ever had it working when the device has a direct connection to the net, wither via activesyncing to a PC which has a direct connection, or using GPRS.

Complaining like this smacks of heresay, because its hard to be specific about just where and in what manner things are broken. The place is like a wall of mirrors – and the diatribe sounds like someone ranting on without qualification. It sounds like the ravings of a lunatic, of a n00b, of a crazy man.

In truth, a lot of the complaints I have had seem vapous unless you’ve experienced them yourself. I know – I know these problems exist because I’ve sat for hours struggling with the damn things, and I’ve managed to set up plenty of networking devices before, and they work, so I have to lay it back on the device in question rather than any outstanding incompetence on my part.

I think the problem is this: if you add to the various configuration craziness mentioned before the fact that wireless can be flaky, you have a test environment with shifting terrain which makes it difficult to baseline and describe properly, let alone start mapping out solutions. Regardless, Windows Mobile devices are set to become part of the widespread IT landscape at my workplace very soon, and it will be at least partly up to yours truly to ensure it happens as smoothly as possible, so a-testing I must go.

UPDATE: I have posted a few solutions to some of these issues in the post Windows Mobile 5/6 Networking Profiles, Proxy and VPN setup.

Visit glenscott.net for more content. Some rights reserved: Except where specified otherwise, the content of this feed is licensed under a Creative Commons Attribution-Noncommercial-No Derivative Works 3.0 License.

By far the most time consuming and painful task has been configuring samba / kerberos for AD. Given that this is mostly because of my inexperience in this area: but the entire process seems a lot more painful than it should be. Next time around will certainly be smoother, mostly through what I’ve learned about the available tools such as kinit and net.

Some gotchas:

AllowGroups in sshd_config gave me more hassle than it should have.

Firstly, AllowGroups is CASE sensitive in a funky manner. What I mean by this is, regardless of the actual case of the group according to the active directory, it only works for sshd if the group is specified all lowercase. If not, the group isn’t recognised.

It took me a while to figure out the next reason AllowGroups was not working: it didn’t like co-existing with AllowUsers. To my mind this is a no-brainer, both directives should work. However, once I commented out AllowUsers, the groups started authenticating properly. To maintain access for my local admin users, I added their local group to AllowGroups, so AllowUsers is no longer required.

A puzzlement with account creation which was pretty funny when I discovered the reason: I had left the session PAM module active for samba: every user who browsed the samba share was being created a user account. I twigged to this when I accessed it myself and watched local accounts created both for my user account and my workstation account. Since I don’t want accounts created via samba I split the session, auth, account sections off into seperate PAM files for finer control and made sure samba wasn’t using session anymore.

Once the assorted hassles with AD auth are sorted and behaving as planned, I’ll actually be able to get started on some of the other items in the list. When its all done I’m certainly going to generate some fat HOWTO documentation for my department to east the struggles of the next tech who needs to deal with it.

Visit glenscott.net for more content. Some rights reserved: Except where specified otherwise, the content of this feed is licensed under a Creative Commons Attribution-Noncommercial-No Derivative Works 3.0 License.

Firstly, as this box is to be shell-accessible by students, it needs to be as idiot-proof as possible while still allowing reasonable access to a development environment. This means limitations on runaway forks, inode creation, et cetera.

This machine will need to authenticate and create users based on pre-existing groups in the Active Directory. As we do not have schema access there will be no installation of extensions to the AD for unix support. I am personally against this approach anyway: For simple authentication of users, directly modifying the AD schema seems excessive.

Security is also another fairly prominent concern : I am well aware that once a potentially malicious (or simply curious) user has access to a local account, without appropriate countermeasures and vigilance by the keepers of root, a security breach is quite likely. One of the core study streams at the School of Computer and Information Science where I work is Computer and Network Security: consequently we have a number of potential users fairly au-fait in that area: so adequate security on a system they will be accessing is quite important.

Security concerns currently on my list:

• Preventing local system exploits
• Constraining system resource abuse
• Preventing unauthorised network access*

* As this machine will have internet access outside the firewall scope of other networks accessible to the students (wireless, computer labs etc) an important consideration is preventing its use to circumvent network access restrictions (for example via a userland proxy or chat bot, or unrestricted SMTP). As the development machine is located in a secure DMZ, compromise from the outside due to an insecure listening process of some kind started either accidentally or maliciously by a user is of less concern to interneal network security in general, but measures should still be taken to prevent this sort of thing.

For the time being, no X environment is required, which will save me some work in the short term until I can look at it later in the ‘optional extras’ category.

Items on the agenda to setup:

• User Authentication and auto local account / home directory creation via Active Directory
• C dev environment
• Tomcat based java dev environment
• Access via: SSH, FTP, HTTP, SMB (latter 2 with intelligently mapped user paths)
• Local system firewalling according to security policy
• Auditing of local user logins
• Tripwire integrity auditing
• Backups of config, tripwire db, user homes, and MSQL db to active directory based server

In Part 2 I will cover a few of the items in the list including User Auth via AD, C Dev environment, SSH setup, and any incidentals I encountered along the way.

Visit glenscott.net for more content. Some rights reserved: Except where specified otherwise, the content of this feed is licensed under a Creative Commons Attribution-Noncommercial-No Derivative Works 3.0 License.