The computer I am using is  an Asus 1215t, with no optical drive and a 12″ screen  it is what some have termed “notbooks”, as in bigger than a netbook, but not really a full sized notebook. This model has the AMD neo low power cpu, integrated ATI graphic, typicall intel HD audio, and most notable a dreaded Broadcom 4313 wireless b/g/n. I think it bears mentioning that I have had little trouble with broadcom wireless in the past, though this may have just been luck. With Ubuntu wireless wouldn’t work until you otherwise connect to the internet and use the Proprietary drivers applet to install the correct driver.

Initially I installed Fedora 15, not wanting to use the beta, and also not wanting to wait the few weeks until it is finalized and  shipped, but I was not able to get the wireless card to work no matter what I tried. For this wireless card there are several drivers available: the old b43 driver that needs proprietary firmware (typically this is what Ubuntu uses), the newer broadcom-sta (aka wl) driver from Broadcom, the fallback ndiswrapper wich just wraps around an appropriate windows driver, and a newcomer the brcm80211 which is open source but still very much in the experimental stage. I tried everything, even compiling the wl driver from the source provide from Broadcom even though there are binaries in the Fedora repos. I never did get the brcm80211 driver to compile, but considering the shape it is currently in that is not a big surprise. It is likely that ndiswrapper would have worked had I not been using a 64bit distro, and the only available drivers are 32bit. I had just about given up hope when I decided to try the beta before running back to Ubuntu. The Fedora 16 beta didn’t have the wbroadcom-sta driver in the repositories, only the b43, which didn’t work. However, after installing the kernel-devel package I was able to compile the source provide by broadcom and it works perfectly.

I have only had a few days with the new gnome3 desktop, but so far I like it, even more than Unity. It is a little different, and takes getting used to but I think in the end it is an improvement over the old classic gnome interface. Some have complained that nothing in the GnomeShell (the default graphical environment that ships with gnome3) can be customized like in classic gnome, but gnome3 is just the framework, in time no doubt people will write their own graphic shells that use gnome3, or applications that allow the tweaking of elements within an installed shell. Since most of the shell is written in JavaScript (with a couple additional libraries) anyone with a little programming knowledge should be able to play with the interface, or write their own.

However, most of the issues I need to tackle are not ones that static analysis will uncover: inelegant code, method misuse, unnecessary variables. A lot of these things are technically correct in that they work and can’t throw any errors, but that doesn’t mean they are easy to understand and maintain or that they are the most efficient way  to achieve the goal of the program.

And of course  that ignores the all issues of UI design. Currently both the CLI and GUI of mdb2mysql need lots of work; the program is functional, but the GUI is missing many “GUI” features such as tab focus switching, execute/next input  with ‘enter’ from text input, additionally I would like to eliminate the modal dialogs. The CLI also is extremely clunky; there are branches that trap the user into irreversible action were the only options are to move forward or kill the application.

Then it was Swing time. While I had hand coded all the Swing exercises in my java classes, it didn’t take me long to realize I needed to use a more full featured IDE than gedit (ok, gedit is not an IDE). So I spent some time just trying to figure out how to use NetBeans. I have to say Netbeans can be really slow sometimes. It was on one of those mornings that I was waiting for NetBeans to finally warmup that I started poking about on google and found the (perhaps bad) tip that OpenJDK performs significantly slower than Sun/Oracles JDK. So a Yak shaving I went. I downloaded the JDK directly form Sun/Oracle and installed it in /usr/local/etc/, which may not have been the right place for it( later moved it to /usr/lib/jvm/). Just for good measure I installed the latest NetBeans, version 7.0. I though that I was all ready to go, but while netbeans had no problem finding the new JDK the rest of the computer was somewhat confused. I though that If I removed the open JDK it would settle on the new one but that turned out not to be the case.

After uninstalling and reinstalling things numerous times I finally got it all working, but for a little while I had the unfortunate experience of having one JDK not compile and another not work at runtime. Not to be slowed down I for a little while I was composing the code and compiling it in netbeans and then testing it from the command line. In the end the problem was that each JDK had different libraries, when I got them all sorted out everything finally just worked.

Then I went through the Depot application from Pragmatic Programmer’s Agile Web Development with Rails. It introduces a lot more features than the Blog application, and the most of the 700+ pages of the book are a pretty deep exploration of the Rails environment. In fact a lot of it was too deep for me to really understand the first time through.

I felt that I needed to explore on my own rather than simply following a guided track to producing a half finished sample application. I have had a project in mind for awhile, so after getting about halfway through Foundation Rails 2 I struck out on my own. Rails2 is a significantly different kind of book than Agile Web development, were Agile is a long deep look, even the depot application is fairly involved, Rails2 takes a number of quick strafing runs at the framework, each one designed to expose one or just a few aspects of the framework. The result is that any code unrelated to the lesson at hand is not present to possibly confuse the reader. I found that this provides a much clearer understanding of how Rails works. Undoubtedly this approach glosses over some details, but that is were the strength of Agile Development comes in, it provides the deep background to fill in any gaps from Rails 2.

About my application; I designed it to be written in three phases: first just a adaptable brochure-ware website. Rails is total overkill for this, but I figured it would be a good starting point as I developed my skills. Next I would add a public photo gallery option for the sites target; a photography business. The third and final stage adds private proofing galleries for customers to pick photographs for processing and ordering of prints, all automated through the site.

I accomplished most of the first phase, although the application is very brittle. I started out by thinking about my data structures for several hours, I still got them all wrong. I have added, subtracted and changed columns in the database a half dozen times as the application evolved. I still have several changes that need to be made, although I am thinking about taking what I have learned and just starting over from scratch. I have learned a lot, in fact I have already rewritten every original line of code several times (and no doubt need to again).

At this point the application has page objects that have associated image and text objects, these objects can be created and edited through an interface (still under construction) and they can be linked to new pages for navigation purposes. The objects are all shaped and position by using inline css values that are held in the database. An approved administrator can log in and edit the applications elements through the web interface. As I mentioned the application is still very brittle and I will likely start over from scratch now that I have a better idea of what the objects should look like and how the applications logic will work. So, until then, I have no code up, although I will eventually get it up on github, or some other code hosting site.

For now I have to get back to the Fall semester at school.

This little project was prompted by the ‘Microsoft Windows Vista Task Reference Recommended Method’ list in the back of my Windows Vista text book. The reference lists 251 different tasks with a single recommended method for each. since there are numerous ways to achieve each task, charting them all out is probably a ridiculous idea. I knew that it would be a big job, so I first searched around for one on the internet, but came up empty.

Just looking the little portion that I have charted, I can’t say that it really makes anything much clearer; this may be the real reason that nobody else thought to try this.

“I’m afraid. I’m afraid, Dave. Dave, my mind is going. I can feel it. I can feel it. My mind is going. There is no question about it. I can feel it. I can feel it. I can feel it. I’m a… fraid.”

[HAL's shutdown]

I decided to take the Ubuntu 10.04 plunge after waiting a week for any last minute glitches to be fixed. Ubuntu 10.04, dubbed Lucid Lynx brings a number of changes to the distro including the 2.6.32 Kernel a new boot splash mechanism called Plymouth and the removal of HAL, the Hardware Abstraction Layer , not the homicidal Heuristic ALgorithmic computer from 2001: A Space Odyssey.

I was happy enough to move up to the new Kernel, in fact that was one of my primary reasons for the change. The 2.6.32 kernel integrates a small change I submitted to the psmouse driver, so now I would no longer have to patch the driver with every kernel update to have a working touchscreen. This change only affects Panasonic Toughbook CF-72 touchscreens, so only me and a few other people will notice it.

Update Manager did not present me with an upgrade option; perhaps because I have some non-standard repositories on my apt sources list. Instead I edited the list with sed;

sudo sed -i 's/karmic/lucid/g' /etc/apt/sources.list
sudo apt-get udate
sudo apt-get dist-upgrade

On the first reboot the touchscreen worked, but after purging residual config from packages that were no longer installed it stopped working. It wasn’t exactly clear to me what happened, but after investigating I found that it had to be linked to the removal of HAL. Oddly enough the rules in /et/hal/rules.d are still there. In fact, earlier when I booted the live disk and looked in /etc/hal, the rules.d folder was still there which had led me to make the false assumption that HAL was still present.

More pressing than the touchscreen was the failure to resume from suspend. To be fair it comes out of suspend, but the screen is scrambled. A quick look around finds that many people are having this problem with older ATI and Nvidia graphics cards, usually because a framebuffer conflict with closed source manufacturer supplied drivers. Even though I use the open source radeon driver I had the same framebuffer conflict. The problem was easily fixed by switching to the uvesafb framebuffer.

First install v86d which is necessary for uvesafb

sudo apt-get install v86d

Next blacklist the default radeondrmfb framebuffer;

sudo bash -c "echo 'blacklist radeondrmfb' >> /etc/modprobe.d/blacklist-framebuffer.conf"

and change  this line in /etc/default/grub;

GRUB_CMDLINE_LINUX="quiet splash"

it should read (all one line);

GRUB_CMDLINE_LINUX="quiet splash nomodeset
video=uvesafb:mode_option=1024x768-24,mtrr=3,scroll=ywrap"

Then edit /etc/initramfs-tools/modules to include uvesafb like this (all one line; notice the escape);

sudo bash -c "echo 'uvesafb mode_option=1366x768-24 mtrr=3 scroll=ywrap'\
 >> /etc/initramfs-tools/modules"

Next Force the Plymouth splash to use the framebuffer;

sudo bash -c "echo FRAMEBUFFER=y >> /etc/initramfs-tools/conf.d/splash"

Lastly update grub and the boot image;

sudo update-grub
sudo update-initramfs -u

With suspend working again, I turned my attention to the touchscreen. The original move to HAL from xorg.conf occurred as most device configuration was automated through various mechanisms. HAL was not well liked and seen as a bit of a kludge; it was the best that could be done at the time. However due the HAL code becoming increasingly unmaintainable, there has been a move to deprecate it and move on resulting in its exclusion from the current Ubuntu distribution.

According to wiki.ubuntu.com the xorg input configuration tasks are to be moved to udev. I have written a few udev rules, but quite frankly I have found the syntax even more complex than HAL xml files (which were pretty simple actually). A little crash course was in order, between this document and the udev man page I thought I had a rule that would work. It didn’t work.

Despite what has been posted at wiki.ubuntu.com the current official method is to use either xorg.conf or device specific snippets in separate files placed in /etc/xorg.conf.d/(e.g. /etc/xorg.conf.d/10-synaptics.con containing touchpad configuration). By separating the the machine/device specific configurations into separate files in xorg.conf.d the configuration files will be more machine editable, which has always been one of the big complaints about xorg.conf and one of the few things that HAL had going for it. If you use xorg snippets you need to make several to cover all your inputs, I tried just one for the touchscreen and ended up with a functioning touchscreen and nothing else. Acording to this blog post you should be able to combine a general purpose xorg.conf with snippets, but I never had any luck with that. What did work for me was making three snippets;
for the touchscreen

# /etc/xorg.conf.d/90-touchscreen.conf
Section "InputDevice"
        Identifier      "Fujitsu TouchScreen"
        Driver          "evtouch"
        Option          "Device" "/dev/input/event6"
        Option          "DeviceName" "touchscreen"
        Option          "MinX" "230"
        Option          "MinY" "220"
        Option          "MaxX" "3900"
        Option          "MaxY" "3850"
        Option          "MoveLimit" "5"
        Option          "ReportingMode" "Raw"
        Option          "SendCoreEvents" "true"
        Option          "Emulate3Buttons" "true"
        Option          "Emulate3Timeout" "40"
        Option          "AutoServerLayout" "on"
EndSection

the keyboard

# /etc/xorg.conf.d/97-keyboard.conf
Section "InputDevice"
        Identifier      "Default Keyboard"
        Driver          "kbd"
        Option          "AutoServerLayout" "on"
EndSection

and the evdev catchall

# /etc/xorg.conf.d/99-evdev.conf
Section "InputClass"
        Identifier      "evdev-catchall"
        Driver          "evdev"
EndSection

sudo

Challenge 19 is titled ‘please!’. It presents you with a strange picture of India with the colors, well… off (inverted?). Looking in the page source you will find an email hidden in a block comment. The email is from Leopold the subject line says;

what do you mean by “open the attachment?”

the message text reads;

It is so much easier for you, youngsters.
Maybe my computer is out of order.
I have a real work to do and I must know what’s inside!

It is a multipart message with a file named “indian.wav” attached at the bottom. As is standard (I now know) the attachment is encoded in base64. Since encoding/decoding email attachments is common place I figured this part should go quickly. I installed the tmail ruby gem and quickly wrote a small script to open the email. It didn’t work. I was able to read the message and any of the tags in the email, but it always failed to open the attachment. Finally I went to the tmail website and read all the way to the bottom of of the quick start guide I find this;

. . . TMail does not touch the body of the mail message, for example, it doesn’t encode, decode, handle character sets etc.

Well that’s pretty useless!

Ok, not entirely I suppose, it gives you access to the elements within an email message and then you are expected to write your own code to actually manipulate the data.

Luckily the author of tmail wrote a new mail gem called ‘mail’ that can automatically decode attachments. It still turned out to be a lot harder than it should have. Although ‘tmail’ had no trouble with the email header ‘mail’ choked on it until I changed the content_type: from “Multipart/mixed” to “multipart/mixed”

With indian.wav successfully decoded you will find a bit of white noise interrupted by somebody saying “sorry!” sorry. html is no good (c’mon, it wouldn’t be that easy!). There is something else hidden in the wave file. Lets go to Stanford and learn a little bit more about the wav format.

Before I even read the page one word jumped out at me ‘endian’. Endian, Indian, get it. So the riddle has something to do with the endianness of the wav data. To test this theory I fired up Audacity and imported the file as raw data changing the data to big endian from the default wav standard of little endian. The resulting sound clip is of a singer calling me an idiot and laughing at me. That solves the riddle, but how to code it? Ruby has no support for audio manipulation (that I could find). However, the wav format is pretty straightforward, there is a 44 byte header followed by 4 byte audio data sub chunks. Since endianness is simply the order of data by importance (little endian is least important to most; big endian is most import to least) I should be able to simple grab each sub chunk in order and reverse them to get the decoded file.
Surprise! it worked!

#! /usr/bin/env ruby1.9
require 'rubygems'
require 'mail'

email = Mail.read('please.txt') # open the email message

rd, wr = IO.pipe # use a pipe; mail attachment has Strong data type
if fork
  wr.close
  wav_bytes = rd.read # read data from the pipe into a string
  File.open('idiot.wav', 'wb') { |f|
    new_wav = wav_bytes.first(44) # First grab the wav header
    d = 44
    while d < wav_bytes.length # then grab 4 byte wav sub chunks
      new_wav += wav_bytes[d, 4].reverse  # and reverse them
      d += 4
    end
    f.write( new_wav ) # write "idiot.wav"
    f.close()
    }
  Process.wait
else
  rd.close
  wr.write( email.parts[0].body ) # write email attachment into pipe
  wr.close
end

On to Challenge 20: go away!

deltas.gz contains a textfile with two columns of of 18 two digit hex values. They start out the same but they quickly start to change. It seems pretty obvious that the differences need to be found; my suspicion was backed up by a google search of ‘deltas python’ with the difflib page being top ranked.

Looking around a bit I was unable to find a standard library in ruby for finding the differences in two files or strings the way gnu diff or pythons difflib does. This makes sense when you consider that most of the time ruby is running on a unix/linux system with perfectly capable binary tools for such work.

I did find several gems that provide similar functionality, but nothing that would work quit right. HTMLDiff was the closest, its output is formated in html which was a minor annoyance. However the bigger problem I had was that it frequently falsely classified lines as new and old rather than just marking the lines around them as changed. This resulted in unchanged lines being sent to both the new and the old categories rather than the unchanged category. After banging my head against various gems I finally gave up and used gnu diff. The output is three png files.

#!/usr/bin/ruby
# Challenge 18
require 'zlib'

Zlib::GzipReader.open('deltas.gz') {|gzFile|
# Found lots of hexidecimal values in two columns

        column, output = ['', ''], ['', '', '']
        splitLines = gzFile.read.split("\n")
# Split the two columns apart row by row
        splitLines.each { |line|
                if line[0, 53].rstrip != ""
                        column[0] += (line[0, 53].rstrip + "\n")
                end
                if line[56, 53].rstrip != ""
                        column[1] += (line[56, 53].rstrip + "\n")
                end

# Since I am using gnu diff I need to write them to files
                column.length.times {|i|
                        f = File.new("column" + i.to_s, "w")
                        f.write(column[i])
                        f.close
                }
        }
        system("diff column0 column1 --old-line-format='- %L' --new-line-format='+ %L'  \
               --unchanged-line-format='%L' >output")

# Separate the output of diff into three strings
        differences = File.open("output", "r")
        differences.read.split("\n").each { |line|
                if line[0, 2] == "+ "
                        output[0] += (line.gsub("+ ", "") + ' ')
                elsif line[0, 2] == "- "
                        output[1] += (line.gsub("- ", "") + ' ')
                else
                        output[2] += (line + ' ')
                end
        }
        differences.close
# Finally, change the hex values into their equivalent characters and write them to file.
        output.length.times { |i|
            chrString = ''
                output[i].split().each {|hexVal|
                        chrString += hexVal.hex.chr
                        f = File.new("deltas18_" + i.to_s + ".png", "w")
                        f.write(chrString)
                        f.close()
                }
        }
}

With a new url, login and password we arrive at challange 19: “Please!”

I have another hard drive that I keep in an USB enclosure for backups that is a lot bigger it seemed less risky to copy the partitions onto it and enlarge them there. This way I would still have a backup of the original partitions. Everything seemed to go well, no errors. I knew from the gparted website that after the move I would need to run windows chkdisk and use the install disk to repair the the boot loader, but since I was going create a fresh install of Windows 7 I figured I would let it take care of these details. The install went fine, it even detected that the Vista partition was flagged for a run of chkdisk and scanned it during the install. After the last reboot into Windows 7 I gave Vista a try. It would boot up fine but after login it would just stall. I poked around and found that I could bring up the task manager and manually load explorer, but doing so brought up numerous dll errors (actually failures to find various procedure entry points in various dll’s). In this way I could get a desktop but only with guest privileges. I poked around the internet and found that other people had similar problems but none of them had the dll errors and they where able to fix it using the administrator account. Since I could only get guest permissions I couldn’t even make any system changes or even set up a new login account, but since I was getting additional errors my problem was different and undoubtedly bigger.

I finally gave up and decided to re-copy the partition with gparted. Just as it had the first time it completed without errors. This time following the directions from the gparted website exactly I put in the Vista install disk, selected the repair option and when it rebooted it ran chkdisk. Watching what it was scanning I noticed that it was rebuilding security descriptors(I assume this has something to do with User Account Control) . After scanning and rebooting it was fine, although it then requested another reboot. I can only guess that when It was scanned the first time by Windows 7 the security descriptors were set for it (7) instead of Vista.

A side note; The clean install of Windows 7 is only about 5GB, although I still need to update it and install antivirus. I still don’t see that ballooning it out to 13GB