Monday, September 24, 2007

Mac OS X Mirror: The RAID for disks that never fail

I built a mirrored array (RAID-1) on a Mac OS X machine, and a disk failed. Which is why you mirror disks in the first place.

The instructions for re-creating a mirror basically tell you to drag a replacement disk into the array -- presto chango, you're done! Except, I can't. Because when I originally created the array, I didn't dig through the options and enable "Auto-rebuilding".

Or, as Tom said, "Did you go into 'advanced' and deselect `corrupt randomly`?"

Friday, September 21, 2007

Macbook evisceration


Dumping a glass of water on your Macbook keyboard is double-plus ungood, but I got a lucky break: when I flipped it over to dump the water, I flipped it towards the DVD drive. The machine was dead, to be sure, but the water went to the optical drive instead of the motherboard.

If this happens to you, I'd recommend this guide to Macbook disassembly, and access to a desiccator.

Saturday, September 15, 2007

Archive

The post preceding this one is archival: the past few years of technical notes, minus the past couple months' worth. Those months were lost when my previous host went away.
Thur, 31 May 2007

SMB under OSX

Wonderfully annoying: when you mount a Samba share under OSX, only root and the user who mounts the share can see the files, irrespective of file permissions and ownership. So, if you want a web server to see a SMB mount, here's the magic: sudo -u www mount_smbfs //username:password@server.domain.com/share /share

Note that /share must be owned by www, in this scenario


Wed, 16 May 2007

Image Magick hints

Convert x.gif to grayscale x.png: convert x.gif -type grayscale x.png

Isolate the red channel from RGB image x.jpg: convert x.jpg -channel R -separate x.png


Tues, 8 May 2007

Python whinging

A couple of quick notes, for my own reference. 1. 16-bit grayscale images can be created from a binary string slike so: Image.fromstring("I", (250,250), s, "raw", "I;16B") 2. the python zipfile modules is SLOW.


Tues, 8 May 2007

The vagaries of coincidence

The .cxi files generated by Applied Biosystems 8200 FMAT imaging software are NOT Clemex format images, though they share a file extension and are both used in biology. No. Instead, .cxi files are RAW image data.


Sat, 7 April 2007

Ugly shell tricks

One particular shell incantation I've managed to remember for some years now: 2>&1

This redirects both STDERR and STDOUT to one channel, and is handy for piping things sensibly into less. But care of O'Reilly's, "Unix Power Tools," here's a discussion of why dumping the resulting stream into a file isn't intuitively done:


One of the common questions about the Bourne and Korn shells is why only
the second command will redirect both stdout and stderr (13.1) to a file:

$ cat food 2>&1 > file
cat: can't open food
$ cat food > file 2>&1
$

Although lots of sh manual pages don't mention this, the shell reads
arguments from left to right.

1. On the first command line, the shell sees 2>&1 first. That
means "make the standard error (file descriptor 2) go to the same place
as the standard output (fd1) is going." There's no effect because both
fd2 and fd1 are already going to the terminal. Then > file redirects
fd1 (stdout) to file. But fd2 (stderr) is still going to the terminal.

2. On the second command line, the shell sees > file first and
redirects stdout to file. Next 2>&1 sends fd2 (stderr) to the
same place fd1 is going - that's to the file. And that's what you want.


Mon, 2 April 2007

Recommended to me by Tom: The Software Project Survival Guide


Fri, 30 Mar 2007

On installing linux on your cheapo Linksys router

The Linsys WRT54G router can easily be had for $50 and converted to a half-decent Linux router. Most discussions of the Linsys WRT54G advise against buying the newer versions (v5 or v6), since they've cut the available flash RAM to 2M. However, it CAN be made to work: first, only use v23_SP2, not the newer SP3. The instructions are pretty good, but omit one important detail: when time comes to upload the firmware, you're computer's ethernet card has to be set to 10Mb/s / half-duplex. Auto-detect won't work, at least on my Intel Macbook.


Tues, 14 Mar 2007

On JSON, Python, Javascript, and types

A Python server and Javascript client interact pretty well, BUT: associative arrays in Javascript are a fragile hack. To create an associative array (or "dictionary," or "hash table") in ECMAscript/Javascript, you treat a generic object, which necessarily has keys and values, as that's what an Object is, as an associative array. But the keys are STRINGS; so a numeric associative array is now keyed off of STRINGS, not INTs. Therefore, if your JS client receives a dictionary keyed off of ints, those ints will automatically be converted to ASCII representation. If this dictionary is then sent back to the server, Python will maintain the string key representations, and everything goes to hell.



Tues, 6 Feb 2007

De gustibus non disputandem est



Fri, 12 Jan 2007

Cell phones & Disappointment

Ok, Apple's iPhone has been unveiled, to much hoopla. But wait: it doesn't support 3G! That's livable, because it's not a cell phone, it's a COMPUTER, right? It runs OS X! But it's NOT: it can't run the software I'd like it to. I guess I'll have to buy a relatively ugly linux-based phone.


Fri, 1 Dec 2006

the cut command

somehow, i've never seen the cut command before. very useful: it returns the n'th field from a line, where the -d option can set the delimeter (eg, the default of tab, or "|", or whatever you want). handy!


Tues, 21 Nov 2006

A thought on the Macbook keyboard, with specific regard to Ubuntu Linux, the Parallels Virtualized Machine, and the <F1> key

By default, the <F1> key isn't a function key; it adjusts screen brightness. Nor is the <F3> key a function key: it mutes the speakers. And so on. In order to access the F functionality of those keys, hold down Fn. That is all.


Fri, 10 Nov 20006

Multi-value inserts in Postgres

the basic syntax is as follows--and it's ugly:

copy xha_param(param_name,param_datatype) from STDIN delimiter '|';
Target identifier|var
Campaign name|var
Injection date|var
Injection type|var
\.


Thurs, 2 Nov 2006

Griping about Python

There really aren't many: Python feels more like a "real" language than a sloppy scripting language, and, frankly, I like it. I'll restrain from doing my "it lacks functional language feature X" routine, as that's not very interesting, and to Python's credit, it has more functional features than you'd expect--including currying as of 2.5!


But: I've got a couple of small complaints.


  1. Mutable types, eg, lists, should return the object in question when a class method is run. For example, list.sort() doesn't return the sorted list, but ... nothing at all. so you can't do something like ",".join(list.sort())

  2. The pass statement. This is a by-product of Python's whitespace-as-syntax design, but it doesn't feel NECESSARY.

  3. The lack of a tertiary operator. This has been discussed at length elsewhere, so I won't belabour the point. [UPDATE: one has been added: A if CONDITION else B -- 10 Nov 2006]

More to come.


Mon, 2 Oct 2006

mod_python on Mac OSX

First off, compilation of mod_python for the Mac OSX defaults--apache 1.3 and python 2.3--is described at http://www.modpython.org/pipermail/mod_python/2004-August/015994.html.

Next, apache config is discussed at http://www.dscpl.com.au/wiki/ModPython/Articles/GettingModPythonWorking. the important bit is the AddHandler python-program .py line: all the other guides only mention the newer syntax, which won't work w/ apache 1.3 (or, more accurately, the older version of mod_python, required for apache 1.3)


Fri, 18 Aug 2006

Hammerhead, web server load tester

i used to use Hammerhead for stress testing web servers, and it was great. but now i've switched from Debian Linux to Mac OSX, and fink doesn't include a hammerhead port ... much fun ensues, getting it to compile. suck, sucky autoconf files. first, add the line target_os=freebsd before the case "$target_os" in block in configure. next, run ./configure --host=x-y-freebsd. then, edit config.h and replace #define HAVE_SSL 1 with #undef HAVE_SSL. edit hammer.mk and remove -pthreads. last, set LIBS=-lresolv in hammer.mk. jesus.

Update: ultimately, i gave up on hammerhead and used time wget.


Wed, 9 Aug 2006

More on Postgres on Mac OSX (fink)

to stop postgres, issue this command: /sw/bin/pg_ctl stop -D /sw/var/postgresql-8.0/data/ ... though i imagine your milage may vary depending upon your exactly version


Tues, 8 Aug 2006

PostgreSQL on Mac OSX Tiger

I used fink to install postgres. The only major problem here is the postgres user, which is added by fink but unusable, as its login shell and home directory are set to /dev/null. Worse, OSX is non-traditional in its account setup, so you can't just edit /etc/passwd, or /sw/etc/passwd-fink. Instead, use NetInfo Manager, which can be found in /Applications/Utilities.


Mon, 17 July 2006

A New Way to Waste Time

Actually, it's an old way, made new again: DOSBox for Mac OSX. And Syndicate, one of my favorite games from the early 1990s.


Thurs, 29 June 2006

The find Command

I finally bothered to read the find man page. Here's how you recursively grep for string: find ~/ -type f -exec grep -l string '{}' \;


Wed, 21 June 2006

Fun with mysqld

FARW left for Seattle today, to the best of my knowledge.


Anyhow, two thoughts about mysqld: in my.cnf, setting bind-address=0.0.0.0 allows connections from all hosts. Also, to enable a user to connect from anywhere, when she is created, give her the name 'somename'@'%' -- that percent sign is the key, here.


Tues, 20 June 2006

Yet another reason to loath Windows

Shiho's Windows XP machine once again Autoupdate'd (didn't i DISABLE that?!?), which renders the machine inoperative--it just reboot automatically ad infinidum, and gives the error message
STOP: c000021a {Fatal System Error}
Windows Logon Process system process terminated unexpectedly
with a status of 0xc0000018

The fix can be found here.


I'll copy it, for posterity:

The newest Critical Patches can be the reason for the error, please

uninstall KB893066, KB890923, KB890859 and KB893086



----------------------------------------

Please follow the procedure suggested by microsoft to solve BSOD

problem from the latest windows update.



1. Insert the Windows XP startup disk in your floppy disk drive or
insert

the Windows XP CD in the CD drive or in the DVD drive, and then restart

your computer.



Note When you receive the following message, press a key to start your


computer from the Windows XP CD:

Press any key to boot from CD



Note Your computer must be configured to start from the CD drive or the

DVD drive.

For more information about how to configure your computer to start from

the CD drive or the DVD drive, see the documentation that came with your

computer or contact the computer manufacturer.



2. When you receive the Welcome to Setup message, press R to start the Recovery Console.

Note Multiple options will appear on the screen.



3. Select the Windows XP installation in question.

Note You must select a number before you press ENTER, or the computer

will restart. Typically, only the 1:

C:\Windows selection is available.



4. If you are prompted to type an administrator password, do so. If you

do not know the administrator password, press ENTER. (Typically, the

password is blank.)



Note You will not be able to continue if you do not have the

administrator password.



5. At the command prompt, type cd $ntuninstallKB.........HERE THE

NUMBER OF THE PATCH.....$\spuninst, and then press ENTER.



Note After you complete this step, you cannot stop the removal

process.



6. At the command prompt, type batch spuninst.txt, and then press

ENTER.



7. After the Ptach is removed, type exit, and then press ENTER oo

Uninstall the Next



please uninstall

KB893066

KB890923

KB890859

KB893086



Tues, 9 May 2006

Whoa.

William Gibson's books & short stories, though mostly in cyrillic. that can't be legal.



Tues, 9 May 2006

SF, post global-warming

care of boingboing, it's SF after a 7m rise in sea levels! or, it should be, once their server recovers. things look suprisingly dry; we'll loose SOMA, but the outer sunset, where i live, is suprisingly unaffected.

quick note on using google maps to find GPS coordinates: when you first text search for a spot on google maps, your "page link" URL will just have your search string. to fix this, zoom in, then look at the "page link" URL--the coordinates are right there.


Thurs, 9 Feb 2006

Accessing all 4G of memory on an AMD64 system

Setting up linux on an AMD64 system. The BIOS has some kind of option which can reserve the upper 1G of memory for PCI DMA iff you have 4G loaded, leaving you w/ 3G accessable--but, oddly, it only demands this 1G tithe if you have all 4G installed. Turning this feature off in the BIOS generates an unusable system spewing IOMMU errors, UNLESS the pci=nommconf option is passed to the kernel. Ugh.



Tues, 10 Jan 2006

On Subversion

Switching to Subversion (svn) in lieu of CVS. A couple of
thoughts:


  1. On debian, CVS comes w/ administrative niceties, eg, it creates a /var/cvs, owned by group src ... svn just comes w/ binaries -- best to create an svn user & group, add a /var/svn, etc

  2. Use fsfs. When the repository is created, the syntax becomes svnadmin create --fs-type=fsfs /var/svn/REP_NAME

  3. If you want to commit, checkout syntax looks like this: SVN_SSH="ssh -l kieran" svn co svn+ssh://your.fileserver/var/svn/REP_NAME


Tues, 5 April 2005

vsftpd & the linux kernel "security options":


So, there i was, working on a new system, and having NO luck getting vsftpd, the "very secure" ftp daemon, working. lots of these sorts of error messages from the ncftp client:

OOPS: cap_set_proc
OOPS: vsf_sysutil_recv_peek


It turns out that i'd compiled the kernel w/ "different security models" enabled, and needed to load the "capability" module. weird.


Mon, 14 Mar 2005

Almost a year since my last post. In deference to the technical-fix purpose of this page, I'll describe my recent struggles w/ a failed terabyte RAID:

I have 5x Maxtor 200g drives crammed into one machine, most of them sharing channels on a Promise IDE card. The whole thing adds up to 0.75TB after parity, ext3 journal, etc. One drive failed catastrophically. This caused the other drive on that channel to irredeemably corrupt a number of sectors, which meant two drives were, in the eyes of the kernel, bad.

Bad bad bad. This means the RAID is _dead_. The fix went like this:


  1. purchase a new drive of exactly the same model, and copy the drive w/ bad blocks onto it via dd if=/dev/bad_drive of=/dev/new_drive conv=noerror,sync . The conv portion tells dd to continue when it hits a bad block, and pad any missing data w/ the appropriate number of zeros.

  2. reinitialize the RAID w/ the copy in place of the bad drive. in my case, this went mdadm -A /dev/md0 -f /dev/hda1 /dev/hde1 /dev/hdf1 /dev/hdg1, where -f tells mdadm to continue w/ only 4 drives. now you should have a functional device.

  3. fsck the RAID

  4. replace the missing drive--in my case, the drive w/ the bad blocks was fine once reformatted; only attempts to read the wonky blocks caused trouble

  5. at this point, everything seemed to be ok, but after a bit, the kernel started remounting the drive ro after finding problems w/ the journal. so, i recreated it: tune2fs -O '^has_journal' /dev/md0 removed the journal, turning the partition into an ext2 partition, and tune2fs -j /dev/md0 recreated the journal.


ugh.


Tues, May 25, 2004

I'm extraordinarily happy w/ my newest laptop, a hand-me-down IBM Thinkpad 600e donated by Inf. It's amazing how useful a PII 400mhz w/ 192meg, a 1024x768 TFT, and 6gig drive still is. The only complication was getting sound working under linux: though lspci reports a "Cirrus Logic CS 4610/11," using ALSA drivers, `modprobe snd-cs4231 port=0x530 irq=5 dma1=1 dma2=0` did the trick.


Sun, Feb 29, 2004

Ahhh, now I can experience one of life's most satisfying treats from the comfort of my home--or, at least, my therapist's office. It's the virtual crack house, under construction as part of a theraputic technique that uses simulated experiences to help people come to terms w/ the real ones. In this case, the virtual crack house is supposed to help addicts learn to supress their cravings. Good fucking luck, there.


Sat, Feb 14, 2004

I've spent various portions of the past month trying to get mod_caml, the ML analog to apache's essential mod_perl, working under debian stable. And I might finally have solved it.


The big hurdle is OCaml's instability, as a language: the chances of a package written w/ version 3.07 of the compiler working on a system running 3.06 are pretty low. And version 3.06 was the backport, ferchristsake. So I've created my own OCaml backports.


How? I first added the testing source repository to /etc/apt/sources.list: deb-src http://ftp.us.debian.org/debian/ testing main non-free contrib. I then used the "apt-get build-dep packagename" command to get install those dependancies which could be found, and "apt-get source --compile packagename" to build those which needed backporting.



Mon, Jan 5, 2004

My species astounds (care of Bruce Sterling's State of the World Interview).


Sun, Jan 4, 2004

Wonder what your Latitude and Longitude are, but don't wanna buy a GPS?


Thur, Jan 1, 2004

The NYT lists "The Futile Pursuit of Happiness" as their most emailed Magazine article of 2003, and i can see why: it's probably provoked more conversation than any other article i've read lately. (here's a cached copy, in case it falls back into pay-to-read land)


Tues, Dec 30, 2003

Now that I've got a DVD+/-RW burner, it'd be nice to understand the difference between the two standards. DVD+RW seems particularly suited to general computing, as it supports true random write access, but that doesn't tell us what the underlying difference is. Apparently, "[the] +RW format uses phase-change media with a high-frequency wobbled groove that allows it to eliminate linking sectors," while "-RW uses groove recording with address info on land areas for synchronization at write time." Oh, well, that clears everything up.


Unrelated: 802.11b/g cards w/ a 1 MILE range???



Sat, Dec 6, 2003

Alright, a bit of politics: is it possible that the Neocons who pushed us into war buy into the crackpot idea that Saddam was not only a genocidal tyrant, but singly responsible for every terrorist attack on the US in the past decade, including, get this, the Oklahoma City bombing? Oh my...


Sat, Nov 29, 2003

I've been certified an Anonymous Coward!!!


Sun, Sep 21, 2003

Just set up MySQL, requisite for b2evolution, a PHP-based blog engine my gf chose, apparently w/ one criterion in mind: it needs to support multiple, discrete blogs. I'm not yet certain how well it does this, as it's pretty out-of-the-box. The MySQL Manual was was pretty handy.


unrelated: I really need to document how I moved a Windows 2000 Pro installation between drives w/ minimal pain. The key here is Sysprep.


Sun, Sep 14, 2003

Dashboard: the open source world's answer to MIT's Haystack project. Both purport to automagically connect associated ideas from within one's individual data stream, thereby reminding one of that URL for the Government Information Awareness site while perusing, say, a discussion of the corrupting effects of campaign donations. (Ok, that's a pretty odd example)


So, how to get it working on a Debian unstable system? I'm not sure yet, but reasonable directions are available at this wiki. That'll refer you to an apt source, which is really handy. Except that the gtk-sharp provided isn't new enough, so you wind up having to compile from CVS. Which sounds fine, BUT: I couldn't get the CVS version to provide gnome-sharp.dll, so i first had to install the .deb version, then compile & install. Not quite sure why that worked. THEN, you need the at-spi package, though noone mentions this. And you need libgtkhtml3.0-2. Ok, but after all this work ... well, no, it still doesn't do anything. Goddamnit.


Sun, Aug 31, 2003

ok, a list of links re: achieving some sort of 'net telephony between either my gf's win2k or linux box. first off, the problem w/ using speakfreely, wonderful little unix/windows utility which worked wonders back in '99 but has apparently been killed by NATs: MacOS X doesn't have a /dev/audio.


next idea, care of inf: GnomeMeeting, which is apparently compatable w/ M$' NetMeeting and runs under OS X. Or so they say.


this entry is much closer to my original idea for this blog... in that i'm really writing this for google's sake.



Fri, Aug 8, 2003

GAR. Much wrestling w/ [Debian's version of] vsftp, the "very secure" FTP daemon. No idea if it lives up to it's name, but the version packaged w/ Debian Stable, 1.0.0, has at least one signifigant shortcoming: whatever the man page says, you must call the anonymous FTP user, which gets an entry in /etc/passwd, ftp. It's hard-coded in tunables.c, so, you could alternately hack @ that and recompile.


Wed, July 30th, 2003

Wrestled, successfully, w/ two different wonky technologies over the past few days: 1. port forwarding w/ Linux 2.4 iptables routing, and, 2. the 2.6.0-test1 kernel.


The voodoo necessary for Port Forwarding looks like this: iptables -t nat -A PREROUTING -p udp -d $PUBLIC_IP --dport $DESIRED_PORT -j DNAT --to-destination $INTERNAL_IP


Oh, and there's a nice flush script, too.


Now, onto 2.6: everything SEEMED to be working--X came up, mozilla loaded w/out a hitch--but there was one glaring omission: xterms. No login prompt to speak of. No virtual console, no ssh login, no xterm. An entry in /etc/fstab instructing init to mount a new fake fs, type devpts, under /dev/pts brought my xterms back, while virtual consoles required that I enable "VGA Text Console" under "Graphics Support -> Console display." That's it. What do I think of 2.6 thus far? Well, by and large, I haven't noticed a difference, except in one very tangible category: responsiveness. Linux on the desktop (read: X) just got a lot friskier.


One other rather big linux-related problem, though I'll address this some other time, is my recent discovery of the fallability of linux' software RAID implimentation. It's Achilles heel is simple: bad blocks. Now, everyone's experienced a bad block or two, and, so long as their numbers stay in the 1-2 range, they don't pose any serious problem. It's only when they begin to multiply that you've got yourself a paperweight for a disk. However, linux' RAID implimentation is a perfectionist: it can't tolerate a single bad block. The second one is detected, the whole disk is booted from the array. This is a Bad Idea (TM).

Monday, September 10, 2007

Machine readable format.

Much of my time is spent cajoling clunky technologies into submission, and I occasionally document the process for myself and any desperate search-engine users who stumble across this page.

The rest of you can move along, there's nothing to see here.