monitor websites with websec & cron

Websecretary is not a new piece of software but is still the best free off-the-shelf page monitoring solution that I've been able to find.

It's really as simple as this:

• un tar the archive, 
• alter the config file (url.list) - add your mail addresses and the URLs would want to watch
• run the script (./websec) - it will grab initial copies of your pages
• run it again later & it will indicate any changes that have occurred

This instant gratification is seductive - running from cron is a little more complicated & this is obviously software which you will want to run on a scheduled basis most of the time - monitoring pages & alerting you to any changes automatically. Aside from the fact that 'cron doesn't run as you' (which I always seem to have forgotten about when I need to setup a new job)  there's a couple of things you need to do for websec in particular:

Let's set it up 

As in the man page - websec's data dir is actually ~/.websec/ - something not immediately obvious after the ease of running interactively like above. The essential components:

• url.list - the config file
• ignore.list - configurable stuff to ignore & not count as a change
• archive - containing copied of the watched pages which are fetched & stored so they can be diff'ed

- need to live in in the above data directory (even if you don't have any ignore data - it needs to be there). So a recipe for for the setup is:

• configure your websec
• run it once so it can store initial state (in the archive directory)
• copy all the files/dirs noted above into  ~/.websec/
• make sure that the directory which contains the scripts (websec & webdiff) is in your PATH

reate your scheduled job. In cron, something like:

59 * * * * /full/path/to/websec > /dev/null 2>&1

or, if you want the full output for testing etc:

09 * * * * /full/path/to/websec 2>&1 | mail -s "websec run" your@email.dot.dot

Although you still need to full path to the command in there, it's smart enough to find webdiff (if in the same directory) & also to find the data dir (in this sense, cron IS 'running as you' :) )

osx86 hibernate problems

Specifically addressing an issue that occasionally arises on my eeePC running OSX 10.6 Snowleopard: it tries to start from a hibernate image (i.e. the screen shows a greyed-out image of the Mac desktop with a segmented progress bar along the bottom) - which never completes restoring & freezes at some point near the end.

The quick solution: boot in safe mode (just like a real Mac - hold the shift key until the silver screen with your disks in the center appears). You can type commands at this point which are arguments to the Chamelon boot loader: in this case try:

WakeImage=(name of a non-existant file) - then hit return

You should now have booted normally...

Longer-term solution: conce you can startup and login normally, the following commands in a terminal:

sudo pmset -a hibernatemode 0
sudo rm /var/vm/sleepimage

i.e turn hibernation off (should still sleep OK on lib close etc) and also free disk space by deleting the hibernation image file (roughly the size of your RAM).

free QR code generator

this is the one I've been using: kaywa.com. It just does what it says for me so far: encoding URLS with fairly simple query strings - that scan fine off the computer screen. Google will tell you about x. million other free ones, some of which want you to sign up prior to getting access which is a bit annoying.

a working Perl on xampp

Returning to both Windows and Perl after something of an absence.

The XAMPP application stack is a quick way to get a web server, database and scripting languages onto a Windows box (& other platforms as well). I seem to remember that it was one of the very first one-click application server installers back in the mid-2000s.

Anyway, the version of Perl bundled in the current release (1.8.1) is broken from the point of view of  adding new modules. There's a complex Google trail to be followed but the upshot is: it awaits fixing. It's perfectly usable as a script interpreter but when you try to add modules from CPAN it's missing required system libraries and tools.

The simplest solution is to install Strawberry Perl, which is a port of the language and it's supporting ecosystem to win32 or 64:

* install xampp
* install strawberry perl
* check your environment variables to ensure that strawberry's perl.exe is in the system path (by default: C:\strawberry\perl\bin - rather than inside C:\xampp somewhere).
* alter the shebang line of any cgi scripts you want to call on xampp's apache server to point at strawberry perl (by default at: C:\strawberry\perl\bin\perl.exe).

So your cgi scripts will still live in the location defined in xampp/apache (default: C:\xampp\cgi-bin)  but they will call the strawberry interpreter instead of the bundled one.

This way, your can use the standard CPAN method of adding Perl modules from the command line (perl -MCPAN -e shell) and it will compile and install them into a location where they can be found by strawberry Perl (C:\strawberry\perl\site\lib) - i.e. it's INC path.

ideas for a Wordpress theme

some ideas to start on a sort of template Wordpress theme. That incorporates the following presentation elements:

• minimum-height page/content area, which feeds into the next item:
• sticky-footer - ie. that really, properly, sticks to the bottom of the browser window
• a single depth-index for positioning elements in front/behind each other, using a common CSS position/z-index mechanism.
• completely relative or screen-scaling placement of elements (I've recently been reminded how the last-mention 'depth' aspect can interact with simple things like the scrollability of elements).
• this list should grow...

- probably based upon Thematic or something kind of rational like that. Making the relationships that allow the above kind of interactions explicit and obvious: in that case building a theme is probably the way to get the most general value out of the exercise - it's effectively a level above just doing it as an HTML/CSS/some-script combo because it has to conform to the higher-level organisation of a WP theme for a start.

Then if you begin to add some of the very bread and butter functionality that I think is missing from WP - all logic - ie. theme functions... widgets perhaps...

The last few jobs have involved some deeply defective sites that couldn't be rebuilt for various reasons. Items like the above were really just scratching the surface of nasty dependencies and instant breakages possible with fairly poor HTML/CSS when buried in a Wordpress theme. Somethimes you simply can't do the kind of refactoring or even the rebuild-from-scratch. The last item should always be the last resort but code-analysis/refactoring skills seem to be thin on the ground...

samba: running swat from xinetd

(swat - the web-based tool for managing samba servers and accounts)...well, on some (possibly most these days) linux distros it's run out of xinetd, which makes it pretty manageable: After you install your 'samba-swat' package, edit /etc/xinetd.d/swat to enable & configure it.

the critical lines being:
only_from = IP addresses/ranges you want access from
disable = no # the default being 'yes'

Now, less commonly-know it seems is the xinetd directive (ie. would work for all the apps you can wrap with it):

server_args = as-read-from-the-man-page

The server_args in this case being '-P' which limits swat to really only showing the password-change page by hiding the server config and monitoring elements. I don't want normal users to be able to see any of those items.

atypical web applications

on reflection, there's probably no such thing as a 'typical web app' - they all bring their challenges. I often seem to get the slightly off-centre ones. Not that I'm complaining really but people get told either that something is 'impossible' or is going to cost them 8 grand (which is another way of telling the client to go away).

It often seems that there's a lot of software waiting to be written in the small-business, web/print design space - apps that will encapsulate someones workflow in building or maintaining a complex website, converting images en mass or pulling data into some useful order or interface. Some of these would just be scripts & some real, hosted web applications.

How do you bring these opportunities together? the prospective client and their requirements (and some cash too I hope) and someone who can do the work - to the point where it's complete and actually does what the client wants? It's pretty hard, as I've been finding out. I'm wearing out the opportunities I can see in my network of contacts right now...

***
Actually, I really just want to write about data persistence in the context of work I've performed for clients recently - and crucially - the tech (ie. programming languages, web hosting) I've had little choice but to employ... because this aspect has been really central to even small projects.

I keep coming back to the same thought: Java Servlets(& JSP, JSF kinds of teck built on top of the spec) was the only half-way elegant, well thought out web app framework - and they were more or less complete by 2000. Given the rise of scripting in the intervening decade & the decline of Java outside of enterprise systems we are are actually worse off now than then...