Sometimes a process should just run for a maximum amount of time. A nightly long network transfer, a backup, or a statistical report shouldn’t accidentally run until next business hours. A watchdog timer which kills a process once a certain time has passed by is needed.
As usual in Shell programming multiple ways are possible, all of them have certain drawbacks. The simplest one would be just to call the long running process in the background and capture its PID. But then it is not that easy to capture the return value of it. Here is my shot with an additional at job:
#!/bin/sh
MY_PID=$$
TIMEOUT=1 # in minutes
# Install at job as watchdog to remove long running process
WATCHDOG_CMDFILE=/tmp/`basename $0`-$MY_PID
echo "# watchdogfile script" > $WATCHDOG_CMDFILE
echo "kill -0 `echo $MY_PID` 2>/dev/null" >> $WATCHDOG_CMDFILE
echo "if [ $? -eq 0 ]; then" >> $WATCHDOG_CMDFILE
echo " ps -o pid= --ppid `echo $MY_PID` | xargs kill" >> $WATCHDOG_CMDFILE
echo " echo "long running process aborted because it ran too long"" >> $WATCHDOG_CMDFILE
echo "fi" >> $WATCHDOG_CMDFILE
echo "rm -f `echo $WATCHDOG_CMDFILE`" >> $WATCHDOG_CMDFILE
at -f $WATCHDOG_CMDFILE now + $TIMEOUT min
# Start my very sophisticated long running task
sleep 3600 # 1 hour
RET=$?
# do whatever you normally do after the long running process finishes
echo $RET
(Note that at sends out emails with the stdout/stderr. If you have another notification method to indicate an aborted job, ensure that nothing is printed to std & stderr.)
Programming stuff
I don’t know if my slightly outdated Rails 2.2.2 is the problem. But unlike in the fixtures for normal unit tests, I can’t use ERb for my ActionMailers templates.
Add this method to your ActionMailer::TestCase to let ERb pre-process your fixtures:
def read_fixture(action)
a = super
template = ERB.new(a.join)
template.result(binding)
end
Programming stuff, Ruby || Rails
- Not satisfied with the way Outlook (2003) handles birthdays and anniversaries?
- Want to have multiple birthdays just for one contact entry (maybe because you have one contact for a family, but want to track the birthdays of the countless kids)?
- Do you need to control the reminders for the birthdays?
- Wish you had the classical Palm solution (Feiertage) for your iPhone or iPod touch?
Yippee! Macro time – (again)!
LegeGeburtstageUndJahrestageImKalendarAn
Add multiple birthdays to the notes of your contact and create proper recurrent events. Basically enter for every birthday you want to track for this contact in a new line with the format GEB Name: 24.12.2000 (Note the European style date format day.month.year and the prefix GEB (german abbreviation for Geburtstag = birthday)). Simply add all of your birthdays or anniversaries to your contacts, run the macro, and of you go. For more details see the linked docu.
LoescheGeburtstageUndJahresTageImKalendar()
Remove all (automatically) created recurrent events (e.g. for cleanup purpose)
And if your Outlook asks you for every single modification for allowance, use Advanced Security for Outlook to suppress those questions.
Read more…
Programming stuff
OK. Not really “Programming stuff” anymore. But still important:
Don’t allow remote root logins at all, even with SSH
Why not? Because:
- User name of root is known, therefore the account is vulnerable for brute-force attacks
- Working as root should be an explicit switch and not the default policy. Just like being aware of switching hats.
- Bad for auditing, if multiple users have root access.
But now you say: “I don’t care as I use SSH for logins”.
- Depending on the auth method, password is still transfered over the wire.
- But using public key auth instead of passwords might be even worse. Still you have to trust all(!) clients that the private key is stored safely. Read:
- “Good enough” passphrase.
- There is no way to tell from the public key (which is the only thing known by the server), if the private key has a passphrase at all.
- Trust the client system (that it is not compromised)
- Auto lock of the client system must be enabled after a few minutes of inactivity.
- Sensible use of background daemons like ssh-agent or Pageant(Putty for Windows) on client systems necessary. What if the users start the keyring app, enters his passphrase and never shuts down his system. And now imagine a laptop running out-in-the-wild without any local password protection having an open private key in its memory!
What to do? Dunno. Maybe:
- Use “ordinary” user accounts
- SSH with either public key or passwd auth (depending on your decisions reg the previous points)
- Enforce sudo (better) or su (less better) to gain temporarily root privileges
Anything else???
Programming stuff
Are you using SSH for your secure maintenance of your servers? – Sure.
Do you copy files with SCP between hosts? – Most likely.
Is there a need to transfer big files over slow and unreliable network connections (*)? – Could be.
Can SCP resume a download after the connection crashed? – No.
So why not simply use rsync over SSH for your file transfer. A minor drawback is, that unless you set up a rsync daemon (not appropriate for my case) you have to call rsync manually. Sadly rsync doesn’t offer something like “automatic retry in case of a connection failure”. (**)
Good for us, because now it’s tool time again; a single bash script does the trick:
#!/bin/sh
# reliable file transfer
# try rsync for x times
I=0
MAX_RESTARTS=5
LAST_EXIT_CODE=1
while [ $I -le $MAX_RESTARTS ]
do
I=$(( $I + 1 ))
echo $I. start of rsync
rsync -av --partial --progress -e "ssh" x-ian@x-ian.net:~/MY_BIG_FILE .
LAST_EXIT_CODE=$?
if [ $LAST_EXIT_CODE -eq 0 ]; then
break
fi
done
# check if successful
if [ $LAST_EXIT_CODE -ne 0 ]; then
echo rsync failed for $I times. giving up.
else
echo rsync successful after $I times.
fi
Ah, just a sidenote as I always forget the syntax: If you need to remote execute a command via SSH with variables from your local shell, take this:
CMD="test -e M_BIG_FILE || cp MY_BIG_FILE `hostname -s`-MY_BIG_FILE"
ssh x-ian@x-ian.net $CMD
—
(*) If you only have an unstable satelitte link, even 150 MB are way too big.
(**) Make sure that you actually test over the network; using rsync with source and destination files on the same system deactivates the delta-calculation algorithm.
Programming stuff
Developing web applications can be fun and hard almost at the same time.
But when it comes down to testing, it can be a ….
Sure every developer has his favorite environment and develops against it. But his special browser is just a part of the big picture and there are way to many different browsers and versions out there in the wild. Of course you know them all:
- Internet Explorer [ 6 | 7 | 8 with Compatibility View | 8 ]
- FireFox [2 | 3 ]
- Safari [ 3 | 4 ]
- Chrome
- Opera
And those are only the traditional desktop browsers for MacOS, Win and Linux. (Personally I assume, that one browser behaves almost the same on different OS when it comes to the programming model. But this is not necessarily true for layout issues). Note that this list totally ignores the growing mobile market. I don’t know yet how to deal with them…
It will get even ridiculous if you could only install one version of a browser at the same time. Yes, IE – most credits go to you! There are some hacks out there to install the Internet Explorer in different versions on one system.
In the past I’ve simply virtualized my system and got one image for every single browser. Worked, but it always felt like “shooting cannons to sparrows“. There is quite some overhead involved to install and actually fire up the VM image.
Why not simply virtualize the browser? That’s exactly what Xenocode is doing. Download, run and test. Nice. The seamlessly network integration even make those things like local proxy servers work.
Programming stuff, Ruby || Rails
What is your favorite environment to develop new web applications?
Currently I think there is no way around good ol’ Firefox. Not because of the browser, but more because of all those addons. It took me some time to find my configuration. Here it is:
But too bad, I’m too lazy to describe them all. Anyway you have to get comfortable with your own list. What’s on it?
Programming stuff
Even my [ of course way too old, grandpa-style, ugly, quite unsexy, and nothing close to a nerd gadget ] mobile phone Nokia 6230 offers a PC sync. OK. They were supposed to offer this even in the last millennium. But ever since then I was struggling with it. Not even with a real day-to-day sync but just with a one-time copy operation of my contacts from my PC to the phone. Surely, if I only had/have a Mac. But life is never easy and you should always challenge yourself…
But why on earth is Nokia only offering one entry for a combined name field. In digits: 1, in letters again: ONE. But even more why on earth am at least I not able to put my Outlook contacts in the “Lastname Firstname” order to the phone. The PC Suite seems just to know it better and doesn’t let me. Therefore an alphabetic sort on the phone just shows of all the firstnames first. Crap and I need another solution.
So as I’m a developer I was refusing to simply look for already working solutions or even spend money. I had to do it the do-it-yourself way (for sure hundreds must have done this before).
So here is a small Outlook macro which takes all of your entries in the address book and places them in a new outlook address book called NokiaAddressBook. There it simply swaps lastname with firstname and I get my “Lastname Firstname” style. Stupid ugly, but also stupid simple.
Too bad my Nokia broke recently and now I switched to a Sony Ericsson.
PS1: This solution was heavily influenced. Thanks!
PS2: If you do the sync, close the Nokia Phone browser as it seems to block to connection of the PC Suite to your phone.
PS3: The Nokia Phone Browser supports an import/export of vCards. But (not surprisingly) Outlook does not. Another but: The standard windows address reads vCards. So import vCards first to the Windows addressbook and from there to Outlook…
Programming stuff
If you’re seriously developing a webapp, sooner than later one of your clients will come with some Internet Explorer 6 related complains. Still in 2009, the year of the IE Death march. So what to do if they say: With IE6 is takes 3x longer to load the page than with _any other browser_?
But even if it is not the IE6, you still might wonder how your app behaves with different connection speeds. There are some (windows) applications out there, which try to simulate different network connections. There must be others, but I’ve found Netlimiter and Speed Simulator as out-of-the-box solutions. But Netlimiter is not free and Speed Simulator seems to have some problems with certain requests (AJAX, forwards, … who knows).
When it comes to HTTP-related things, why not using Squid as the de-facto standard for Proxy Servers?
Installing Squid is not as difficult as it may sound, even under Windows. Just make sure that the Delayed Pools Feature is enabled.
For your personal development you can leave all of the default configs in the squid.conf, except those:
- # allow all requests
acl all src 0.0.0.0/0.0.0.0
http_access allow all
icp_access allow all
- # delayed pools
delay_pools 1
delay_class 1 1
delay_access 1 allow all
delay_parameters 1 4000/4000
# 64000 = 64 kbytes = 512 kbits
# 8000 = 8 kbytes = 64 kbits
# 4000 = 4 kbytes = 32 kbits
To change the connection speed, simply tweak the values for delay_parameters e.g. to 64000/64000.
And if your are using Firefox as one of your development browsers, simply install the SwitchProxy Addon. This allows you to quickly switch between preconfigured proxy settings on-the-fly.
[And if you really what to go "pro", set-up multiple instances of Squid with different connection speeds, provide their services under different ports, and create different proxy configs for SwitchProxy. Then you don't even have to modify the squid.conf every time you want to switch the speed.]
Programming stuff
Even though I feel at bit embarrassed to point to jmove.org because of my (current) effords, I’m one of the co-founders. It was finally released as version 1.0!
jmove eases the understanding and valuation of the design and architecture of complex software written in java. It provides dependency analysis, metrics, design rule checking and impact analysis. Define your desired architecture model and check consistency with the implementation.
The complete thing (framework, UI) is implemented in Java and therefore has Java programs in it’s focus. If you want to measure and improve the quality of your Java-based software, give it is still worth giving it a try.
Programming stuff
