Archive

Archive for December, 2008

Tiff to JPEG

December 20th, 2008 No comments
for file in `ls *.tiff`; do file2=`basename $file .tiff`;/bla/bin/tifftopnm "$file" | /bla/bin/pnmtojpeg > "$file2.jpg"; done
Categories: Unix Tags:

Per virtual user sa-learn training

December 17th, 2008 3 comments

Context

I use a LDA that use Virtual User, and store email in /some/path/mail/<domaine.tld>/<user>/, this is a quite standard way to do.
I also use spamassassin but wanted to have a per user bayes database and configuration. It’s still simple with spamc/spamd by running spamd -c --virtual-config-dir=/some/path/to/spamassassin/%d/%l ... and invoking spamc from you MTA with spamc -u ${recipient} -f -e /path/to/your/LDA so that i have user preference in /some/path/saconf/<domaine.tld>/<user>/.
Now I would like to provide 2 imap folder to users, LearnSpam and LearnHam so that they could train their bayes database.
Here the problem start, especially if you are not using one of the latest spamassassin version.

The bad way

What sa-learn command will you run to take care of LearnSpam and LearnHam folders ? sa-lean has an –username option, you may want to use that but this is not intended to be use in this case, it’s to be used when bayes database are stored in an SQL database instead of file (this is correctly documented in latest SA version). So don’t try sa-learn --username=<user>@<domaine.tld> --spam /some/path/mail/<domaine.tld>/<user>/.INBOX.LearnSpam/cur/* it will not work. Imagine how this can work ? it can’t, how sa-learn could convert <user>@<domaine.tld> to /some/path/saconf/<domaine.tld>/<user>/ ?

The good way

So the right command to use is sa-learn -p /some/path/saconf/<domaine.tld>/<user>/user_prefs --spam /some/path/mail/<domaine.tld>/<user>/.INBOX.LearnSpam/cur/* Using the -D (debug) option could be very helpfull to check if it’s work correctly, you must see dbg: bayes: tie-ing to DB file R/O /some/path/saconf/<domaine.tld>/<user>/bayes_toks

Categories: Unix Tags:

Backup file of multiple user with rsync

December 17th, 2008 No comments

root problem with rsync

Imagine that you want to backup the /home directory of server ‘A’ to server ‘B’ using rsync.

There is two way to do this :

  • You can run rsync on the server ‘A’, but if you want to correctly backup (I mean, having correct uid/gid/.. on backuped files) files you should connect to the server ‘B’ as root. I’m sure you don’t want to do that.
  • You can run rsync on the server ‘B’, but you should connect to ‘A’ with an user that can read all file in /home. This could be complicated depending of your gid managment.

When Tar start to be your best friend

So how can you do ? the solution would be to store (uid/gid/permission/..) information in a dedicated file, so that you can apply them if you need to restore data.
How can you do that ? I’m sure you are too lazy to write a shell/perl/python/.. script to do that. You’re right ! Use tar.
What ? What ? You want me to tar /home and rsync it ? Are you mad ? I don’t use rsync to transfer 20Go at each backup.

When 1 option and 2 lines can save you

Tar as an incremental option. This mean that you can make a 1st tar file with /home then you can do a 2nd tar file with only modified file since previous tar. This option is -g.
Here is a 2 lines shell script to do the job

gtar -g /var/backup/home/home-backup.snar -cpvzf /var/backup/home/home-backup.`/bin/date +%s`.tgz /home/
rsync --delay-updates -avz -e ssh /var/backup/home backupuser@'B':/var/backup/

–delay-updates is very important because if you don’t use it if ‘A’ crash when rsync is copying the .snar file (used to store incrementation information) you will miss it on ‘B’ and can’t retore tar file correctly.
-g only exist in GNU Tar. You may have to install it if you’re running *BSD. First check if you have a gtar binnary

Categories: Backup, Unix Tags: , ,

Misconfigured Perl install path

December 17th, 2008 No comments

I got a strange problem on a server. CPAN was installing module outside of perl @INC, this is quite a nightmare.
I found what was wrong and it was quite easy to solve.
Run this command:

# perl -V:'install.*'

Look at installprivlib and installarchlib. If they don’t match @INC you can change them at the end of perl Config.pm (in tie %Config, ‘Config’, {..})

To find your Config.pm file type

# perl -MConfig -le 'print $INC{"Config.pm"};'

If you only use CPAN to install module I guess you can just change PREFIX in the CPAN Config.pm (but I haven’t tested this)

N.B.: If you can’t find installprivlib and installarchlib in tie %Config just add them.

Categories: Perl Tags: ,

Getting human reading time info from RRD files

December 17th, 2008 No comments

You have a .rrd file but you don’t remember how much time RRAs can store data ? all right, no problem.

First save this Perl code into a .pl file.

$step = $1 if (m/step = (\d+)/);
$rows = $1 if (m/rra.*\.rows = (\d+)/);
if (m/(.*)\.pdp_per_row = (\d+)/) {
$pdp = $2; $time = $step*$rows*$pdp;
if ($time > 31536000) { $time = sprintf("%.2f year",$time/31536000) }
elsif ($time > 86400) { $time = sprintf("%.2f days",$time/86400) }
elsif ($time > 3600) { $time = sprintf("%.2f hours",$time/3600) }
print "$1: $step*$rows*$pdp = $time\n";
}

Assuming the filename of the script is rrd_info.pl, run this Shell command:

rrdtool info file.rrd | perl -n rrd_info.pl

Here is an output example:

rra[0]: 300*864*1 = 3.00 days
rra[1]: 300*864*5 = 15.00 days
rra[2]: 300*702*25 = 60.94 days
rra[3]: 300*840*125 = 364.58 days
rra[4]: 300*840*625 = 4.99 year
rra[5]: 300*864*1 = 3.00 days
rra[6]: 300*864*5 = 15.00 days
rra[7]: 300*702*25 = 60.94 days
rra[8]: 300*840*125 = 364.58 days
rra[9]: 300*840*625 = 4.99 year
rra[10]: 300*864*1 = 3.00 days
rra[11]: 300*864*5 = 15.00 days
rra[12]: 300*702*25 = 60.94 days
rra[13]: 300*840*125 = 364.58 days
rra[14]: 300*840*625 = 4.99 year

The formula is update step time * numbers of rows in the rra * pdp_per_row

Categories: Perl, RRDTool Tags: , ,

Booting NetBSD on Mac (iMac DV+)

December 17th, 2008 No comments

This is what to use to boot NetBSD MacPPC CD on an iMac DV+ (at least):

boot cd:,ofwboot.xcf netbsd.macppc

or

boot cd:0,ofwboot.xcf netbsd.macppc

You may not need to put netbsd.macppc depending of the CD


To boot on HD try this:

boot cd:,ofwboot.xcf hd:/nebtsd

or

boot cd:,ofwboot.xcf hd:13/nebtsd

Where 13 is the number of the netbsd partition (usualy near 13)

Categories: NetBSD Tags: ,

PPP Over SSH

December 17th, 2008 No comments

You may want to establish a full IP connection to a remote host (or remote lan) but you may not have any VPN software on the remote host, or on your host.
There is a solution using SSH and PPP with the command

pppd pty 'ssh -x -t -e none user@server /usr/sbin/pppd passive noauth 9600' noauth 10.0.0.1:10.0.0.2

You have to use key authentication because the tty is redirected to pppd so you can’t be prompted for a password.
With this command, you can reach server at IP 10.0.0.1.
By playing with pppd and route tables you can extend the IP tunnel to the entire remote LAN.

This has been tested between Mac OS X and NetBSD but should work with any system.
It works but it’s very slow.

On some system you may have TTY problem, if it’s your case take a look to http://shinythings.com/pty-redir/ or http://www.ishiboo.com/~nirva/Projects/vpn/

Categories: Unix Tags: ,

Best hints for Mac OS X 10.3 (Panther)

December 17th, 2008 No comments
Categories: Mac OS X Tags: ,

Get HTTP arguments when using PHP as CGI

December 17th, 2008 No comments

Normaly when you use PHP as CGI, $_GET and $_POST should work as usual.
But in certain configuration, they may not be set. In this case this code is the base to get args

foreach (split('&',$QUERY_STRING) as $tmp) {
$tmp2 = split('=',$tmp);
$$tmp2[0]=$tmp2[1];
}

Categories: Uncategorized Tags: ,

Counter reset problem with RRDTool

December 17th, 2008 No comments

A common problem with rrdtool is to have a rrd file with one or more datasource configured as COUNTER, but your counter reset and you get an ugly high peak.

There is 2 solutions to solve this

  • In the “script” you use to generate the graph file you have something like DEF:A=/path/to/your/file.rrd:dsname:AVERAGE to get data, then you graph it with something like LINE1:A#…..
    Try to add CDEF:B=A,MINVALUE,GT,A,MAXVALUE,LT,A,UNKN,IF,UNKN,IF and graph B instead of A.
    Replace MINVALUE with the minimun value you want to graph (typically 0) and MAXVALUE with the maximum value you want on the graph (100000000 or something like this).
    This will suppress ugly peak on the graph.
  • The second (better) solution is to change from COUNTER DS to DERIVE DS, to do so follow theses commands:
    # cp file.rrd file.rrd.bkup # this is to backup the file !
    # rrdtool tune file.rrd
    # rrdtool tune file.rrd --data-source-type dsname:DERIVE # this change DS dsname from COUNTER to DERIVE
    # rrdtool tune file.rrd --minimum dsname:0 --maximum dsname:100000000 # Set the min and max value of data you have to graph
    # rrdtool tune file.rrd
    # rrdtool dump file.rrd > file.xml # Dump data to xml file
    # rrdtool restore -r file.xml file.rrd.new # This restore xml data in rrd file with a check on value
    # to suppress value higher than --maximum you choose
Categories: RRDTool Tags: ,