Archive

Author Archive

Install ESXi 7 with less than 4GB of RAM

July 19th, 2022 No comments

If you are looking a way to install ESXi with less than 4GB of memory, you’re on the right place.

Before starting, please note that ESXi 7 will use about 1.4GB for itself, so if you have less than 4GB of memory, you will end up with a very limited amount of RAM to run VMs. But that may be enough for a test system.

Like me, when googling on how to solve the issue, you probably ended up on this page https://open-sourced.be/installing-esxi-with-less-than-4gb-of-ram/

The problem is that with ESXi 7 you will not be able to edit update_precheck.py because of filesystem restriction preventing to alter files.
Please note that even if the file is called update_precheck.py it apply to fresh installation.
It’s interesting to see in update_precheck.py that the actual requirement is not 4GB but 4GB minus 3.125% = 4096M – 128MB = 3968MB
My system do have a 4GB of RAM but 192MB is used for the integrated video card, leaving me with 3804MB seen by the OS.

def memorySizeComparator(found, expected):
    '''Custom memory size comparator
    Let minimum memory go as much as 3.125% below MEM_MIN_SIZE.
    See PR 1229416 for more details.
    '''
    return operator.ge(found[0], expected[0] - (0.03125 * expected[0]))

def checkMemorySize():
    '''Check that there is enough memory
    '''
    mem = vmkctl.HardwareInfoImpl().GetMemoryInfo()
    if hasattr(mem, 'get'):
       mem = mem.get()
    found = mem.GetPhysicalMemory()

    MEM_MIN_SIZE = (4 * 1024) * SIZE_MiB
    return Result("MEMORY_SIZE", [found], [MEM_MIN_SIZE],
                  comparator=memorySizeComparator,
                  errorMsg="The memory is less than recommended",
                  mismatchCode = Result.ERROR)

If we can’t modify the file during the installation, let’s do it on the installation media.
To do that, you will need another computer (likely the one with which you created the installation media), a tool to compress/decompress the gzip format and an hexadecimal editor.

In my case I did the modification on macOS. On the root directory of the installation media, you will found a file called WEASELIN.V00, this file is a tar file compressed with gzip. Let’s uncompress it

$ file WEASELIN.V00 
WEASELIN.V00: gzip compressed data, max compression, original size modulo 2^32 2548792
$ mv WEASELIN.V00 WEASELIN.V00.gz
$ gzip -d WEASELIN.V00.gz 
$ file WEASELIN.V00 
WEASELIN.V00: tar archive

For a reason I did not look for, if you try to untar the file, you will get errors

$ tar xf WEASELIN.V00 
tar: Damaged tar archive
tar: Retrying...

If we can’t untar the file to directly edit upgrade_precheck.py, let’s edit the tar archive directly as a tar file is a collection of file “as is” (I mean there is no compression, no encoding, etc.)

So open WEASELIN.V00 in your favorite hexadecimal editor, in my case i used Hex Friend

Search for MEM_MIN, you will see MEM_MIN_SIZE = (4 * 1024) * SIZE_MiB, edit 4 by something fiting your need. In my case I changed 4 by 3

Save your change and compress WEASELIN.V00

$ gzip WEASELIN.V00 
$ mv WEASELIN.V00.gz WEASELIN.V00

Your are good to eject the installation media (close your terminal windows if you’re in your installation media directory) and retry the installation. Enjoy

Categories: Uncategorized Tags:

Trend / Prediction with RRDtool

December 4th, 2009 18 comments

I’ve not used RRDtool for a while and put back my attention on it few weeks ago. I found out that lots of new cool stuff are avalaible, like LSLSLOPE, LSLINT. These function return the parameters of the Least Squares Line (y = ax +b) approximating a dataset (LSLSLOPE return a, LSLINT return b).
This is interesting because with the function approximating your data you can graph a prediction of future data. Of course a Least Squares Line function will work best to approximate a dataset that tend to grow or shrink (like filesystem usage, memory usage, …) but not for data like temperature. I would say that if your data can be expressed in a percentage, an Least Squares Line can be fine. For data not tending to grow or shrink rrdtool provide some other function like TREND and PREDICT.

I will show how to use LSLSLOPE and LSLINT taking memory usage of a device as an example. My exemple will produce a graph like the following :
MemoryTrend

As you see, the graph show trend using two Least Squares Line function, one generated from the full dataset (dataset is starting 24 Oct 2009) and one generated only from last week data. Projection on time axis is done from 90% to 100% of memory usage and the date resulting of calculation for 90% and 100% of usage is displayed. I’ve seen lots of question asking how to do this but did not found any answer, so I hope that my example will provide an answer.

Here is the perl code I’m using to generate this graph. There is no Perl specific code, so it can be converted to a normal rrdtool command.

#! /usr/bin/perl
use RRDs;

$rrd_file = 'MEMORY.rrd';

RRDs::graph "MEMORY_Trend.png",
'--start', "10/24/2009",
'--end', "12/31/2009 00:00am",
'--title', "Memory Usage",
'--interlace', '--width=620', '--height=200',
"--color","ARROW#009900",
'--vertical-label', "Memory used (%)",
'--lower-limit', '0',
'--upper-limit', '100',
'--border','0',
'--rigid',

"DEF:used1=$rrd_file:used:AVERAGE",
"DEF:used2=$rrd_file:used:AVERAGE:start=10/24/2009",
"DEF:used3=$rrd_file:used:AVERAGE:start=-1w",
"DEF:used4=$rrd_file:used:AVERAGE:start=-2w",
"DEF:used5=$rrd_file:used:AVERAGE:start=-4w",
"DEF:free1=$rrd_file:free:AVERAGE",
"DEF:free2=$rrd_file:free:AVERAGE:start=10/24/2009",
"DEF:free3=$rrd_file:free:AVERAGE:start=-1w",
"DEF:free4=$rrd_file:free:AVERAGE:start=-2w",
"DEF:free5=$rrd_file:free:AVERAGE:start=-4w",

"CDEF:pused1=used1,100,*,used1,free1,+,/",
"CDEF:pused2=used2,100,*,used2,free2,+,/",
"CDEF:pused3=used3,100,*,used3,free3,+,/",
"CDEF:pused4=used4,100,*,used4,free4,+,/",
"CDEF:pused5=used5,100,*,used5,free5,+,/",

“LINE1:90″,
“AREA:5#FF000022::STACK”,
“AREA:5#FF000044::STACK”,

"COMMENT:                         Now          Min             Avg             Max\\n",
"AREA:pused1#00880077:Memory Used",
'GPRINT:pused1:LAST:%12.0lf%s',
'GPRINT:pused1:MIN:%10.0lf%s',
'GPRINT:pused1:AVERAGE:%13.0lf%s',
'GPRINT:pused1:MAX:%13.0lf%s' . "\\n",
"COMMENT: \\n",

'VDEF:D2=pused2,LSLSLOPE',
'VDEF:H2=pused2,LSLINT',
'CDEF:avg2=pused2,POP,D2,COUNT,*,H2,+',
'CDEF:abc2=avg2,90,100,LIMIT',
'VDEF:minabc2=abc2,FIRST',
'VDEF:maxabc2=abc2,LAST',

'VDEF:D3=pused3,LSLSLOPE',
'VDEF:H3=pused3,LSLINT',
'CDEF:avg3=pused3,POP,D3,COUNT,*,H3,+',
'CDEF:abc3=avg3,90,100,LIMIT',
'VDEF:minabc3=abc3,FIRST',
'VDEF:maxabc3=abc3,LAST',

"AREA:abc2#FFBB0077",
"AREA:abc3#0077FF77",
"LINE2:abc2#FFBB00",
"LINE2:abc3#0077FF",

"LINE1:avg2#FFBB00:Trend since 24 Oct 2009                      :dashes=10",
"LINE1:avg3#0077FF:Trend since 1 week\\n:dashes=10",
"GPRINT:minabc2:  Reach  90% @ %c :strftime",
"GPRINT:minabc3:  Reach  90% @ %c \\n:strftime",
"GPRINT:maxabc2:  Reach 100% @ %c :strftime",
"GPRINT:maxabc3:  Reach 100% @ %c \\n:strftime",

;

my $ERR=RRDs::error;
die "ERROR : $ERR" if $ERR;
Categories: RRDTool Tags: ,

Tiff to JPEG

December 20th, 2008 No comments
for file in `ls *.tiff`; do file2=`basename $file .tiff`;/bla/bin/tifftopnm "$file" | /bla/bin/pnmtojpeg > "$file2.jpg"; done
Categories: Unix Tags:

Per virtual user sa-learn training

December 17th, 2008 3 comments

Context

I use a LDA that use Virtual User, and store email in /some/path/mail/<domaine.tld>/<user>/, this is a quite standard way to do.
I also use spamassassin but wanted to have a per user bayes database and configuration. It’s still simple with spamc/spamd by running spamd -c --virtual-config-dir=/some/path/to/spamassassin/%d/%l ... and invoking spamc from you MTA with spamc -u ${recipient} -f -e /path/to/your/LDA so that i have user preference in /some/path/saconf/<domaine.tld>/<user>/.
Now I would like to provide 2 imap folder to users, LearnSpam and LearnHam so that they could train their bayes database.
Here the problem start, especially if you are not using one of the latest spamassassin version.

The bad way

What sa-learn command will you run to take care of LearnSpam and LearnHam folders ? sa-lean has an –username option, you may want to use that but this is not intended to be use in this case, it’s to be used when bayes database are stored in an SQL database instead of file (this is correctly documented in latest SA version). So don’t try sa-learn --username=<user>@<domaine.tld> --spam /some/path/mail/<domaine.tld>/<user>/.INBOX.LearnSpam/cur/* it will not work. Imagine how this can work ? it can’t, how sa-learn could convert <user>@<domaine.tld> to /some/path/saconf/<domaine.tld>/<user>/ ?

The good way

So the right command to use is sa-learn -p /some/path/saconf/<domaine.tld>/<user>/user_prefs --spam /some/path/mail/<domaine.tld>/<user>/.INBOX.LearnSpam/cur/* Using the -D (debug) option could be very helpfull to check if it’s work correctly, you must see dbg: bayes: tie-ing to DB file R/O /some/path/saconf/<domaine.tld>/<user>/bayes_toks

Categories: Unix Tags:

Backup file of multiple user with rsync

December 17th, 2008 No comments

root problem with rsync

Imagine that you want to backup the /home directory of server ‘A’ to server ‘B’ using rsync.

There is two way to do this :

  • You can run rsync on the server ‘A’, but if you want to correctly backup (I mean, having correct uid/gid/.. on backuped files) files you should connect to the server ‘B’ as root. I’m sure you don’t want to do that.
  • You can run rsync on the server ‘B’, but you should connect to ‘A’ with an user that can read all file in /home. This could be complicated depending of your gid managment.

When Tar start to be your best friend

So how can you do ? the solution would be to store (uid/gid/permission/..) information in a dedicated file, so that you can apply them if you need to restore data.
How can you do that ? I’m sure you are too lazy to write a shell/perl/python/.. script to do that. You’re right ! Use tar.
What ? What ? You want me to tar /home and rsync it ? Are you mad ? I don’t use rsync to transfer 20Go at each backup.

When 1 option and 2 lines can save you

Tar as an incremental option. This mean that you can make a 1st tar file with /home then you can do a 2nd tar file with only modified file since previous tar. This option is -g.
Here is a 2 lines shell script to do the job

gtar -g /var/backup/home/home-backup.snar -cpvzf /var/backup/home/home-backup.`/bin/date +%s`.tgz /home/
rsync --delay-updates -avz -e ssh /var/backup/home backupuser@'B':/var/backup/

–delay-updates is very important because if you don’t use it if ‘A’ crash when rsync is copying the .snar file (used to store incrementation information) you will miss it on ‘B’ and can’t retore tar file correctly.
-g only exist in GNU Tar. You may have to install it if you’re running *BSD. First check if you have a gtar binnary

Categories: Backup, Unix Tags: , ,

Misconfigured Perl install path

December 17th, 2008 No comments

I got a strange problem on a server. CPAN was installing module outside of perl @INC, this is quite a nightmare.
I found what was wrong and it was quite easy to solve.
Run this command:

# perl -V:'install.*'

Look at installprivlib and installarchlib. If they don’t match @INC you can change them at the end of perl Config.pm (in tie %Config, ‘Config’, {..})

To find your Config.pm file type

# perl -MConfig -le 'print $INC{"Config.pm"};'

If you only use CPAN to install module I guess you can just change PREFIX in the CPAN Config.pm (but I haven’t tested this)

N.B.: If you can’t find installprivlib and installarchlib in tie %Config just add them.

Categories: Perl Tags: ,

Getting human reading time info from RRD files

December 17th, 2008 No comments

You have a .rrd file but you don’t remember how much time RRAs can store data ? all right, no problem.

First save this Perl code into a .pl file.

$step = $1 if (m/step = (\d+)/);
$rows = $1 if (m/rra.*\.rows = (\d+)/);
if (m/(.*)\.pdp_per_row = (\d+)/) {
$pdp = $2; $time = $step*$rows*$pdp;
if ($time > 31536000) { $time = sprintf("%.2f year",$time/31536000) }
elsif ($time > 86400) { $time = sprintf("%.2f days",$time/86400) }
elsif ($time > 3600) { $time = sprintf("%.2f hours",$time/3600) }
print "$1: $step*$rows*$pdp = $time\n";
}

Assuming the filename of the script is rrd_info.pl, run this Shell command:

rrdtool info file.rrd | perl -n rrd_info.pl

Here is an output example:

rra[0]: 300*864*1 = 3.00 days
rra[1]: 300*864*5 = 15.00 days
rra[2]: 300*702*25 = 60.94 days
rra[3]: 300*840*125 = 364.58 days
rra[4]: 300*840*625 = 4.99 year
rra[5]: 300*864*1 = 3.00 days
rra[6]: 300*864*5 = 15.00 days
rra[7]: 300*702*25 = 60.94 days
rra[8]: 300*840*125 = 364.58 days
rra[9]: 300*840*625 = 4.99 year
rra[10]: 300*864*1 = 3.00 days
rra[11]: 300*864*5 = 15.00 days
rra[12]: 300*702*25 = 60.94 days
rra[13]: 300*840*125 = 364.58 days
rra[14]: 300*840*625 = 4.99 year

The formula is update step time * numbers of rows in the rra * pdp_per_row

Categories: Perl, RRDTool Tags: , ,

Booting NetBSD on Mac (iMac DV+)

December 17th, 2008 No comments

This is what to use to boot NetBSD MacPPC CD on an iMac DV+ (at least):

boot cd:,ofwboot.xcf netbsd.macppc

or

boot cd:0,ofwboot.xcf netbsd.macppc

You may not need to put netbsd.macppc depending of the CD


To boot on HD try this:

boot cd:,ofwboot.xcf hd:/nebtsd

or

boot cd:,ofwboot.xcf hd:13/nebtsd

Where 13 is the number of the netbsd partition (usualy near 13)

Categories: NetBSD Tags: ,

PPP Over SSH

December 17th, 2008 No comments

You may want to establish a full IP connection to a remote host (or remote lan) but you may not have any VPN software on the remote host, or on your host.
There is a solution using SSH and PPP with the command

pppd pty 'ssh -x -t -e none user@server /usr/sbin/pppd passive noauth 9600' noauth 10.0.0.1:10.0.0.2

You have to use key authentication because the tty is redirected to pppd so you can’t be prompted for a password.
With this command, you can reach server at IP 10.0.0.1.
By playing with pppd and route tables you can extend the IP tunnel to the entire remote LAN.

This has been tested between Mac OS X and NetBSD but should work with any system.
It works but it’s very slow.

On some system you may have TTY problem, if it’s your case take a look to http://shinythings.com/pty-redir/ or http://www.ishiboo.com/~nirva/Projects/vpn/

Categories: Unix Tags: ,

Best hints for Mac OS X 10.3 (Panther)

December 17th, 2008 No comments
Categories: Mac OS X Tags: ,