There is a major issue with Ubuntu partly due to BulletproofX and the X window system, which requires the presence of a monitor for X window to start. If the monitor is switched off, or is not connected, then Ubuntu / X would not be able to detect it and thus would go into “low graphics mode”. So, basically your GDM would not start.

There are two scenarios in which the monitor is not connected / switched off

1. You wish to save energy and thus switch off the monitor, and then remotely reboot the machine
2. You have Ubuntu running as a Server and thus no dedicated monitor is connected to it and you remotely reboot the machine. Specially, if you have automatic login enabled along with Remote Desktop (vino/vnc) enabled, and rely on Network Manager to start your session (through Wifi perhaps). In this case, Ubuntu would go into low graphics mode, and your network manager would not be able to assign you an IP, and you have lost complete remote access to the machine. Isn’t it pathetic?

I am still not aware on how to take care of the first part, that is, monitor switched off intentionally by you.

However, if you are running Ubuntu as a server and have/need a GUI, then the solution is here. Thanks to nikgare / http://ubuntuforums.org/showpost.php?p=7681232&postcount=8 for the pointer.

1. Modify /etc/X11/xorg.conf and remove all content and add the following
   
   Section "Device"
   Identifier "Configured Video Device"
   Driver "vesa"
   EndSection

   Section "Monitor"
   Identifier "Generic Monitor"
   HorizSync 58-62
   VertRefresh 75-117
   EndSection

   Section "Screen"
Identifier "Default Screen"
Device "Configured Video Device"
Monitor "Generic Monitor"
DefaultDepth 24
SubSection "Display"
Depth 24
Modes "1024x768"
EndSubSection
EndSection


2. You also may want to disable Bulletproof-X by modify /etc/gdm/gdm.conf and commenting out the line
   
   #FailsafeXServer=/etc/gdm/failsafeXServer

Now reboot, and X would start without monitor plugged in.

I was very happy when I had got my new Core 2 Duo E8400 processor. I immediately installed Ubuntu Jaunty 9.04 64Bit on it. Everything I wanted to use worked, except Skype.  After a lot of research, I did manage to install Skype, but the Video always gave a green screen, and crashed when I started the video. Here’s how to make it work successfully.

1. Download the Skype for 64Bit Ubuntu OS. It is not a true 64Bit version, but manages the dependencies well. http://www.skype.com/go/getskype-linux-ubuntu-amd64
2. Sign In to Skype and see all works as per requirements. You may need to go to Skype Options and fiddle with the Audio settings and select your audio card instead of pulseaudio.
3. Through Synaptic, install lib32v4l. Through console, you can do apt-get install lib32v4l
4. Download skype.start script (right click and select Save Link As) and save it in the /usr/bin directory. You may need root privileges for that. Otherwise save it on your Desktop, and use it to run Skype with Video compatibility. The privileges of the script need to be made executable. Through console you can do chmod 755 /path/to/skype.start and through GUI you can right click on the file, select Properties and under the Permissions tab, select “Allow executing file as program”. Remember, you need to start skype.start and not skype for the video to work.

The new version of Skype probably will fix the issues. There has been a long thread on Skype Forums regarding Linux this where the developers have promised that it will be out “Real Soon Now(tm)”. However, a month and a half has passed. Regardless, the solution works for voice and video both, and I’m happy using it.

And yes, if you need the content of the skype.start script, here it is

#!/bin/bash
LD_PRELOAD=/usr/lib32/libv4l/v4l1compat.so /usr/bin/skype

Have fun Skyping!

I have been  using KVM for virtualization for quite some time now, probably more than a year. Though the initial use was on the servers which were rarely shutdown, now I’ve started using it in my home envir0nment where frequent restarts or shutdowns are there. The challenge that I faced was to automatically shutdown the KVM virtual machines when the system was shutdown or restarted.

Thanks to Joern at linux-kvm.com, I was able to setup a script which does an auto shutdown of the Linux (basically any VM which has ssh available) machines. I am using it on Ubuntu 9.04 (Jaunty Jackalope) 64 Bit Desktop edition.

Steps

1. Add the host file entries
2. Make sure you are able to do ssh without entering any password from root into the VMs
3. Put the script into the /etc/init.d directory.
4. Test the script before rebooting the system
5. Reboot the system to check if it worked

The Host File Entries

The VM should  have a static IP, and not through DHCP. The VM name should be added into the /etc/hosts file. If you have a VM named Crayon with IP Address 172.16.0.5, then add the entry in the host file like the following

172.16.0.5            Crayon

This is important as I’m doing it through ssh, so I need to ssh into the domain by its name.

Doing ssh into the VMs through root without password

This is out of the scope of this document. You can look at the “Passwordless ssh” post by Adam Monsen on linuxconfig.org

Just make sure, through the root login you are able to ssh into your VM without entering the password. Try it a few times.  You may need to set the root password through sudo passwd on your Ubuntu machine.

The Script

I have modified the script provided by Joern. I used ssh instead of “virsh shutdown domain” because “virsh shutdown” didn’t work at all for the Ubuntu 8.04 Server VM that I had, and for Ubuntu 9.04 Desktop VM it opened up the dialog where it asks what to do (Shutdown/Suspend/Restart etc.). It didn’t suit me well, thus I used the ssh method

Put the following script into /etc/init.d/shutdownvm

#!/bin/bash

# Original Author :
#    Joern http://www.linux-kvm.com/content/stop-script-running-vms-using-virsh
#
# Modified by : Vivek Kapoor http://exain.com
# Date: 22 May 2009

# Parameters you can modify :: START

TIMEOUT=90
LISTFILE=/root/runvm.lst
LOGFILE=/var/log/kvmshutdown.log
# Parameters you can modify :: STOP

PS=/bin/ps
SSH=/usr/bin/ssh
GREP=/bin/grep
CUT=/usr/bin/cut
VIRSH=/usr/bin/virsh
TR=/usr/bin/tr
CAT=/bin/cat
DATE=/bin/date

# Function to shutdown the virtual machine
kvmshutdown () {
COUNT=0
PID=$($PS ax|$GREP $1|$GREP kvm|$CUT -c 1-6)

echo kvmshutdown \: Shutting down $1 with pid $PID

#$VIRSH shutdown $1
$($SSH root@$1 halt)

while [ "$COUNT" -lt "$TIMEOUT" ]
do
 $PS --pid $PID
 if [ "$?" -eq "1" ]
 then
 return 0
 else
 sleep 5
 COUNT=$(($COUNT+5))
 fi
done

echo kvmshutdown \: Timeout happened. Destroying VM $1

$VIRSH destroy $1

return 1

}

# The program begins here

$VIRSH list 2>/dev/null|$GREP running|$TR -s \ |$CUT -f3 -d\  > $LISTFILE

VMN=`$CAT $LISTFILE`

for vm in $VMN
do
 echo "$vm" is running
 kvmshutdown "$vm"
 if [ "$?" -eq "0" ]
 then
 echo VM "$vm" normally shutdown
 echo `$DATE +%Y-%m-%d\ %H:%M:%S` VM $vm normally shutdown >> $LOGFILE
 else
 echo VM "$vm" destroyed !
 echo `$DATE +%Y-%m-%d\ %H:%M:%S` VM $vm destroyed >> $LOGFILE
 fi;
done

Now for some more activities so that it initializes on halt and reboot

1. chmod 755 /etc/init.d/shutdownvm
2. cd /etc/rc0.d
   ln -s ../init.d/shutdownvm K18shutdownvm
3. cd /etc/rc6.d
   ln -s ../init.d/shutdownvm K18shutdownvm

I am using K18 as the kvm is having K20 value, and the above script should execute before kvm.

The beauty of the script by Joern is, it keeps on checking if the process has shutdown. So it will not wait for the complete Timeout value – in case the machine is shutdown earlier, it will proceed further. This I found to be great.

Testing the scripts

You may want to comment out “virsh destroy $1” in the script while you are testing it.

Using root, run the script /etc/init.d/shutdownvm

Keep on doing it as long as you don’t get the desired results. You probably would need to start your VMs a number of times to test it completely.

Reboot the system

Once you are satisfied that it is working as per your requirements, reboot the system. Prior to that you may want to uncomment the “virsh destroy $1” line in the script. While the system is rebooting, quickly try to shift to terminal 7 by pressing ALT+7. You should see the virtual machines being shutdown from there. If it doesn’t work successfully, you may need to do some more testing – probably the ssh connections are not getting established correctly as it is asking for password.

Probably a bug in Google’s search engine (http://www.google.co.in), all the searches showed the message “This site may harm your computer.” today, 31 January 2009 20:22 Hrs IST (GMT +0530). I thought, one of the rare things like this should be on a blog

Google India - All Sites are Harmful for your Computer

Few weeks back I purchased a Fujitsu U1010 (U810 in US market). It ran Windows Vista. Though I have been using Ubuntu for all my computing needs, I thought let the UMPC run Windows much like my Nokia runs Symbian. However, in a matter of few days I got fedup with Vista and decided to install Ubuntu 8.04 – there were challenges, and fortunately with the help of some good souls on internet, I overcame majority of them. Now the UMPC is running full-time Ubuntu Linux.

Disclaimer

The steps mentioned below may not be suitable from security point of view. Please use them at your own discretion. I take no responsibility for anything done to your machine by following the instructions below.

Tasks covered

1. Installation of Operating System
2. Touchscreen Configuration using evtouch – Not perfect though
3. Switching On/Off of Keyboard Lights/LEDs/Headlights
4. Rotating the screen
5. Associating the keys / , // , and Screen Rotate on the panel with their desired functions, that is, switching on keyboard lights, running handwriting recognition application and rotating the screen.
6. Using the Webcam – still painful
7. Putting the system into suspend and recovering from it successfully.

Much Thanks to

1. Julian Brown – http://panic.cs-bristol.org.uk/~jules/fujitsu-u810-debian-install-notes.html – Without him, the most important feature for me, the Keyboard lights, would not have worked.
2. Nathan Eckenrode (n8k99) and LordOfThePigs – http://so.nacreo.us/2007/12/touch-me-i-sick.html – For touchscreen configuration and a shell script for screen rotation
3. zmiq2 – http://www.umpcportal.com/modules/newbb/viewtopic.php?topic_id=2202&forum=16&post_id=16973 – For an excellent tip on Suspend and Resume.

So, we begin

I chose Ubuntu 8.04. Why? because it is a Long Term Support release, having a kernel which supports keyboard lights and was a stable release at the time of preparing this document. Intrepid Ibex/8.10 would have come at end of October 2008, which was too late for me.

Steps

1. Download the Desktop ISO from http://www.ubuntu.com/getubuntu/download
2. Download unetbootin – http://unetbootin.sourceforge.net/
3. Take a USB Pen Drive or SD Card.
4. Using unetbootin, select the ISO and copy it to the USB drive/SD Card. Preferably the Pen drive/SD Card shall be formatted in FAT32/FAT16 (the default format in which they come).
5. Once done, reboot the UMPC. While booting press Enter and select ‘Boot Menu’. If it doesn’t show the USB Drive, then it would not be enabled in the BIOS. Go into the BIOS and enable the USB Drive.
6. Booting from the SD Card slot would not work, but SD Card connected via a USB Card Reader would work.
7. Once you’re able to boot, just do the usual Ubuntu Installation.
8. Graphic card, audio, wifi, card reader and bluetooth work out of the box.

Touchscreen works, but not as desired. The sensitivity is very low and somethings happens to it when power adapter is connected. You would not be able to use it smoothly, and forget handwriting recognition, though we’ll install the required applications.

1. In the BIOS set the mode as touch screen instead of tablet. Please note, if you plan to use Windows alongside Linux, then setting this mode will prevent you from using handwriting recognition in Windows.
2. Install the evtouch driver through synaptic.
3. Take a backup of /etc/X11/xorg.conf file and replace it with the one available here – xorg.conf
4. Restart X by logging out and pressing CTRL+ALT+Backspace
5. Now the touchscreen should be working. It will work better if the UMPC is not connected to the power adapter.
6. Install cellwriter through Synaptic. It may require little bit of training before you can start using it.
7. Thank you Nathan Eckenrode and LordOfThePigs (link)

I had also tested the u810_tablet as was mentioned by Julian Brown here, but it didn’t used to work after doing suspend, and also after screen rotation. The above method seemed to be more reliable.

This was the most important thing for me and I learnt a lot about kernels while attempting this. And it works wonderfully.

The basics

1. Julian Brown has written a patch for the fujitsu-laptop module in the kernel. However, as of writing this section, that patch has not been incorporated into the mainstream kernel which is targeted more towards the Fujitsu P and T series.
2. The said patch enables the keyboard light functionality and a small application which toggles the light on and off.
3. I will provide pre-compiled kernel module for those not too technically inclined, and instructions to compile it yourself for your kernel.

Please note: I am using Ubuntu 8.04 without the backports repository enabled. The kernel versions thus would be limited to what’s available in the regular repository. You can set the repositories via synaptic.

Pre-compiled Binaries

1. For Kernel 2.6.24-19-generic. You can check the kernel version by opening a terminal and typing uname -r
2. Download the file fujitsu-laptop.ko and put it in the directory /lib/modules/2.6.24-19-generic/kernel/drivers/misc – replace the one available, it doesn’t do much anyway.
3. Download the toggle-kbd-light application and put it in /usr/bin directory. Make it executable and set suid bit by chown root.root /usr/bin/toggle-kbd-light and chmod 6755 /usr/bin/toggle-kbd-light
4. Move to section ‘Adding the Modules’ to start using it.

Compiling it from source

1. You would need kernel-headers package for your kernel which you can download from synaptic. I think it comes by default.
2. Install build-essential package from synaptic.
3. Create a directory called fujitsu and copy the file fujitsu-laptop.c in it.
4. In the same fujitsu directory, create a file called Makefile and put the following content in it

obj-m := fujitsu-laptop.o
KDIR := /lib/modules/`uname -r`/build/
PWD := $(shell pwd)
default:
$(MAKE) -C $(KDIR) SUBDIRS=$(PWD) modules

1. Now in the terminal go into the fujitsu directory where fujitsu-laptop.c and Makefile are present, and type make
2. It will create a fujitsu-laptop.ko file amongst others which you can transfer to /lib/modules/{your kernel version}/kernel/drives/misc
3. Please note that you have to do it everytime when you update the kernel.
4. Download the toggle-kbd-light application and put it in /usr/bin directory. Make it executable and set suid bit by chown root.root /usr/bin/toggle-kbd-light and chmod 6755 /usr/bin/toggle-kbd-light
5. Much thanks to Julian Brown. You can find the source and patches available here.

Adding the modules

1. The modules are not loaded by default. Add the following in /etc/modules
   led-class
   fujitsu-laptop
2. You can now reboot the system. Once the system has been rebooted, type toggle-kbd-light to switch the light on and off.

The easiest thing to do. It is enabled through xrandr command.

1. xrandr -o left # For left
2. xrandr -o right # For right
3. xrandr -o normal # For the normal screen
4. xrandr -o inverted # For inverted screen

You can use the script toggle-rotate and put it in /usr/bin directory. Make it executable by chmod 755 /usr/bin/toggle-rotate

Now whenever you call toggle-rotate, it will take the current screen orientation and rotate it to the next available orientation.

Tip: To make it easier to do the rotation specially when you know which orientation you require, create a few shortcuts in your gnome-panel (screenshot here) which have the commands xrandr mentioned (as above).

1. Through synaptic install cellwriter application which will used for handwriting recognition. You may not be able to use it successfully though and need to spend some time on it.
2. To detect the keys, the module fsc_btns needs to be loaded. Unfortunately I am not able to recall how I made it functional, but here are few hints
   1. Visit http://fjbtndrv.wiki.sourceforge.net/ and look at the Installation manual available there.
   2. Visit http://fjbtndrv.wiki.sourceforge.net/packages and look at the options for Ubuntu Hardy. I tried following the instructions, but the page maintained at http://home.versanet.de/~khnz15 does not exist anymore
3. Once you are able to load fsc_btns module, then the process is comparatively easy. Create a file .xbindkeysrc in your home directory and put the following in there.

#Lights
“toggle-kbd-light”
m:0×0 + c:105

#Screen Rotation
“toggle-rotate”
m:0×0 + c:203

#Handwriting Recognition
“cellwriter”
m:0×0 + c:99

4. Now run xbindkeys and check that clicking on the / button shall switch on the keyboard headlights, the // button shall open up the cellwriter application and the Screen Rotate key shall rotate the screen.
5. The Window (Super Key) that is there on your keyboard is anyways useless, so let us convert that to a TAB key which shall make the task of working on your system a bit easier since you would not have to press Fn+Space to use a TAB.
6. Create a file .xmodmaprc in your home directory and put the following line in it

keysym Super_L = Tab

7. Run xmodmap and now you can use the Super Key as the TAB key. However, I am not sure why, but this combination does not work for ALT+TAB
8. Now let us put all of this in the system startup so that they execute every time the system starts.
   1. Go to Systems -> Preferences -> Sessions
   2. Under the startup applications, click Add – you need to do this thrice since there are three applications that we will initialize. Under the command, enter

   cellwriter
   xbindkeys
   xmodmap ~/.xmodmaprc

Logout and log back in and it should be functioning.

Though I was able to use the webcam successfully, but the changes don’t persist across reboots. The image appears upside down, and you have to follow the steps exactly as I mention below to use it as desired.

1. The webcam drivers that come by default don’t function too well. It is a Ricoh webcam. You need to download and compile the drivers from http://wiki.mediati.org/Installation
2. Certainly, in the above website the packages are available for Hardy. You can use them too, but they are available for Kernel 2.6.24-21 which we are not using. It is available in the backports repository. There is no problem using them, but then you need to compile the keyboard lights (fujitsu-laptop) module on your own since I am not using the backports repository.
3. Install subversion from synaptic.
4. On a terminal, type svn co http://svn.mediati.org/svn/r5u870/trunk r5u870
5. After the process is done, you will see a r5u870 directory. Go inside that directory.
6. Type make and make install
7. This shall create the driver and you can now load it through modprobe r5u870
8. Add the following in /etc/modules

r5u870
uvcvideo

9. Reboot the system. The drivers should be loaded automatically now. You can check that by typing lsmod | grep r5
10. Install xawtv through synaptic.
11. Run xawtv – it shall open up a pop-up. Click OK. Now right click on the xawtv window, it will display a window. Do a Vertical Flip and it shall immediately display your picture via the webcam and is in the correct orientation.
12. Now you can close xawtv. The webcam will now function in Ekiga and Skype without problems. I was randomly able to make it work in gnome’s Cheese also.
13. Please note that if you put the system to suspend, then you will not get the option of flipping the screen vertically, which would lead to an inverted image. It shall be done before the system is put to suspend, that is, on a fresh reboot.

Much thanks to zmiq2 regarding this.

1. In the file /etc/default/acpi-support, search for the line starting with ‘MODULES’ and replace it with

MODULES=”ath_pci r5u870 uvcvideo videodev v4l1_compat v4l2_common”

2. In the file /usr/lib/pm-utils/default, add the following

SUSPEND_MODULES=”ath_pci ath_rate_sample ath_hal r5u870 uvcvideo videodev v4l1_c
ompat v4l2_common”

3. Now close the screen of the UMPC and it shall go into suspend. After you open the screen, the PC shall return from suspend and ask for the password.
4. It has been working wonderfully till now and I have kept it on suspend for more than 10 days without a reboot, and no problems. Everything works, including the Video.

1. Making the internal microphone work.
2. Making the Fn, the two arrow keys, and the CTRL+ALT+DEL key work.
3. Make the Fingerprint Scanner work.

That is all. It is serving all the desired purpose for me till now. Everything is working absolutely fine and since I put the system to suspend only, the webcam issue doesn’t trouble me much. Hope, by reading this article, you would go through comparatively less agony.

I was facing a lot of issues doing LDAP lookups on my company’s LDAP server. The searches were happening but they were very very slow, so most of the times while replying to an e-mail and selecting addresses, the Thunderbird application just hung.

I am using Ubuntu 7.10 with Thunderbird 2.0.0.14 in my office which has an Active Directory Setup. The comment as posted on http://www.linux.com/articles/114010 with the comment permalink being http://www.linux.com/?module=comments&func=display&cid=1147932

The solution for me was, just using Port 3268 instead of Port 389 which is common for LDAP. I am a happy man now

I have compiled a small list of locations which I happened to visit after purchasing my GPS device. Thought would post it online – maybe it can be of some help.

Converting Degrees to Latitude Longitude

INDIA is North East (NE)

1. The whole units of degrees will remain the same (i.e. in 121.135° longitude, start with 121°).
2. Multiply the decimal by 60 (i.e. .135 * 60 = 8.1).
3. The whole number becomes the minutes (8′).
4. Take the remaining decimal and multiply by 60. (i.e. .1 * 60 = 6).
5. The resulting number becomes the seconds (6″). Seconds can remain as a decimal.
6. Take your three sets of numbers and put them together, using the symbols for degrees (°), minutes (‘), and seconds (“) (i.e. 121°8′6″ longitude)

You can do it fast by visiting http://www.fcc.gov/mb/audio/bickel/DDDMMSS-decimal.html

Convert Degree, Minutes, Seconds to Latitude and Longitude

1. latitude and longitude in a decimal format: 42.1361
2. latitude and longitude in degree, minute, second format: 42deg, 08min, 10sec
3. To convert coordinates from degrees, minutes, seconds format to decimal format, use this easy formula:

degrees + (minutes/60) + (seconds/3600)

The example coordinate above would be calculated as:

42 + (8/60) + (10/3600) = 42.1361
or
42 + (.1333) + (.0028 ) = 42.1361

A selected few would want to delve into this. No, this post is not about how the houses of the future would be. For that you may visit http://news.bbc.co.uk/2/hi/europe/7348940.stm

This post is about how you, as a Linux enthusiast, can take a combination of all wonderful applications available, and leverage them to create something which defines what you are. This is no howto, and I just skim the surface as I myself am trying to figure out how to work out all this.

Following is where I am coming from : You wake up in the morning, ask for the time and weather outside. The system responds back. You give voice commands to take backups of the system, check your e-mails, record voice messages, play music, make phone calls, and watch your favorite TV channels. Then you leave for office. The home is now under surveillance mode. Any activity which happens behind your back is emailed to you instantly as a video. The primary power fails and the system switches to the secondary power. There are solar cells on top of your building which are charging the batteries. You come back with some friends of yours. You put the system in silent mode and record all activity that happens. Its night time now and you go to bed. The infra-red cameras keep a watch on the house.

Next day is a weekend, and you plan a trip outside. You get ready your stuff and take out the mini-computer which is connected with your base station wirelessly. The mini computer has GPS inbuilt which keeps a track of where you are and communicates it back to the base station. Your backpack has solar cells which keep charging the mini-computer. You are continuously updated of what’s happening back at home.

All of this is possible, though with some expenditure on the hardware front. Here’s what I think you would need

Hardware

1. Base Station : A laptop/battery powered device for the base station. A desktop would also do but power consumption may be a concern. A wifi router, UPS, a set of speakers and a good microphone and internet connection would be necessary.
2. Power Generation : Solar Cells, and a battery system which they can charge
3. Electrical Devices Control : I’ve heard about X10 and http://www.smarthome.com – am not sure as none of them is available here (in India).
4. Mini-Computer : A Gumstix motherboard with USB, Flash Memory, Wifi, GPS and LCD screen.
5. Monitoring : A infra-red webcam. A wireless one would be better, but USB one would also do.
6. Internet Connection : A USB based internet connection device which allows you to connect from anywhere. Shall be compatible with Linux & wvdial.

Software

1. Speech Recognition : HTK, Julius/Julian
2. Text to Speech : Festival, with MBROLA
3. Motion Detection : motion
4. Scripting Language : PHP and bash – this will help in defining and acting upon commands
5. Home Automation : Mister House
6. Others : wvdial for net connection, audacity for audio recording, mail tools for sending out e-mails, Apache for providing a connectivity to base station

The intention is to create a home of the future, today. I would keep on adding stuff here as and when I find time; the first one probably would be about Speech Recognition. I personally believe that the technology is available, and if used effectively, it can be really exciting.

If you want to have unlimited space like what google.com is doing with gmail, then here’s what you need to do. I used Ubuntu Feisty Fawn and Gutsy Gibbon to test out the solution.

1. Setup and understand the usage of a LVM
2. Setup iSCSI or AoE for network devices so that network storage can be used easily
3. Setup RAID so that redundancy can be provided to LVM Physical Volumes

Important: Following steps may corrupt your data or partitions. Use them at your own risk. Also, I have listed down the steps as I went ahead doing them. They may not be accurate in your environment or all scenarios may not be covered. I would recommend going through the entire article before attempting anything.

Assumptions

1. You are comfortable using Linux and fdisk
2. You have backed up your data so that in case something goes wrong, nothing is lost
3. You can connect to network and have access to more than 1 Linux machine to do the testing

This article is not complete and does not include the iSCSI and AoE related information as of now. I am working on it and will update it soon. However, you may continue on this page and learn about the LVM

Setup a LVM

Much help from http://linuxhelp.blogspot.com/2005/04/creating-lvm-in-linux.html

Quick Steps

1. Identify the partitions which are to be used. These are referred to as physical volumes (pv). The steps are
   1. Identify the partitions such as /dev/hda1, /dev/sda1 etc.
   2. Set the correct partition type of Linux LVM, which is 8e
   3. Format them using pvcreate
2. Create a Volume Group (vg) which consists of the partitions identified above. THis is the place where you would add/remove partitions to increase/decrease space.
3. Creating the Logical Volumes (lv) which are partitions that you can format (using ext3 or gfs etc.).
4. Use the partitions created as per your requirements. Add them into /etc/fstab if necessary.

So, diagrammatically it is something like the following

Analysis

So the analysis is

1. We can add/remove partitions in a volume group, that too on the fly.
2. A logical volume is created from the volume group. Its size is defined in extents.
   1. So, whenever we want, we can increase the size, or decrease the size as per requirements.
   2. If there are two partitions, /backup and /recordings and we have space available in /recordings, then we can reduce the size of /recordings and add it to /backup

Hands On LVM

Following is what I did for experimentation

1. Install lvm2 through synaptic

2. Create the partitions and tag them as LVM. Three partitions created – /dev/sda5, /dev/sda6 and /dev/sda7, 500 MB each. Tagged them as LVM (code 8e available in fdisk)

3. Reboot the machine as the lvm2 needs to be initialized, and since changes in the partitions were done, they also need to be initialized

4. Ran the command pvcreate

   1. pvcreate /dev/sda5 /dev/sda6 /dev/sda7

5. Ran vgscan to check if any Volume Groups were present. None were

6. Created a Volume Group using vgcreate for a total size of 1 GB.

   1. vgcreate my_disks /dev/sda5 /dev/sda6 # For Extent Size of 4 MB which means 256 GB of maximum space supported by that VG
   2. I could have also used vgcreate -s 16 my_disks /dev/sda5 /dev/sda6 for Extent Size of 16 MB so that I can use a maximum space of 1 TB. Since this is an experiment, no such need is necessary.

7. Ran vgdisplay to view the details of the my_disks volume group

8. Created a logical volume using
   1. lvcreate -l 75 -n vol1 my_disks
   2. - l 75 means 75 extents. Considering the size of 4 MB per extent, the total size of the logical volume would be 4×75 = 300 MB
   3. vol1 is the name of the logical volume. This can now be formatted.
   4. Verify it using lvdisplay

9. Created another logical volume vol2 of 75 extents.

10. Formatted both the volumes in ext3. I am using ext3 only because I have experience using it, and it is possible to resize it while still being mounted (online resize as of now is supported only for growing a partition, not shrinking)
    1. mkfs.ext3 /dev/my_disks/vol1
    2. mkfs.ext3 /dev/my_disks/vol2

11. Mounted the partitions in /lvm

    1. mkdir /lvm
    2. mkdir /lvm/recordings /lvm/data
    3. mount /dev/my_disks/vol1 /lvm/data
    4. mount /dev/my_disks/vol2 /lvm/recordings

12. Added the following entries in /etc/fstab so that they auto mount on reboot

    1. /dev/my_disks/vol1 /lvm/data ext3 defaults 0 3
    2. /dev/my_disks/vol2 /lvm/recordings ext3 defaults 0 3

13. Rebooted the system and all was fine

Important executables

1. pvcreate

2. pvmove

3. pvdisplay

4. vgdisplay

5. vgextend

6. vgreduce

7. lvdisplay

8. lvresize

9. resize2fs

Test Cases for LVM

A. /lvm/data needs 400 MB of space instead of existing 300 MB which can be taken from the Volume Group available

1. Extents required : 100
2. Commands
   1. lvresize -l 100 /dev/my_disks/vol1
   2. resize2fs /dev/my_disks/vol1 400M

It did online resizing (resize2fs) only because we were growing the partition and it was ext3 on Kernel 2.6.

B. /lvm/data needs to be resized to 800 MB (200 extents), but space available in volume group (my_disks) is only 268 MB (67 extents)

1. Extents required : 200
2. Extents available : 67
3. Commands
   1. lvresize -l 200 /dev/my_disks/vol1

It shows an error message of insufficient extents.

C. Free up extents in vol2 so that they can be used in vol1 (for Case B.)

1. Extents to be freed : 33
2. Current Extents : 75 (300 MB)
3. Commands
   1. umount /lvm/recordings
   2. e2fsck -f /dev/my_disks/vol2
   3. resize2fs /dev/my_disks/vol2 168M
   4. lvresize -l 42 /dev/my_disks/vol2 # This gave a warning which I ignored
   5. vgdisplay # 100 Extents available
   6. Run Case B. again – It works!!

The command e2fsck needs to be run before resize2fs, otherwise resize2fs would give an error.

D. The Logical Volume (vol2) of 42 extents has data upto 120 MB (30 extents) and has to be resized to 28 extents

1. Commands
   1. mount /lvm/recordings
   2. cd /lvm/recordings
   3. Created a 120 MB file using dd if=/dev/zero of=myfile bs=12280 count=1024
   4. echo "hi" >> myfile
   5. md5sum myfile # To verify what happens if data is reduced. This helps verify corruption if it occurs
   6. umount /lvm/recordings
   7. e2fsck -f /dev/my_disks/vol2
   8. resize2fs /dev/my_disks/vol2 112M

This reports an error. So it will not resize when there is data

Try again, but this time increase it to 140 MB (35 extents)

1. lvresize -L 140M /dev/my_disks/vol2
2. resize2fs /dev/my_disks/vol2 140M
3. mount /lvm/recordings

It works!. Doing md5sum /lvm/recordings/myfile is accurate.

E. vgdisplay has 242 extents of which only 7 are free. Now add the /dev/sda7 partition without unmounting anything

1. Commands
   1. vgextend my_disks /dev/sda7

And it is extended without unmounting anything. Total Extents available : 363

F. Hard Disk /dev/sda6 has gone bad and has to be taken out from service. So it’s data has to be moved

1. Commands
   1. modprobe dm-mirror # This has to be done as dm-mirror module is not loaded by default. You may add it in /etc/modules
   2. pvmove /dev/sda6

pvmove is used to move the data of a physical volume residing inside a volume group. This helps removing the physical volume easily.

Now removing the pv /dev/sda6 from the volume group

1. vgreduce my_disks /dev/sda6

And it works too! Doing a md5sum also is successful, so no data got corrupted.

G. System got corrupted and is now unbootable. Now the data needs to be retrieved from the logical volumes.

1. Restart the machine boot using a Ubuntu Gutsy Gibbon (7.10) Alternate Install CD. The Alternate install CD has a rescue mode available.
2. Start the rescue mode

3. The volume groups and logical volumes (/dev/my_disks, /dev/my_disks/vol1, and /dev/my_disks/vol2) were available and I was able to mount them successfully. md5sum also returned correct results.

It seems that Ubuntu Rescue detects the LVM volumes and initializes them so that they’re usable.

Observations

• Over a Wifi network using iscsi was painfully slow if I tried using pvmove. So, if there was an iscsi target /dev/sdb1 which was on a different machine (machine B), and I wanted to remove /dev/sdb1 from my LVM on machine A, then I used “pvmove /dev/sdb1″. So it moved all the information available in /dev/sdb1 to extents available on my LVM on machine A which took ages (9m14sec for 50 MB of data). However, reversing the scene, wherein /dev/sdb1 was part of my volume group, and I wanted to free up a partition on my machine A (say /dev/sda8), then “pvmove /dev/sda8″ worked flawlessly and was fast. No such problems occured when using ethernet.
• AoE has better performance as compared to iSCSI in cases of pvmove. I was able to transfer 836 MB of data in approx 1.35 minutes as compared to 2.19 minutes for iSCSI.
• iSCSI seems to be more reliable than AoE. Using AoE my system was hung a few times, and there were other issues which didn’t seem fit to be in production where anyone can be asked to manage a machine. Case in question is, it stuck when doing vgscan/lvdisplay and i was able to view the listing only after I somehow managed to rmmod aoe – probably I’d mounted the partition back (using vblade) and then only it started working. But another issue was, though I was able to do ‘pvmove /dev/sda8′ (of machine A) successfully with the network traffic appearing on machine B (sdb1), but when I tried to do ‘pvmove /dev/sdb1′, it gave some ioctl related errors and within 16 seconds said that 100% of the data was transferred. A md5sum was also successful! I don’t know what happened here.

Errors Encountered

• /dev/sdc1: read failed after 0 of 4096 at 0: Input/output error

This had occured because I had made changes in fdisk to the primary hard disk. So it was not syncronized properly. Once reboot was done, all was fine.

One more problem occured – if you restart open-iscsi then open-iscsi changes the device name from /dev/sdb1 to /dev/sdc1 which causes the above mentioned problem. This needs to be worked upon.