Home Uniquely NZ Travel Howto Pauline Small Firms Search
Diary of System and Website Development
Part 20 (October 2011 - March 2012)

Early October 2011

Ubuntu 11.10 Oneiric Ocelot with Unity Desktop LiveCD and LiveUSB

I have been periodically downloading the various Betas and finally the real Ubuntu 11.10 Oneiric Ocelot and running them to see if there were going to be any major problems. The LiveUSB produced using the Ubuntu application now has the ability to have a persistent file system and also to be updated and programs loaded for testing. There seems to be some delay requiring a restart before everything shows up but it certainly makes testing easier. There seemed to be no show stoppers like lack of Wifi or other crucial support so I went ahead and did a Distribution Upgrade on my MSI Wind U100 and have updated Ubuntu Unity - Evolution or Revolution and The MSI Wind U100 Netbook for Global Communications and Computing directly so I will say ne more here on the end results other than there were no showstoppers. It however set me think about the philosophy of upgrading, of the backups prior to upgrading and the way of spliting applications and data within a machine and sharing between machines.

19th October 2011

 

Files that have to be configured in systems area. Colour coding will reflect whether a Distro Upgrade resets them.

  1. sudo gedit /etc/fstab # not changed in Distribution Upgrade # - see www.pcurtis.com/ubuntu-files.htm#permanent_mount
  2. sudo gedit /etc/apparmor.d/local/usr.bin.firefox #not changed in Distribution Upgrade # see Fix to AppArmor to use a Firefox 4 or 5 profile on a separate drive/partition
  3. gksudo nautilus /usr/share/fonts/truetype # Changed in Distro upgrade see Adding Fonts in Linux so they are accessible from OpenOffice and Wine
  4. sudo gedit /etc/nsswitch.conf # not changed in Distribution Upgrade see Enabling Hostname Resolution under Ubuntu
  5. sudo gedit /etc/samba/smb.conf # not changed in Distribution Upgrade - also see Enabling Hostname Resolution under Ubuntu
  6. sudo gedit /etc/ssh/ssh_config # not changed in Distribution Upgrade see Secure access to a Remote File System using SSH and SSHFS (Advanced) and Using Public Key Authentication with ssh, sshfs and unison to avoid password requests
  7. sudo gedit /etc/ppp/options # Changed in Distro upgrade - needed for some mobile connections
  8. gksudo gedit /etc/init.d/bluetooth # See www.pcurtis.com/ubuntu-mobile.htm#rfcommcreate #Need to modify different file when using Oneiric
  9. sudo gedit etc/pm/sleep.d/99_bluetooth-restart # Can probably be removed in Oneiric - see www.pcurtis.com/ubuntu-unity.htm#blue-sleep
  10. gksudo gedit /etc/bluetooth/rfcomm.conf # not changed in Distribution Upgrade - need reduced with normal bluetooth telephone modems www.pcurtis.com/ubuntu-mobile.htm#rfcommcreate
  11. sudo gedit /usr/share/polkit-1/actions/org.freedesktop.upower.policy # Inhibit suspend and hibernate bug fix covered at www.pcurtis.com/ubuntu.htm#suspend
  12. sudo gedit /etc/modprobe.d/blacklist.conf # only for toshiba and Madwifi driver
  13. sudo gedit /etc/modprobe.d/blacklist-ath_pci.conf # only for toshiba and madwifi Driver
  14. gksudo nautilus /usr/share/icons/Humanity/apps/48 #not changed in Distribution Upgrade - only for special themes
  15. sudo gedit /etc/pm/config.d/unload_module # Only for MSI U100 Wind
  16. sudo gedit /etc/hdparm.conf # Only for MSI U100 Wind and not required
  17. sudo gedit /etc/sysctl.conf # Only for MSI U100 Wind and not required
  18. sudo gedit /etc/default/mouseemu # Changed in Distro upgrade

Other changes required

Change from Wine1.2 to Wine1.3 but keep to version 28

 

Is it ever worth changing to a non Long Term Support (LTS) version?

The pace of development of hardware means that each version is likely to support some of the latest hardware much better than earlier versions. This was true of Jaunty where the network manager was dramatically improved and supported many of the USB Mobile Broadband Sticks in an integrated manner. It also had a one year support which took it to the issue of the next LTS version.

The issue was much less clear when Ubuntu 9.10 Karmic Koala came out and one had to ask if the advantages are sufficient to merit an upgrade when a LTS version is due in 6 months. I did end up upgrading 3 of our 5 machines each for different reasons. The Toshiba Satellite Pro L20 lost a hard drive so a new install was required anyway and 9.10 had less problems with some of the hardware than 9.04 and was much better than 8.04. The HP Compaq DX2250 needs the most up to date versions of software as it is used for Video work and the Home Build desktop was in need of a reinstall as it had been progressively upgraded from 6.06 to 8.04 Hardy Heron and also needed more disk space and repartitioning of the disk drives. The MSI Winds were not upgraded as there were issues with Webcam support and also random Brightness variations. These seem to be a chip problem and the surprise is more that they work perfectly under Jaunty!

So what were the major changes in Lucid:

There are many changes under the hood (hal and pulse audio to name a couple) and some have had unexpected results on some systems - Karmic was an essential development step before the next LTS version Lucid but it does mean that you should check out carefully with a LiveCD version before upgrading. If you are happy with what you have then it is prudent to stay with it until you have had a chance to do some comprehensive web searches for peoples experiences with your own machine and also to read the Ubuntu Release notes with great care. That said you will find that the improvements are well worth getting used to a few changes.

Major changes expected in 12.04

Unity, Gnome 3 and Kernel version 3 on there way by 12.04

 

Updating Your Distribution

Backing up a Home Directory prior to a Distribution Update

Backup using a Standard Tar Archive

There is also a very powerful command line archiving tool round which many of the GUI tools are based which should work on most Linux Distributions. In many circumstances it is best to access this directly to backup your system. The resulting files can also be accessed (or created) by the archive manager accessed by right clicking on a .tgz .tar.gz or .tar.bz2 file. The following commands will back up your home folder /home/yourusername if you change pcurtis to yourusername - all other programs must be closed.

In a terminal type:

sudo tar cvpPzf "/media/WD Passport/mybackup.tgz" /home/pcurtis/ --exclude=/home/*/mybackup.tgz --exclude=/home/*/.gvfs

This is a single line if you cut and paste. The options cvpzf are: create archive, verbose mode (leave this out after the first time) , retain permissions, -P do not strip leading backslash, gzip archive and file output. Then follows the name of the file to be created, mybackup.tgz which in this example is on an external USB drive called 'WD Passport' - the backup name should include the date for easy reference. Next is the directory to back up, in this case /home/pcurtis). Next are the objects which need to be excluded - the most important of these is your back up file if it is in your /home area (so not needed in this case) or it would be recursive! It also excludes the folders (.gvfs) which is used dynamically by a file mounting system and is locked which stops tar from completing. The problems with files which are in use can be removed by creating another user and doing the backup from that user - overall that is a cleaner way to work.

The backup process is slow (15 mins plus) and the file over a gbyte for the simplest system. After it is complete the file should be moved to a safe location, preferably a DVD if you did not use an external device. If you want to do a higher compression method the command "tar cvpjf mybackup.tar.bz2" can be used in place of "tar cvpzf mybackup.tgz". This will use bzip2 to do the compressing - j option. This method will take longer but gives a smaller file.

You can access parts of the archive using the GUI Archive Manager by right clicking on the .tgz file - again slow on such a large archive. A backup is most useful if you can effectively restore your entire system or data. This can be again done by a few commands but do not try this for fun as it will overwrite the entire part of the Ubuntu file system we archived, thus restoring the older image that we took.The following assumes the backup image is in the root directory of an external drive called 'WD Passport' :

sudo tar xvpfz "/media/WD Passport/mybackup.tgz" -C /

The restoration uses the options - extract, verbose, retain permissions, from file and gzip. This will take a while because all your files will be overwritten with the versions from the image you previously backed up. The "-C / ensures that the directory is Changed to the root so the files are restore to the original locations.

If the old system is still present note that it only overwrites files, it does not deleted files from the old version which are no longer needed.

you want to delete the archive file then you will find it is owned by root so make sure you delete it in a terminal - if you use a root browser then it will go into a root Deleted Items which you can not easily empty so it takes up disk space for ever more. If this happens then read http://www.ubuntugeek.com/empty-ubuntu-gnome-trash-from-the-command-line.html and/or load the trash-cli command line trash package using the Synaptic Package Manager and type

sudo trash-empty

Updating your Distribution

The above backup of ones home folder(s) is all in preparation for a distribution upgrade. In theory this is possible without lose of any of your files or configuration - in practice I and others have had some problems especially if you have loaded any addition software via extra repositories or directly via .deb files as the automated update is very good but can not cover every contingency. You should also use the procedure for backing up your home folder if you intend to try the automated system in the Synaptic Package Manager then you have the choice of doing a full reinstall if it is not what you expect. Distribution upgrades only work for a single change of version or to the next Long Term Support version so if you have skipped versions you either need to do a succession of updates or a full system install.

The approach when you do a full install of the system varies a little depending on whether you have created or installed with your home folder /home on a dedicated partition. If you have a separate home partition you should still back it up as above but hopefully you will not need it as you can use the advanced option when the installer gets to the partitioning stage and edit the existing partitions so the mount points are as before and make sure you only format the root partition - this should leave your /home partition intact for when you restart with the new distribution which will find all your old settings for applications even if you have to reload them.

If you do not have a separate /home partition already this is the time to consider generating one and there are full instructions at Moving a Home Folder to a dedicated Partition (Expert level) on you can create one during the partitioning stage of the installation of the new Distribution to make life easy for the future and then use you backup from above to reload it.

My standard procedure splits into a number of parts which I use on all my machines:

  1. I periodically make a backup of my /home folder in a tar archive, as cover above, which should have all the data and configuration.
  2. I keep a record of all the programs, utilities and libraries (packages) I have loaded on all my machines and I have created a script to both load them and as a way of documenting for the future ie as every PPA or other repository is added and every package is loaded, I add it to the script for the future. You can have a look at my latest script for Lucid Lynx - loadstandardprograms-lucid or for Oneiric loadstandardprograms-oneiric
    • Note - Script files need to be made executable (Right click -> properties and tick box near bottom) and also need to be files with unix line endings - bash hates Windows files and the scripts do not run properly. If you ever get a file with Windows line endings which can happen when you download a script in your browser then create a new file under Linux and copy and paste the old contents into it.
  3. I keep a list of packages added which are separate of the PPA and repository system such as Picasa3 (there is now a development repository (PPA) which has it), and Truecrypt, and in some cases special drivers for Wifi, Scanners, Printers etc and keep all the downloads in a folder so I can access them easily - again it self documents.
  4. I likewise keep copies of all the extensions for firefox and thunderbird
  5. I make a copy of the latest version of every system file I have had to edit such as fstab, rfcomm.conf, ppp.conf, menu.list and bashrc etc so again it is self documenting.
  6. Any special set up procedures are documented on my web site, there are a number of examples on this page.

Now when I need to rebuild the system or come to do clean install of a new distribution like Lucid Lynx I can:

  1. Do a fresh install from CD or from a USB stick. On systems with the /home on a separate partition your home folder with all the configuration information for your desktop and programs should be left intact if you make sure you do not reformat that partition during the installation.
  2. Optional - Carry out any tuning of the Grub boot procedure - Grub2 on Lucid and higher is quite different and this is best left as it is or done at an early stage so you can start again. This is documented at Customising Grub2 - The new Bootloader used in Ubuntu Lucid Lynx rather than on this page as it is rather specialised.
  3. Install all the extra repositories and packages, in my case using the scripts described above.
  4. Install any extra packages such as Picasa and Truecrypt from the folder of .deb files and install scripts or from the latest copies downloaded from their web sites.
  5. Edit/replace the system files which were changed and check that drives mount and the boot is as you want.
  6. Carry out any special procedures for installing drivers, permanently mounting drives, using modems and mounting Truecrypt volumes without an administrator password etc.
  7. Extra steps needed if you do not have a separate home folder.
    1. Create an extra user with Administrative privileges (so it can sudo), change to the new user, rename your normal user's home folder and replace it with the home folder folder from the tar archive.
    2. Log back into your usual username.
    3. Delete the backup user home folder - use sudo in a terminal or use the proceedures above or you will end up with it root's trash.
  8. Set up File Sharing by System -> Administration -> Samba. Samba requires a reboot.
  9. Set up Printers by System -> Administration -> Printers -> Add printer and follow through the screens.
  10. Set up SSH - some system files need to be changed to allow host resolution to work correctly.

You should now be back to where you were.

I have used the same basic proceedure to change partition sizes, add a partition for the /home directory, upgrade from Hardy to Jaunty, Jaunty to Lucid, Lucid to Natty and even to move a complete 'user' from a netbook to a laptop all in one operation.

Files that have to be configured in systems area after an upgrade.

Files in black means that they will not need to be changed after a Distribution Upgrade, only after afresh install.

Files in Red need to also be changed after a Distribution Upgrade

 

  1. sudo gedit /etc/fstab # not changed in Distribution Upgrade # - see www.pcurtis.com/ubuntu-files.htm#permanent_mount
  2. sudo gedit /etc/apparmor.d/local/usr.bin.firefox #not changed in Distribution Upgrade # see Fix to AppArmor to use a Firefox 4 or 5 profile on a separate drive/partition
  3. gksudo nautilus /usr/share/fonts/truetype # Changed in Distro upgrade see Adding Fonts in Linux so they are accessible from OpenOffice and Wine
  4. sudo gedit /etc/nsswitch.conf # not changed in Distribution Upgrade see Enabling Hostname Resolution under Ubuntu
  5. sudo gedit /etc/samba/smb.conf # not changed in Distribution Upgrade - also see Enabling Hostname Resolution under Ubuntu
  6. sudo gedit /etc/ssh/ssh_config # not changed in Distribution Upgrade see Secure access to a Remote File System using SSH and SSHFS (Advanced) and Using Public Key Authentication with ssh, sshfs and unison to avoid password requests
  7. sudo gedit /etc/ppp/options # Changed in Distro upgrade - needed for some mobile connections
  8. gksudo gedit /etc/init.d/bluetooth # See www.pcurtis.com/ubuntu-mobile.htm#rfcommcreate #Need to modify different file when using Oneiric and higher
  9. sudo gedit etc/pm/sleep.d/99_bluetooth-restart # Can probably be removed in Oneiric+ - see www.pcurtis.com/ubuntu-unity.htm#blue-sleep
  10. gksudo gedit /etc/bluetooth/rfcomm.conf # not changed in Distribution Upgrade - need reduced with normal bluetooth telephone modems www.pcurtis.com/ubuntu-mobile.htm#rfcommcreate
  11. sudo gedit /usr/share/polkit-1/actions/org.freedesktop.upower.policy # Inhibit suspend and hibernate bug fix covered at ubuntu.htm#suspend
  12. sudo gedit /etc/modprobe.d/blacklist.conf # only for toshiba and Madwifi driver
  13. sudo gedit /etc/modprobe.d/blacklist-ath_pci.conf # only for toshiba and madwifi Driver
  14. gksudo nautilus /usr/share/icons/Humanity/apps/48 #not changed in Distribution Upgrade - only for special themes
  15. sudo gedit /etc/pm/config.d/unload_module # Only for MSI U100 Wind
  16. sudo gedit /etc/hdparm.conf # Only for MSI U100 Wind and not generally required
  17. sudo gedit /etc/sysctl.conf # Only for MSI U100 Wind and not generally required
  18. sudo gedit /etc/default/mouseemu # Changed in Distro upgrade

Specifics on partitioning when installing if you have a separate /home partition

Make sure you know which partitions are used for the various mount points before you start. Usually it is obvious from the sizes and file system types but if you are unsure then running blkid in a terminal is also a good way to find out – typical output (on Lucid) looks like:

pcurtis@vortex-ubuntu:~$ blkid
/dev/sda1: LABEL="WINRE" UUID="80AE-9D55" TYPE="vfat"
/dev/sda2: LABEL="WINDOWS" UUID="F6EC309DEC305A57" TYPE="ntfs"
/dev/sda5: LABEL="DATA" UUID="49D3-6030" TYPE="vfat"
/dev/sda6: UUID="8993fcc3-6a5a-4f94-9e6a-c77f5091c1ad" TYPE="swap"
/dev/sda7: UUID="50fca34f-45d2-4231-8663-71ba99fc756f" SEC_TYPE="ext2" TYPE="ext3"
/dev/sda8: UUID="e31b581a-85fa-4681-bf11-aebb9a9656cd" SEC_TYPE="ext2" TYPE="ext3"
/dev/sdb1: SEC_TYPE="msdos" UUID="3934-3935" TYPE="vfat"
pcurtis@vortex-ubuntu:~$ gedit /etc/fstab

combined with looking at your existing file system table (red bits of both) which is /etc/fstab which on my system looks like:

# /etc/fstab: static file system information.
#
# Use 'blkid -o value -s UUID' to print the universally unique identifier
# for a device; this may be used with UUID= as a more robust way to name
# devices that works even if disks are added and removed. See fstab(5).
#
# <file system> <mount point> <type> <options> <dump> <pass>
proc /proc proc nodev,noexec,nosuid 0 0
# / was on /dev/sda7 during installation
UUID=50fca34f-45d2-4231-8663-71ba99fc756f / ext3 errors=remount-ro 0 1
# /home was on /dev/sda8 during installation
UUID=e31b581a-85fa-4681-bf11-aebb9a9656cd /home ext3 defaults 0 2
# /media/DATA was on /dev/sda5 during installation
UUID=49D3-6030 /media/DATA vfat utf8,uid=pcurtis,umask=000,gid=46 0 1

# /windows was on /dev/sda2 during installation
UUID=F6EC309DEC305A57 /windows ntfs defaults,nls=utf8,umask=007,gid=46 0 0
# swap was on /dev/sda6 during installation
UUID=8993fcc3-6a5a-4f94-9e6a-c77f5091c1ad none swap sw 0 0
/dev/scd0 /media/floppy0 auto rw,user,noauto,exec,utf8 0 0

all should be clear!

Backing up and reloading the entire package list

This is an idea I got from the book Ubuntu Hacks (which is in my book list) as an alternative to my script LoadStandardPrograms . One copies the entire list of installed packages to a file and then loads them all, some 1600, back in - of course most of them will already be there but the system ignores them and just loads the extra 30 packages which should all be in my script with associated dependencies, perhaps 100 in total. It should work well for a reload of the same distribution or a minor upgrade. The terminal command to write the list of packages is:

sudo dpkg --get-selections | grep '[[:space:]]install$' | awk '{print $1}' > package_list
The operation is a nice example of piping throuagh a series of programs and redirection of the output to a file. grep selects the lines containing the string 'install' and awk is used to select and print only the first item on each line. awk, with its own self-contained language, is actually one of the most powerful data processing engines in existence — not only in Linux, but anywhere. It allows you to create short programs that read input files, sort data, process it, perform arithmetic on the input, and generate reports, among myriad other functions. To put it the simplest way possible, AWK is a programming-language tool used to manipulate text. When first created, AWK was designed to work in the text-processing arena, and the language is based on executing a series of instructions whenever a pattern is matched in the input data. The utility scans each line of a file, looking for patterns that match those given on the command line. If a match is found, it takes the next programming step. If no match is found, it then proceeds to the next line. This link is a good starting point http://www.oracle.com/technology/pub/articles/dulaney_awk.html . In fact the grep action could also be carried out in awk and when I have more time I might try that out.

The command to read the file back in is also a good example of the power of linux commands and piping output from one into another:

cat package_list | xargs sudo apt-get install

cat is used to send the list through a pipe to xargs which calls apt-get install with parameters made up from the input piped into it. It divides the input into chunks and calls apt-get install as many times as are required.

Whenever I see command lines such as these I look up the various commands using man commandname to see what they do. You will often find that people have piped together a series of simple ways of using the commands which they know and understand rather than save a a few characters by using many options with a single command.

Xubuntu

Understanding Drag and Drop behaviour in Thunar

Modifiers:

(Note how the mouse pointer changes)

This is true for both desktop and thunar, however each seems to default to its own liking.

Saving session on logout from Xubuntu

Certian ways of exiting give a check box so that the session which is running can be saved. This can give a confusing situation if you have saved a session and you may have to close everything then save that session to clean up. This menu is reached from the Appindicator which looks like a little door.

Extra Appindicators

Load xfce4-goodies using the Synaptic Package Manager

Useful Appindicators I have added.

I have moved the workspace switcher to the lower panel

SSH in Thunar

This mounts the filesystem using fssh

Disabling Suspend and Hibernate in Ubuntu Linux

I had problems when testing suspend and hibernate on the Portege which resulted in some very peculiar happenings. It is also a recipe for disaster to use hibernate on a dual booted machine. GNOME contains a database for storing your preferences called 'gconf, which is a similar database to the Windows registry.

There is a Configuration Editor Program for Gnome which can be accessed via Applications -> System Tools -> Configuration Editor to easily make changes for the current user. To make global changes it is best called in a terminal by:

sudo gconf-editor

Navigate through the left hand tree to apps -> gnome-power-manager. Find the options named 'can_hibernate' and 'can_suspend' and uncheck them both.

Right-click on each in turn and click Set as Mandatory to make sure that it applies to all users then exit the Configuration Editor. Changes will not appear until after a reboot.

Suspend and Hibernate bug workround

There is however a bug in Karmic and higher https://bugs.launchpad.net/ubuntu/+source/gnome-power-manager/+bug/432598 which means that the suspend/hibernate disabling through gconf-editor do not work. The following is a short term fix from http://ubuntuforums.org/showthread.php?t=1305081 and extra information in https://bugs.launchpad.net/ubuntu/oneiric/+source/gnome-settings-daemon/+bug/860485 the options still show up in the menu but if you click on one of them it just locks the screen. The following is correct for Lucid to at least Oneiric:

sudo gedit /usr/share/polkit-1/actions/org.freedesktop.upower.policy

And change the code:

<allow_active>yes</allow_active>

entries for suspend and/or hibernate to:

<allow_active>no</allow_active>

and restart.

The following shows results:


pcurtis@matrix:~$ pkaction --action-id org.freedesktop.upower.suspend -v
org.freedesktop.upower.suspend:
description: Suspend the system
message: Authentication is required to suspend the system
vendor: The UPower Project
vendor_url: http://upower.freedesktop.org/
icon: system-suspend
implicit any: no
implicit inactive: no
implicit active: no

Changing to a Static IP Address

First set up the connection as usual which gives dynamic IP address (DHCP) then use the drop down menu on the Network Manager App to first display the existing connection then the Edit Connection to select the connection to edit and bring up the editing screen as below. You first change the only change the IPv4 settings tab - first change Automatic DHCP to Static then fill in the required IP Address then use the information from the Connection Information to obtain the Netmask, Gateway and Primary DNS settings. This only applies to that connection so a hardwired ethernet connection or other Wifi will still have an Automatic IP generated as you would want when using an internet cafe or other friends ethernet.

If you have a static address it is possible to set up the /etc/hosts file to contain the host name on the machine which needs to access the machine with a fixed IP. I need to do that because the hostname resolution does not seem to be working on the machines with Xubuntu.

One then has to modify the /etc/hosts file on every machine which needs to access the machines

sudo gedit /etc/hosts

Xubuntu 11.10 apparmor changes needed.

A look at dmesg showed that the apparmor rules for were blocking parts of the host lookup

dmesg | grep apparmor

I put that section into a complain (log but not enforce) mode by

sudo apt-get install aa-complain
sudo aa-complain /usr/sbin/ntpd
sudo /etc/init.d/apparmor status
dmesg | grep apparmor

Xubuntu Firewall changes required.

I am not sure if I inadvertently made some changes to the firewall when problem solving but I did a dmesg and found the firewall was blocking ssh (port 22) and renabled that port and also the netbios lookup ports by:

sudo allow ufw 22
sudo allow ufw 139
sudo allow ufw 445
sudo ufw status


Problems with Wine 1.3.31 and Dreamweaver

Pre Oneiric: I have found that some of my programs such as Dreamweaver crash frequently with the latest versions of Wine from the WineHQ Ubuntu PPA using both natty and oneiric. This is certainly the case with version 1.3.31 so I have backed off to the stable versions Wine 1.2.x where possible. I disabled the PPA using tthe Synaptic Package Manager -> Settings -> Repositories -> Other Software tab -> Untick Wine then Uninstalled wine, wine1.3, winetricks, wine-gecko and ttf-symbol-replacement then installed wine1.2, winetricks and wine1.2-gecko. Currently I am editing this using wine 1.2.3 and Ubuntu Natty 11.04 with Unity. That has seemed to solve the problems under 10.04 and 11.04.

Oneiric: The Wine 1.2.3 version did not work at all with Oneiric so I had to reload 1.3.28 which was the version in the standard Ubuntu repository rather than the WineHQ PPA and that seems OK. I will just have to make sure it does not update as far as 1.3.31.

Madwifi Driver under Xubuntu 11.10

I have been operating the old Toshiba under Xubuntu 11.04 using an external Wifi dongle because the Madwifi driver did not 'make' correctly. I then downloaded the latest build of Madwifi and that works fine with Xubuntu 11.10 so my existing instructions MadWifi drivers for the Atheros Wifi card were OK apart from the addition of the specific need to download a new version when changing Ubuntu Distribution. When there are kernel updates one still has to run my script as well.

Preventing need for login password when returning from Suspend in Oneiric

It used to be possible to not have the screen lock on suspend while still having the option to manually lock the screen in the session menu in Lucid and Natty (Gnome 2). It has been files as a bug https://bugs.launchpad.net/ubuntu/+source/gnome-power-manager/+bug/871560 .

The following does what is required but is only a workround because it disables screen locking system wide and one can no longer lock a screen manually when, for example, one has sensitive documents displayed.

Type the following command in terminal to inhibit the request for a login and password when returning from a suspend. Do NOT sudo it, it must be for the current user.

gsettings set org.gnome.desktop.lockdown disable-lock-screen 'true'

There is more on this problem and some useful introductory information on use of gsettings at http://ubuntuforums.org/showthread.php?p=11395502

Lubuntu

The Lubuntu 11.10 ISO file contains a known issue that causes it to fail to load a live CD session on some hardware, instead loading to a command prompt. Users are required to enter sudo start lxdm at the prompt to run the live CD session.

sudo start lxdm

O2 Voicemail ring time

You can change the ringing time before Voicemail 901 is activated, simply enter the following code **61*901*11*(NO OF SECS)# send. No of secs must be 5,10,15,20,25 or 30 - also see http://www.talk3g.co.uk/showthread.php?6655-O2-Extending-the-Ring-Time-%28iPhone-and-others%29

I have carried this out and it seems to work fine for 30 seconds

Firewalls and Xubuntu

I have had a long chase to get hostname resolution enabled for connections TO Xubuntu machines. The most likely scenario seems to be that the firewall shipped with Xubuntu has different defaults to Ubuntu.

Linux has an extremely powerful firewall built in, commonly referred to as iptables, but more accurately is iptables/netfilter. Iptables is the user interact that allows one to enter firewall rules into predefined tables. Netfilter is a module built into the kernel, that actually does the filtering based on these tables. There are a number of command line and gui front ends for iptables that allow users to add or define rules.

Iptables uses the concept of ip addresses, protocols (tcp, udp icmp) and ports. Iptables places rules into predefined tables (INPUT, OUTPUT and FORWARD) that any network traffic (IP packets) is checked against and a decision is made on a packet by packet basis to, for example, accept or drop the packet.

Iptables also has the ability to monitor the state of a connection and redirect, modify or stop data packets based on the state of the connection, not just on the source, destination or packet data content. Iptables is what is known as a stateful firewall as is aware of the context from which a data packet arises and can thus make better decisions on the fate of packets and connections.

Most Linux distributions come with Itables already configured to give excellent protection and no adjustments are required. When programs or daemons are loaded which need additional special access they the configuration is normally done automatically when they are installed. The normal default configuration in Ubuntu is that all outgoing ports are open and all incoming ports are closed except for expected packets responding to outgoing traffic. For the last 5.5 years since I started using Ubuntu this has all been completely transparent to me and I have not tried to change the firewall, in fact I have never even checked the settings!

Unfortunately when I came to use Xubuntu I found I had difficulties getting hostname resolution to work even after I had installed and set up Winbind which provides the old Netbios host name resolution - it worked from the Xubuntu machine but none of my other machines could get theaddress of the Xubuntu machines. In despiration I gave them fixed IP addresses and set up the /etc/hosts file on the machines which needed to access them for Unison synchronisation.

As a matter of principal I did a lot of searching around and started to play with the firewalls which I think may have made things worse. The breakthrough came when I found references to a very powerful diagnostic package for seeing what ports are open on a machine called nmap (Network MAPper).

Nmap ("Network Mapper") is an open source tool for network exploration and security auditing. It was designed to rapidly scan large networks, although it works fine against single hosts. Nmap uses raw IP packets in novel ways to determine what hosts are available on the network, what services (application name and version) those hosts are offering, what operating systems (and OS versions) they are running, what type of packet filters/firewalls are in use, and dozens of other characteristics. While Nmap is commonly used for security audits, many systems and network administrators find it useful for routine tasks such as network inventory, managing service upgrade schedules, and monitoring host or service uptime.

The output from Nmap is a list of scanned targets, with supplemental information on each depending on the options used. Key among that information is the "interesting ports table". That table lists the port number and protocol, service name, and state. The state is either open, filtered, closed, or unfiltered. Open. means that an application on the target machine is listening for connections/packets on that port. Filtered. means that a firewall, filter, or other network obstacle is blocking the port so that Nmap cannot tell whether it is open or closed. Closed. ports have no application listening on them, though they could open up at any time. Ports are classified as unfiltered. when they are responsive to Nmap´s probes, but Nmap cannot determine whether they are open or closed. Nmap reports the state combinations open|filtered. and closed|filtered. when it cannot determine which of the two states describe a port. The port table may also include software version details when version detection has been requested. When an IP protocol scan is requested (-sO), Nmap provides information on supported IP protocols rather than listening ports.

In addition to the interesting ports table, Nmap can provide further information on targets, including reverse DNS names, operating system guesses, device types, and MAC addresses.

It is not installed by default so open a terminal and do:

apt-get install nmap

The way I have been using is like this:

pcurtis@matrix:~$ nmap -vAPN 192.168.2.102

Starting Nmap 5.21 ( http://nmap.org ) at 2011-10-31 17:03 GMT
NSE: Loaded 36 scripts for scanning.
Initiating Parallel DNS resolution of 1 host. at 17:03
Completed Parallel DNS resolution of 1 host. at 17:03, 0.00s elapsed
Initiating Connect Scan at 17:03
Scanning 192.168.2.102 [1000 ports]
Discovered open port 445/tcp on 192.168.2.102
Discovered open port 22/tcp on 192.168.2.102
Discovered open port 139/tcp on 192.168.2.102
Completed Connect Scan at 17:03, 2.94s elapsed (1000 total ports)
Initiating Service scan at 17:03
Scanning 3 services on 192.168.2.102
Completed Service scan at 17:03, 11.04s elapsed (3 services on 1 host)
NSE: Script scanning 192.168.2.102.
NSE: Starting runlevel 1 (of 1) scan.
Initiating NSE at 17:03
Completed NSE at 17:03, 0.42s elapsed
NSE: Script Scanning completed.
Nmap scan report for 192.168.2.102
Host is up (0.040s latency).
Not shown: 997 closed ports
PORT STATE SERVICE VERSION
22/tcp open ssh OpenSSH 5.8p1 Debian 7ubuntu1 (protocol 2.0)
| ssh-hostkey: 1024 4a:22:c7:c8:2c:df:d9:1b:ca:f3:47:b2:d9:36:91:97 (DSA)
|_2048 ca:47:35:e4:a3:9e:27:60:c0:dc:72:6a:23:09:58:f3 (RSA)
139/tcp open netbios-ssn Samba smbd 3.X (workgroup: OFFICE)
445/tcp open netbios-ssn Samba smbd 3.X (workgroup: OFFICE)
Service Info: OS: Linux

Host script results:
| nbstat:
| NetBIOS name: VORTEX-UBUNTU, NetBIOS user: <unknown>, NetBIOS MAC: <unknown>
| Names
| VORTEX-UBUNTU<00> Flags: <unique><active>
| VORTEX-UBUNTU<03> Flags: <unique><active>
| VORTEX-UBUNTU<20> Flags: <unique><active>
| OFFICE<1e> Flags: <group><active>
|_ OFFICE<00> Flags: <group><active>
| smb-os-discovery:
| OS: Unix (Samba 3.5.11)
| Name: OFFICE\Unknown
|_ System time: 2011-10-31 17:03:58 UTC+0
|_smbv2-enabled: Server doesn't support SMBv2 protocol


Read data files from: /usr/share/nmap
Service detection performed. Please report any incorrect results at http://nmap.org/submit/ .
Nmap done: 1 IP address (1 host up) scanned in 14.80 seconds

You can see it has identified the open ports for SSH, samba and netbios (winbind) and picked up the hostname without problems on an standard Ubuntu machine (and Lubuntu) whilst nothing showed up on a Xubuntu machine.

I found that one can scan a larger range of ports to get to 5353 for mDNS by

nmap -vAPN 192.168.2.102 -p 1-6000

UFW Uncomplicated FireWall

Ubuntu has a package called ufw which is an acronym for Uncomplicated FireWall. This package provides a command line interface for managing and configuring rules for the Netfilter iptables based firewall. The gufw tool provides a user friendly graphical interface to ufw designed to make firewall management possible without the need to issue ufw commands at the command line.
Installing gufw on Ubuntu

Whilst ufw is installed on Ubuntu by default, the gufw package is not.

Install Firewall and Samba utilities with netbios resolution for hostnames directly and from Windoz machines via netbios.

apt-get install ufw gufw samba samba-tools system-config-samba winbind

Set up Firewall for Samba

sudo ufw allow proto tcp to any port 135 from 192.168.2.2/254
sudo ufw allow proto udp to any port 137 from 192.168.2.2/254
sudo ufw allow proto udp to any port 138 from 192.168.2.2/254
sudo ufw allow proto tcp to any port 139 from 192.168.2.2/254
sudo ufw allow proto tcp to any port 445 from 192.168.2.2/254

You should also change /etc/default/ufw by adding the netbios_ns communication tracking line so you can use samba fully:

# The nf_contrack_netbios_ns has been added below
IPT_MODULES="nf_conntrack_ftp nf_nat_ftp nf_conntrack_irc nf_nat_irc nf_conntrack_netbios_ns"

Set up Firewall for avahi/bonjour/zeroconf - for mdns hostname resolution using hostname.local

sudo ufw allow proto udp to any port 5353 from 192.168.2.2/254

Access system-config-samba via System -> Administration -> Samba and do all your configuration there.
Access gufw if required by Adminstration -> Firewall

Useful Links:

I final traced my problems to the Samba installation had not set up the ports it needed correctly yet seemed to over-ride UFW settings. Reinstalling using Synaptic Package Manager cleared the problems with hostname resolution - the strange thing was that Samba itself seemed to be OK and printer and filesystem access over the network seemed to be OK. It only occured on the Xubuntu machines not those with any version of Ubuntu or the Lubuntu machine. A strange bug. nmap was key to understanding where the problem was occuring.

Zeroconfiguration/MDNS

Ubuntu can use Multicast DNS Service Discovery which allows programs to publish and discover services and hosts running on a local network without any specific configuration. For
example you can plug into a network and instantly find printers to print to, files to look at and people to talk to.

This feature known as zeroconfiguration does not seem to be set up to be working by default as Port 5353 has not been opened up although the software (avahi) is present and the daemon running. When port 5353 is opened using UFW mdns discovery works and machines can be accessed by hostname.local without needing netbios - I assume one would then not need samba/winbind to use Unison with SSH

Lubuntu

Lubuntu is a lightweight Linux operating system based on Ubuntu but using the LXDE desktop environment in place of Ubuntu's Unity shell and GNOME desktop. LXDE is far less resource hungry.LXDE stands for Lightweight X11 Desktop Environment. Lubuntu is targeted at "normal" PC and laptop users running on low-spec hardware. Users who may not know how to, or wish to, use command line tools, but don't have enough resources to run the with all the bells and whistles of the "full-featured" mainstream distributions. Lubuntu received official recognition as a formal member of the Ubuntu family, commencing with Lubuntu 11.10. This means that the Ubuntu team take care of LXDE and other packages that are part of the Lubuntu install and one can even swap between desktops without a complete reinstall. All the programs available under Ubuntu are available through the Synaptic Package Manager and PPAs can also be used.

LXDE is a very lightweight and minimal desktop environment using the Openbox window manager, the pcmanfm file manager, and a carefully selected list of default applications. It is compatible with basic freedesktop standards and all necessary parts are already packaged in Ubuntu. It’s surprising what can be achieved when one restricts unnecessary eye-candy and the bloat that Ubuntu is rapidly gaining.Lubuntu only requires 128 MB of RAM to run and 256 MB of RAM to install with the graphic installer although the recommended minimum RAM to run a live CD session is 384 MB.

The LXDE/Lubuntu window manager is Openbox - an OpenSource window manager for the X Window System. Looking under the hood, Openbox is designed to be small, fast, and fully compliant with the Inter-Client Communication Conventions Manual (ICCCM) and Extended Window Manager Hints (EWMH).It supports many features such as menus by which the user can control applications or which display various dynamic information. Openbox allows a right-click "main menu" on the desktop,and allows users to configure the way windows are managed. When a window is minimized it becomes invisible which initial can cause consternation. To bring windows up again most use Alt+Tab or the Desktop menu, accessible from the right-click menu. Openbox is extended under LXDE with small programs that add icons, taskbars,etc.

The PCManFM file manager is arguably a little too light and does not compete with Thunar, leave alone Nautilus but it's a competent and robust application that doesn't hog resources like Nautilus.

Another major reason why Lubuntu runs fast and uses less resources is that it starts less daemons and services than Ubuntu by default, as you can see in the process list as well as in the menu (Preferences -> Desktop Session Settings -> Automatically Started Applications).

In summary LXDE fulfills the goal that Xubuntu, with the XCFE desktop was intended to address but currently falls well short of, namely that of providing a lightweight desktop environment on machines with low ammounts of RAM. Tests show Lubuntu can use half as much RAM as Xubuntu making it an attractive choice for netbooks, mobile devices and, in particular, older PCs with "minimal" or legacy hardware.

LXShortCut: Creats a Launcher for the Lubuntu Oneiric desktop

You can put any item in the menu onto the desktop with a right click but there seems to be no built in way to create your own Desktop Launchers for LXDE (the graphic environment of Lubuntu), you can create .desktop files with a text editor (imititating the ones in /usr/share/applications) but it's much easier to run lxshortcut in a terminal.

lxshortcut -o ~/Desktop/new_shorcut.desktop

And a window, will pop up with all the fields you need to define (including an icon), save - that's it, you have a new shortcut icon on your desktop which you need to make executable (right click -> properties -> Permissions and tick box) and you should also rename.

Autostart Programs in Lubuntu

1. Using .desktop files in the autostart folder (for a single user)

Any shortcut files in the folder at /home/username/.config/autostart will be run during the login sequence.

Use lxshortcut as above to create a new shortcut which you can test on the desktop then copy it into ~/.config/autostart/

Once .desktop files have been added you can manipulate them with the GUI configuration tool lxsession-edit (not yet tried)

You can also link a program's .desktop in /usr/share/applications/ file to ~/.config/autostart/. For example, to execute lxterminal automatically at startup:

$ ln -s /usr/share/applications/lxterminal.desktop ~/.config/autostart/

2. Using the autostart files (single user or global files exist)

The second method is to use a ~/.config/lxsession/LXDE/autostart file. This file is not a shell script, but each line represents a command to be executed, if a line begins with a @ symbol, the command following the @ will be automatically re-executed if it crashes. For example, to execute lxterminal and leafpad automatically at startup:

~/.config/lxsession/LXDE/autostart

@lxterminal
@leafpad

Note: The commands do not end with a & symbol.

There is also a global autostart file at /etc/xdg/lxsession/LXDE/autostart. If both files are present, all entries in both files will be executed.

Lubuntu and Synaptic Quick Search

The quick search feature in Synaptic is disabled by default, due to hanging on some systems with it enabled. However if you are confident that this won't happen to you, it is easily enabled. Just open a LXTerminal session and enter the following command:

sudo apt-get install apt-xapian-index

It has an additional dependency it wants to install to function properly. After installation it will index the repositories, which will take few minutes the first time. After this small venture to the terminal it should work properly again.

Rhythmbox and Exaile in Lubuntu

Both these media players crash in Lubuntu because there is effectively an unmet dependency in that they need gstreamer0.10-alsa to link to the sound system when they start to play. You will find various messages on the screen or in a terminal if you run from the command line when you start to play. These include:

Cannot connect to server socket err = No such file or directory
Cannot connect to server socket
jack server is not running or cannot be started

Another message is

audiosink bin 1 is not working

This took me a good part of a day before I discovered from a remote posting that the solution is simple - just install gstreamer0.10-alsa by

sudo apt-get install gstreamer0.10-alsa

You will also need ubuntu-restricted-extras or specific other gstreamer packages to play MP3 files. ubuntu-restricted-extras brings in a wide sweep of useful programs, fonts and codecs but does take up a fair amount of space. If not try gstreamer0.10-plugins-good gstreamer0.10-plugins-bad gstreamer0.10-plugins-ugly and gstreamer0.10-ffmpeg

Auto mount partitions using Storage Device Manager (pysdm).

PySDM is a GUI to manage fstab (/etc/fstab) and allows customizing hard disk mountpoints.

Install PySDM in Ubuntu using the following command:

sudo apt-get install pysdm

Before proceeding, even though nothing should go wrong if you follow the exact steps below, it's still a good idea to back up fstab:

sudo cp /etc/fstab /etc/fstab.old

Video Formats for the Humax HDR FOX-T2 PVR - Handbrake

I have been experimenting to find a better format for video files for the Humax which is very choosy about the formats it will play both from a storage source on its USB port or over the network. The files formats which are acceptable do not always overlap between USB and Network. All my video files are in a 'straight' mpeg 2 format which can be fed onto a DVD without a video transformation. The Humax will play them from USB but non of the 'trick' control works - the best you have is pause/play and return to play from the same spot. Local recordings and sme external files can be played with positional control via the cursor - steps forward and back from keys and 2x and 4x fast forwards and back.

Searches in the conferences indicate that H264 video and AAC sound in an MP4 container allow all the 'tricks' from a server and that VLC conversions and those from Handbrake are worth experimenting with. Handbrake is a sophisticated and well developed cross platform open source programme so I thought I would give it a try. I installed using a PPA under natty and upgraded which is still fine and I have the latest version still. There is a problem with Oneiric as there is a missing dependency as well as no PPA yet - see http://askubuntu.com/questions/71128/how-do-i-install-handbrake.

I have used a modification of the 'High Profile' for video which will be streamed. Changes of significance are:

Main Window

Picture settings window

I have saved these settings for future use

Subsequent experiments have shown that it is possible to crop the video and force the video to be scaled/transcoded to be 720x576 - this also plays over DLNA with all the tricks working. The cropping is useful for VHS videos which one has captured as they have lots of artifacts at the edges. I can save the main settings but it does not seem to save those one has to 'play with' to force the transcoding to 720x576. I am not sure how the Humax would like files of arbitary aspect rations and pixel dimensions to scale so the scaling seems a good idea.

 

MediaInfo

MediaInfo is a way to get information about a media file format - it is in a PPA for Ubuntu. The Linux Gui is very simple and a bit idiosyncratic but usable.

Saving the .deb from a specific Wine Version

I know that Wine1.3.28 and Dreamweaver work well and 1.3.31 fails so I wanted to save a copy of the .deb file for 1.3.28 so i could reload it if it got updayed accidently. I expected to find a copy in /var/cache/apt/archives but it was not present although 1.3.30 and .1.3.31 from downloads from the WineHQ PPA were.

I therefore open the Synaptic Package Manager and started a reinstall of Wine (1.3.28 still) but when I got the list of files etc it would download I ticked the box for download only then after the download had completed I exited Synaptic and picked up the freshly downloaded Wine 1.3.28 from /var/cache/apt/archives and copied it to a safe location

I should also be able to downgrade to a version in cache by

apt-get install wine1.3=1.3.28-0ubuntu1_i386

but have not tried it yet

Mounting a LiveCD .iso for Booting with Grub2 for a minimal system

It is possible to add extra entry to the Grub2 boot menu which will mount ISO files (.iso) via a loopback and to boot from them as if it they were an ordinary LiveCD or LiveUSB

Copy the ISOs in , for example, /home, find the partition where is /home which is expressed as (hd"number_of_drive","number_of_partition"). NOTE: Drives start with 0, partitions always with 1 (check grub2 ubuntu documentation...). Then do:

sudo gedit /etc/grub.d/40_custom

and for every iso add code in the following form:

#!/bin/sh
exec tail -n +3 $0
# This file provides an easy way to add custom menu entries. Simply type the
# menu entries you want to add after this comment. Be careful not to change
# the 'exec tail' line above.
# This is for ISOs in /home folder of sda2
menuentry "Boot Lubuntu 11.10 Alternate Install ISO" {
loopback loop (hd0,2)/home/lubuntu-11.10-alternate-i386.iso
linux (loop)/casper/vmlinuz boot=casper iso-scan/filename=/home/lubuntu-11.10-alternate-i386.iso file=(loop)/preseed/ubuntu.seed quiet splash --
initrd (loop)/casper/initrd.lz
}
menuentry "Boot Partition Magic ISO" {
loopback loop (hd0,2)/home/pmagic-11.11.11.iso
linux (loop)/casper/vmlinuz boot=casper iso-scan/filename=/home/pmagic-11.11.11.iso file=(loop)/preseed/ubuntu.seed quiet splash --
initrd (loop)/casper/initrd.lz
}
menuentry "Boot Lubuntu 11.10 Alternate ISO" {
loopback loop (hd0,2)/home/lubuntu-11.10-alternate-i386.iso
linux (loop)/install/vmlinuz iso-scan/filename=/home/lubuntu-11.10-alternate-i386.iso quiet splash --
initrd (loop)/install/initrd.gz # note .gz
}
menuentry "Parted Magic 11.11.11 ISO (Low RAM Live mount) " {
set isofile="/home/pmagic-11.11.11.iso"
loopback loop (hd0,2)$isofile
linux (loop)/pmagic/bzImage iso_filename=$isofile boot=live load_ramdisk=0 prompt_ramdisk=1 lowram livemedia noeject noprompt
initrd (loop)/pmagic/initramfs
}
menuentry "Parted Magic 11.11.11 ISO (Standard Ramdisk mount)" {
loopback loop (hd0,2)/home/pmagic-11.11.11.iso
linux (loop)/pmagic/bzImage iso_scan/filename=/home/pmagic-11.11.11.iso boot=live load_ramdisk=1 prompt_ramdisk=0 noeject noprompt
initrd (loop)/pmagic/initramfs
}

You must now rebuild the grub2 confoguration in the following manner or nothing will happen

sudo update-grub2

Note: You can hold as many distributions as you want at once and have them displayed as menu entry in grub2 and also any other LiveCDs such as Parted Magic

References:

Video on Web Sites

The following are a couple of tests - the first with JW Player and the second taken from the code provided to embed video from the Touch Reading site for Theale Fireplaces.

Loading the player ...

Routine to remove colons from file names

find /tmp/ -depth -name "*:*" -execdir rename 's/:/-/g' "{}" \;

 

Valid HTML 4.01 Copyright © Peter & Pauline Curtis
Content revised: 2nd August, 2020