|Home||Uniquely NZ||Travel||Howto||Pauline||Small Firms|
Diary of System and Website Development
Part 16 (September 2009 -> December 2009)
Ubuntu does not automatically upgrade to the latest issue of OpenOffice, it just applies any updates to the version which came with the original Distribution ie 2.4 in the case of Hardy Heron. There are advantages in some cases in upgrading to version 3.x which is in Jaunty.
There is now a repository set up for the latest versions of OpenOffice which can be used to keep OpenOffice updated automatically. This is the Personal Package Archive PPA set up by the Openoffice Scribbler - see https://launchpad.net/~openoffice-pkgs/+archive/ppa and http://www.rebelzero.com/ubuntu/ppa-for-openofficeorg-301-for-hardyintrepid/94.
The way to include this repository is to:
Add the OpenOffice PPA repository to your sources.list file by System > Administration > Software Sources. Click on the Third-Party Software tab and click the Add… button. Copy the PPA’s repository address in the APT Line box, and click the Add Source button. Hardy users should use:
deb http://ppa.launchpad.net/openoffice-pkgs/ppa/ubuntu hardy main
Replace hardy by jaunty or the version you are using as appropriate
You will be asked to update the repository list
Next it is important that you click on the Ubuntu Software tab and make sure the universe repository is enabled as the PPA packages need some hardy packages from that repository.
Finally you need to add the authentication keys for this repository, this is moste asily done in a terminal by:
sudo apt-key adv --keyserver keyserver.ubuntu.com --recv-keys 247D1CFF
You can use the Synaptic Package Manager to update everything (Add/Remove was not happy with these changes until I had used Synaptic). System -> Administration -> Synaptic Package Manager
First update the package lists by clicking Reload
Then click Mark all Possible upgrades which should show all the ones for Open Office upgrade and click Apply.
You should now find that OpenOffice has been upgraded.
A number of 'Services' are required if one is to self publish books and sell them.
A print on demand self-publishing site preferably with country specific versions but with online worldwide retailing - the service by which others are judged is Lulu.
A means to make and receive payments for internet transactions. Paypal is the best known service for individuals and is now widely used and is the service by which others are judged.
Email, preferably a separate account used only for the publishing and other sales activities as email address is also used as a login address for some services and/or is available to customers.
A full bank account which allows direct debit access to be set up and trans fers in by BACS, CHAPS etc transfers. It is best if this has internet access as some of the validation techniques need a rapid response. There are advantages if this is again a separate account to reduce the risks of internet fraud and also make accounting easier in the unlikely event one makes a lot of money.
The Publishing and Payment services both obviously have to be able to handle payments to you and be linked to a bank account so the proceeds can be passed to you. They need to have good and visible internal accounting with transactions details stored for a long period, preferably with the ability to do analysis.
Lulu enables one to upload files in a number of formats as well as a create or upload a cover page some of the details have already been covered above but this section will expand that to the actual use of Lulu. Before you can start you have to create an account but even before that most of the help files are available.
Once you have understood exactly what you need the actual production of a book on Lulu is fairly straight forwards but you should have a practice run before you produce the final version as changes become more difficult as you reach the end stages of the process. Repeating the production process with existing tested files to get a clean copy for printing a first copy for printing then takes only a few minutes. Once you have finalised the book enough to print it become slightly more difficult to make changes, in effect you are then under a version control system and although it can be withdrawn and hidden you can never completely remove a book that has reached that stage. You should however always produce one or two physical copies before committing to a larger print run or taking the next stage and making it available to everyone via Lulu as there may be a few errors which are not obvious on the screen - we made two sets of changes following printing, the first was a few cases of number being use as say 9 when convention says it should be nine and after that correction was printed a we found that version had a slightly different size of page number in the appendices to the main body of the book. Neither would have been a cause to reprint or add a correction to a conventional book but print on demand allows such minor corrections to be made. However if you make anything more than the most minor correction after it is fully in print and your copies have been sent to the copyright libraries then a new ISBN is required.
If you are going to publish a book through Lulu which they distribute and which has an ISBN they impose some additional quality control as their name is on the line. In particular the final PDF file which is sent to the printer has to be generated by genuine Adobe software or by their own converter from other formats. This converter allows you to add together several files and merge them. We are using the OpenOffice word processor which outputs an acceptable for most printers but we have used there converter to put a 'stamp of approval' on it by appending a blank page from another file at the end - the blank page is required by most printers so this does not increase the length in practice. So, at the end of the day, we have to upload three files to Lulu as we work through their publication Wizard, The PDF of the books text, a file containing a blank page to terminate the book, force checking of the PDF and producing a PDF which is hopefully approved for any form of sales package or publisher and the last file is the cover, again in PDF format.
Go to www.lulu.com
Click login and fill in username (email account) and password
Go to My Lulu tab
My Lulu Tab has 6 tabs (Dashboard, My Projects, My Accounts, My Revenue, My Orders, My Files)
Money you earn can only be remitted by cheque or to a PayPal Account and this is set up by My Lulu -> My Accounts tab -> Change Remission Settings (on the left hand side menu)
Payments you make for books and services can be paid by credit card which can be changed or set up My Lulu -> My Accounts tab -> Credit Card Information (on the left hand side menu)
Paypal is very popular way of making and receiving payments and money transfers through the Internet. PayPal serves as an electronic alternative to traditional paper methods such as checks and money orders. A PayPal account can be funded with an electronic debit from a bank account or by a credit card. The recipient of a PayPal transfer can either request a check from PayPal, establish their own PayPal account for the deposit or request a transfer to their bank account. PayPal is an example of a payment intermediary service that facilitates worldwide e-commerce. Whilst PayPal is a wholly owned subsidiary of eBay with corporate headquarters in the United States, it also operates as a Luxembourg-based bank with appropriate regulation within the EU. Currently, PayPal operates in 190 markets, and it manages over 184 million accounts, more than 73 million of them active. PayPal allows customers to send, receive, and hold funds in 19 currencies worldwide. It handles about £40 billion of transactions per year.PayPal offers three types of account:
Each type of account has different features, including different sending and receiving limits. When you sign up on their site you are seemingly only offered the choice of Personal and Business accounts but that is because the personal and premier accounts now seem to have converged and in practice you seem to get an account which is able to receive payments - this account also allows you to use some of the 'Merchant' facilities which means you can set up for sales of one off items (rather than a full emarketing service with shopping basket etc) on you web site or by a link in an email.
When you first set up the account everything is in place and useable but there are a number of restrictions on the amounts that you can receive or send per transaction or per year. These restrictions can then be lifted - the first and essential step is to validate the link with your nominated bank account. Validation is achieved by PayPal making a series of very small transfers into your account which you then have enter on the PayPal web site within a time limit. These transfers take several days to arrive so you need internet banking to keep checking or access to telephone banking or via an ATM. This raises the limits to around£1000 per year beyond which more verification is required because of money laundering etc regulations. You also need to fill in some information which they call business information but has the ability to handle an individual - this needs information on your expected methods of sales, average transaction sizes and monthly transaction estimates.
When starting up with a low level of transactions two of the Merchant services are appropriate - Email Payments and Website Payments Standard. Both are available form a Premium Account.
Email Payments allows you to accept payments by credit card, debit card and bank transfers all by email. You don't need a website – you just send an invoice or request for money from PayPal's website to your customer's email address. They just click on a link to pay you via PayPal. Your customers don't even need a PayPal account to pay you
This Service allows you to use a tool on the PayPal web site to create a button for you web site which is linked to an payment form on the PayPal web site which allows customers to purchase using credit cards, debit cards or a PayPal account. You specify all the basic information on the PayPal web site such as the item name, price, postage etc. and the Wizard will produce the HTML code to copy onto your site. There is considerable customisation possible and you can, for example, use your own image for the button and specify two web pages to be transferred to following successful and unsuccessful translations. It is reached by Paypal Web Site -> Login -> Merchant Services -> Buy Now Buttons (at right of section headed Website Payments Standard).
The advanced options also allow you to personalise the payment web page on the PayPal site using some additional codes put into a section called Add Advanced Variables in the third (optional) step when creating a Buy Now Button.
image_url - The URL of the 150x50-pixel image displayed as your logo in the upper left corner of the PayPal checkout pages.
cpp_header_image - The image at the top left of the checkout page. The image’s maximum size is 750 pixels wide by 90 pixels high. PayPal recommends that you provide an image that is stored only on a secure (https) server.
cpp_ headerback_color - The background colour for the header of the checkout page supplied as a case-insensitive six-character HTML hexadecimal colour code in ASCII.
cpp_ headerborder_color - The border colour around the header of the checkout page. The border is a 2-pixel perimeter around the header space, which has a maximum size of 750 pixels wide by 90 pixels high. Valid value is case-insensitive six-character HTML hexadecimal colour code in ASCII.
cpp_payflow_color - The background colour for the checkout page below the header. Valid value is case-insensitive six-character HTML hexadecimal colour code in ASCII.
I have been looking for a while for a way to get mobile broadband through a WiFi router so we can share an internet connection and networking whilst we are away from home which uses a USB Mobile Broadband dongle. One big advantage is the way these work is identical to any other ADSL, Firewall, WiFi router and are independent of operating system as they are set up and controlled by a web interface. This has been available for a while through 3 but it was locked to their system. Edimax, who I know little about, have brought out a box which seems to do almost everything I want and I have been trying it out. The only shortfall is that it runs off mains via the usual fat plug which supplies 12v at 1 amp to the box and there is no car adapter which would make it perfect for our narrowboat - I have discovered that the eeePC runs off 12volts 2 amps and there are lots of cheap adapters and all I will need to do is adapt the plug into the router.
I bought Edimax Wireless 3G Broadband Router 3G-6200n from Digital Components Ltd for £37 plus the usual extortionate postage and packing but that was amortised as part of a larger order. The actual box is quite light and compact (300gms) and likewise the mains adapter is one of the smallest I have seen. It comes with instructions which are quaint but can be understood and a full manual on CD along with a program to install it on Windoz if you do not want to use the web interface. You first need to connect via a network cable which they provide so you can set up the Wifi. This is easy and you connect via a web interface to 192.168.2.1 which is an excellent choice as most people will be using 192.168.1.1 for their normal router. This takes you into a login screen which displays the default username and password of admin:1234 and then to a Quick Setup which takes you through a subset of the setup I cover below for the Mobile Broadband Dongle and WiFi.
You first get a screen to allow you to enter your location, the other items are preset and fine. You can no plug in the Broadband Dongle if you have not already set it up and click 3G on the next screen. The next screen is where you set up the APN for your provider (pp.internet for Vodafone PAYG) and username and password (web web although I think anything will do) and the dial script (this is almost always *99#). You finally get to the Wifi Setup where you set up the SSID (edimax) channel which should be different to any other Wifi boxes to avoid interference (6).
You now need to setup security. I use WEP 128 bit although there are better systems as everybody can use it and add on Mac address filtering at a latter stage as that restricts access to particular machines only - the Mac address is unique and built into the network hardware.
You access the Security at a later stage via General -> Wireless to Security Settings. You need to select Encryption WEP Key length 128 Key format hex and enter a memorable 26 bit hex number (memorable is a joke) but by repeating a shorter number to make it up to 26 long you have a hope of recalling it. As with everything you need to click apply until you end up having to wait for 30 seconds while it set up the router hardware. If you have Linux machines or the latest Windows drivers you will be able to use WPA which is much more secure and also easier to set up as it has shorter passcodes and I am changing to it on my machines which will be away from home.
Mac access is setup by General -> Wireless -> Access Control. You need to know the Mac addresses of all your machines and add them. When you add them you do not include the : between each one although they are displayed with it! If you do not know them you can find out by Wireless -> Basic settings -> Show Active clients which conveniently stays open in a separate window so you can copy the - cut and paste does not work as they are displayed with the colons! Add them all especially the machine you are working on and then tick the box and Apply etc until you have a 30 second wait. If you get it wrong and cut yourself off you can always use the cable interface to correct your Mac address.
Not surprisingly the box seems to work fine for networking between machines and the internet connection does not need to be present.
There is a connection for a Broadband Modem which can be automatically switched into use when available. I have checked that this can also be used to link to a normal ADSL, Firewall, Router to add Wifi and/or act as an access point in a different part of a house and/or to add WPA to an older Wifi system.
There is also mention of use of the USB port for a network printer which could be used in that configuration. It is not available in the Firmware supplied and you need to do a simple firmware upgrade. I have done the upgrade twice now without any problems, it involves downloading a single file and running a menu item in the control panel and browsing to the location of your file - it is best to do it with an ethernet cable connection. Overall the menus are slightly easier to follow in the new software. I will report further if and when I try the printer option out.
Cautions: There seems to be a slight leakage of data. There are various timeout functions which require the connection to be checked and also there is a check of the time from a timecode server which can be seen in the log files. The leakage seems to be about 3 Mbytes per hour which means it is advisable to disconnect by unplugging the dongle when the system is not in use. There are various timeouts which can be set to break the connection but the one for a 3G connection seems to be missing in versions 2.08, 2.11 and 2.12 of the software compared to the manual which covers 2.00. That said the first day it was on for 10 hours and the cost was £0.84 with two of us using machines and a total data use of about 50Mbytes according to the monitors on the two machines which accounted for £0.75 on the vodafone tariff we were using. The other feature is that it is set up to automatically disconnect after 8 hours and has to be re-enabled by unplugging the dongle or turning it off and on - in view of the data leak, however small, this is sensible but if you place it in the loft for a good signal it could be inconvenient. Again I will report further when I upgrade the firmware.
This is largely undocumented and is not available until one has done a Firmware upgrade to 2.12 or higher. This then makes a new menu item available under General settings. I set this up to Enable Print Server, IPR Enable, LPR Enable, Print Server name edimax, and left the Print Name of USB Port as lpt1. That was all the extra activities on the box and in retrospect it would probably have worked without any changes at that end.
I could not find out much even with internet searches so finally I ran Network Tools and did a port scan - that produced a number of pieces of paper out of the printer as well as revealing that there was a LPR printer server was listening on port 515. LPD/LPR is short for line printer daemon/line printer remote, a printer protocol that uses TCP/IP to establish connections between printers and workstations on a network. The LPD software runs as a daemon in the in the print server and the LPR software is already built into most Linux systems. The LPR client sends the print request to the IP address of the LPD printer/server, which in turn queues the file and prints it when the printer becomes available.
I set up the Printer via System -> Administration -> Printers -> Create New Print Queue -> Network -> LPD/LPR Printer and filled in the boxes giving a result as below - note I did not need the Print Server Name at all, just the IP address, port and printer stream.
That got the printer up and running but it could only be accessed by WiFi or the Ethernet connections on the Router as the Router did not pass back to the existing network. I therefore decided to use the Edimax 6200 as the Main Network Router and Firewall and just use my existing Wifi, Firewall, ADSL Router as a fancy modem with a cable connecting it into the WAN connection on the Edimax 6200. An associated advantage is that the Edimax supports WPA as well as WEP which is all I had on my ancient Router. The only problem is that I need to prioritise which Wifi connection is used on the Computers as both are broadcasting still - or turn off Wifi on the old 3Com Router. At least I now do not need to keep a computer running just as a SAMBA print server.
I used the router as a Network Print Server and Wifi Router under firmware version 2.12 for about 6 weeks with no problems until I took it away and tried to use it with for Mobile Broadband. It then became clear that the connection was only staying up for about 1 to two minutes at a time before it disconnected and reconnected. My checks when I had upgraded had been primarily to do with its use as a print server. I downloaded firmware version 2.08 which was still available on the web site and the Mobile Broadband was then perfect again after I had reconfigured all the settings which are lost every time you do a firmware update as you are supposed to do a full restart and load of default settings after every firmware update. There is a facility to save and reload settings but it is still a very undesirable state of affairs and I have contacted the Edimax support and await a response. This may only a problem with my Vodafone Mobile Broadband dongle which is badged as a K3565 but is normally detected as a Hauwei E160E which it is a version of.
I suddenly started having errors on the form handlers on the web sites I look after. It turned out that the Hosting Service had changed the level of error reporting and aborted when there were notices such as unset variables coming from blank boxes in a form. I sorted out some of the inputs with tests but also used the lines
// Report all errors except E_NOTICE
// This is the default value set in php.ini
error_reporting(E_ALL ^ E_NOTICE);
which return to the standard default php settings - see http://php.net/manual/en/function.error-reporting.php for lots of interesting examples as well as the definitions.
I have been going through my procedures for Backing up and Synchronising again. This was provoked by the hard drive failing on Pauline's Toshiba Satellite Pro laptop which is the machine she uses at home most of the time. We lost very little of the Ubuntu system and our data but the Windows system was completely lost. The hard drive was accessible through a panel on the underside and a much larger replacement drive of 120 Gbytes (to match the MSI Winds was obtained and fitted.
The Toshiba was running under Ubuntu Jaunty Jackalope and we had an issue with poor sound which would randomly stop leaving only clicking and also video playback would freeze so this seemed to also be the time to try out the new Karmic Koala. This fixed the sound problem completely but gave another problem with the Wifi drivers if one wanted to use WAP security with the newly developed drivers available in the kernel with the particular Wifi card. Unfortunately the kernel builders had been so confident that they had removed support this time for the proprietary MadWifi driver which had been available in Jaunty. We experimented with fixing the audio versus fixing the Wifi and the upgrade was so much better we chose to go that way and make and install MadWifi drivers for karmic - this is not difficult but will need to be repeated for every kernel update like we used to have to do on the MSI Wind.
This is needed to be able to use the Wifi built into our Toshiba Satellite Pro L20 as the new ath5k driver now built into the latest kernels does not yet support the Atheros driver version used in the Satellite Pro L20 and a number of other machines well if you want to use WEP or WAP security.
After much searching I found a good set of instructions here: http://art.ubuntuforums.org/showthread.php?t=1163380 and there is more information at http://ubuntuforums.org/showthread.php?t=1309072 .
The following is my procedure for use under Ubuntu Karmic Koala using Grub 2 which is the default for a new install if you still have the original Grub loader then you need to modify different files to blacklist the ath5k driver.
First install the utilities for building a kernel driver by typing the following in a terminal:
sudo apt-get install build-essential
Now sort out which drivers will be available by typing the following in a terminal:
sudo gedit /etc/modprobe.d/blacklist-ath.conf
change the last line by adding a # at the start to comment out the blacklisting of the MadWifi driver so it reads # blacklist ath_pci and save it then type the following in a terminal:
sudo gedit /etc/modprobe.d/blacklist.conf
now add blacklist ath5k on a new line at the end.
These two actions mean we will be able to use the new MadWifi driver ath_pci instead of the built in driver ath5k.
We now need to obtain the new driver by downloading the latest version from http://snapshots.madwifi-project.org/madwifi-0.9.4-current.tar.gz , it will initially download to the desktop where you can unpack the archive by double clicking it and dragging the folder within it to somewhere you can easily find it as you need to compile it again every time you update kernel - the best place is your home folder and also renaming it to madwifi from a name which includes the version number. The follow steps assume that is done and the folder is in . Once this has been done we need to compile the driver and install it into the kernel. The stages above only need doing once - the following need to be repeated every time the kernel is updated - you will know when as the Wifi will stop working!
The following is a list of the commands one needs to type in a terminal after every kernel change.
sudo make install
sudo depmod -ae
sudo modprobe -r ath5k
sudo modprobe ath_pci
This reports warnings about an option -e which seem to cause no problems - you can try taking the e off the end of the line sudo depmod -a which should solve some of them but I need to test at the next kernel change.
Ubuntu 9.10 karmic Koala is not a LTS (Long Term Support) version so one has to ask if the advantages are sufficient to merit an upgrade when a LTS version is due in 6 months. I have upgraded 3 of our 5 machines each for different reasons. The Toshiba Satellite Pro L20 lost a hard drive so a new install was required anyway and 9.10 had less problems than 9.04 and was better than 8.04. The HP Compaq DX2250 needs the most up to date versions of software as it is used for Video work and the home build was in need of a reinstall as it had been progressively upgraded from 6.06 to 8.04 Hardy Heron and also needed more disk space and repartitioning of the disk drives. The MSI Winds were not upgraded as there were issues with Webcam support and also random Brightness variations. These seem to be a chip problem and the surprise is more that they work perfectly under Jaunty! I will wait for the next LTS version as they are working well at present.
So what are the major changes:
There are many changes under the hood (hal and pulse audio to name a couple) and some have had unexpected results on some systems - it is an essential development step before the next LTS version but it does mean that you should check out carefully with a LiveCD version before upgrading. If you are happy with what you have then it is prudent to stay with it until you have had a chance to do some comprehensive web searches for peoples experiences with your own machine and also to read the Ubuntu Release notes with great care. That said you will find that the improvements are well worth getting used to a few changes.
This has been a good test of my own procedures and I have updated them where possible.
The legacy Grub basically used only one configuration file which needed to be customised, namely /boot/grub/menu.lst. Grub 2 uses /boot/grub/grub.cfg which is normally not edited as it is automatically generated by /usr/sbin/grub-mkconfig using templates from /etc/grub.d and settings from /etc/default/grub . There is a vast amount of information at https://help.ubuntu.com/community/Grub2 and the following is just enough to get started on configuration. There is also a program which you can install which will do some of this - use synaptic to to search for startupmanager and install it - but it is very basic to that under the legacy Grub.
/etc/default/grub typically contains:
# If you change this file, run 'update-grub' afterwards to update
GRUB_DISTRIBUTOR=`lsb_release -i -s 2> /dev/null || echo Debian`
# Uncomment to disable graphical terminal (grub-pc only)
# The resolution used on graphical terminal
# note that you can use only modes which your graphic card supports via VBE
# you can see them in real GRUB with the command `vbeinfo'
# Uncomment if you don't want GRUB to pass "root=UUID=xxx" parameter to Linux
# Uncomment to disable generation of recovery mode menu entry
GRUB_DEFAULT=0 will boot the first menu item and so on. GRUB_DEFAULT="saved" will boot the same entry as last time.
GRUB_TIMEOUT=3 will display for 3 seconds
After making any changes you must run in a Terminal:
See https://help.ubuntu.com/community/Grub2#Configuring%20GRUB%202 for more options
The set of configuration files in /etc/grub.d are run in order to set up and build up the Grub menu.
30_os-prober finds and adds all the other operating systems - it can be inhibited by setting its permissions so it is not executable. You can then customise the other operating systems by adding them to 40_custom having had a look in /boot/grub/grub.cfg to see and copy what you want. If you want to inhibit display of the memory test options then make 20_memtest86+ to be non executable.
There is currently no way to set the number of kernels which are displayed as in legacy Grub but there is an interesting article which shows how to do so at http://www.linuxquestions.org/blog/drask-180603/2009/12/5/howmany-for-grub-2-2466/
If you reload Windows then it will overwrite the Grub2 bootloaded. I have only reloaded once to make sure there was a fresh copy of the MBR after a virus attack with a MMR trojan on a Windows machine. The proceedure I used used and will use for reloading Grub2 is based on https://help.ubuntu.com/community/Grub2#Configuring%20GRUB%202
First you must find out the device name/partition of the installed system (sda1, sdb5, etc). This partition is then located and mounted from the LiveCD. The files are then copied from the LiveCD libraries to the proper locations and MBR.
Boot a LiveCD (Ubuntu 9.10 or later for Grub2).
Determine the partition with the Ubuntu installation by typing in a Terminal
sudo fdisk -l
The device/drive is designated below by sdX, with X being the device designation. sda is the first device, sdb is the second, etc. In most cases the MBR will be installed to sda, the first drive on their system. The partition is designated by the Y. The first partition is 1, the second is 2. Note the devices and partitions are counted differently. In my case the Linux root file sytem is typically on /dev/sda4 as I have Windows and data filestems below it
Mount the partition containing the Ubuntu installation by:
sudo mount /dev/sdXY /mnt
eg: sudo mount /dev/sda4
Run the grub-install command to reinstall the GRUB 2 files on the mounted partition to the proper location and to the MBR of the designated device.
sudo grub-install --root-directory=/mnt/ /dev/sdX
eg sudo grub-install --root-directory=/mnt/ /dev/sda
Refresh the GRUB 2 menu with
We have been using Unison to synchronise files held in a set of folders between all our machines with success for many months. The main folders are My Documents, My Pictures, My Web Site, Web Sites, My Teaching and Pauline's Documents.
As well as simple files we also need to synchronise/transfer our emails, browser settings and secure information between machines for backup and for when we go away. The Firefox Browser information and the Thunderbird Email, Contact and Task information is held in Profiles. These are folders which have a consistent set of information and synchronising would move files in both directions and destroy the consistency. They have to be copied as a whole between machines - mirrored rather than synchronised.
A further problem is with encrypted file systems produced by truecrypt - these are saved as a single file and the size and date remain unchanged for security and deniability reasons so the only way one can tell they have changed is by a full comparison which is slow. Again we probably want to mirror them. I have added a lot of comments into the template for my synchronisation 'profile' for unison. So I will just provide a copy below.
#The paths are defined by regular expressions which ensure that all the child folders - the
#folders underneath the source - are also mirrored
# The definitions are confusing and the 'root' which is being mirrored is the one that follows the -> for example
# forcepartial = Regex Vaults/.* -> ssh://pcurtis@wind-ubuntu//media/DATA
# mirrors the folder media/DATA/Vaults (and all its children) onto the local machine from wind-ubuntu
fastcheck = false
forcepartial = Regex satellite-ubuntu/.* -> ssh://pcurtis@satellite-ubuntu//media/DATA/Profiles
forcepartial = Regex triton-ubuntu/.* -> /media/DATA/Profiles
ignore = Path wind-ubuntu
ignore = Path vortex-ubuntu
ignore = Path matrix-koala
#Note - we only force these changes between the two machines which have the 'masters' at the time - and ignore updating the others.
# When fastcheck is set to true, Unison will use the modification time and length of a
# file as a ‘pseudo inode number’ when scanning replicas for updates, instead of reading the full contents of every file. Faster for Windows file systems.
# fastcheck = true
# Note: fastcheck must be false to detect changes in encrypted truecrypt volumes as the file size and modification dates are kept the same.
# When times is set to true, file modification times (but not directory modtimes) are propagated.
times = true
# When owner is set to true, the owner attributes of the files are synchronized.
#owner = true
# When group is set to true, the group attributes of the files are synchronized.
#group = true
# The integer value of this preference is a mask indicating which permission bits should be synchronized.
# In general we do not want to synchronise the permission bits (or owner and group)
perms = 0o0000
The file structure on my data partition which is mounted as DATA is shown below - the names should make most of it self explanatory
If you have profiles for Thunderbird/Lightning which you have set up in Jaunty or earlier you may find that all the calendars seem to have disappeared when you change to Karmic Koala which is very disturbing. This is because Lightning is now integrated into Thunderbird in Ubuntu rather than just being in the profile and this intergration also includes the extension for Google Calendars. These extensions can not be in two places at once and even if you have not installed it in Ubuntu explicitely the framework is still in place and the extension in your profile conflicts with it and the calendars disappear from view - this is a bug in my book but once you realise what is going on there is, at least, a simple workround.
The workround this is relatively simple if you just have a single profile. First you need to close Thunderbird then uninstall lightning-extension using the Synaptic Package Manager (if it has been installed) - I did a full uninstall which removes all the control files. Once you have removed the conflict you now run Thunderbird with each of the problem profiles and Uninstall the Lightning 0.9 extension using the Extension Manager (Tools -> Add-Ons -> Extensions) - do not fear it does not remove the calendar data from the profiles. When this is complete you can reinstall lightning-extension in Ubuntu which brings in calendar-google-provider and calendar-timezones as dependences with it and the calendars will now reappear when you open Thunderbird. If you have multiple users you should clean out each users profiles before reinstalling with Synaptic.
There is a bug in Karmic https://bugs.launchpad.net/ubuntu/+source/gnome-power-manager/+bug/432598 which means that the suspend/hibernate disabling through gconf-editor do not work. The following is a short term fix from http://ubuntuforums.org/showthread.php?t=1305081 - the options still show up in the menu but if you click on one of them it just locks the screen.
sudo gedit /usr/share/polkit-1/actions/org.freedesktop.devicekit.power.policy
And change the code:
entries for suspend and/or hibernate to:
I have recently been trying to rescue a seriously virus infested machine which contained valuable business related information which had to be extracted and I also needed to ascertain how much potential damage had been done.
I knew the machine well and knew that care had been taken to keep it protected at all times. The first I heard was an email asking how to check if the virus checking which was working followed shortly afterwards by another saying that the existing virus checker had been replaced by the latest McAfee and a single virus had been detected and quarantined but could I have a look as the machine was now behaving very oddly.
This is a very long story so I will anticipate some of what comes and at this point put forwards what can only be a supposition as much of the evidence was hidden destroyed by the viruses and the removal tools. The bottom line is that there is a significant chance that the machine had been infected for some time and that the existing virus checker, firewall etc had been compromised. One virus source file had a time stamp from 7 months earlier but that could have been deliberately false. The major problems occurred when the latest version of the virus checker was loaded - it seems it was was also quickly neutralised and a fresh load of less stealthy viruses loaded almost as if it was a malicious attack once the earlier stealth had been detected. The best guess is that the payload had been the banking password stealer detected and the problem was hidden by a rare Master Block Record Rootkit which was one of those detected - this works before even the earliest boot-up checks come into play making it almost impossible to detect once it is in place.
By the time I got the machine, which had only run for a few hours after McAfee was loaded it was riddled and not only the new McAfee installation but most of the usual tools were compromised, some problems were immediately obvious, some I found as I progressed:
Additional considerations were:
This looked a fairly hopeless task but I found that I did have access to the command line and Run and msconfig was still alive so I could kill some of the more suspicious start up programs and eventually loaded the Avast 4.8 Virus checker I have found so effective in the past and SpyBot Search and Destroy 1.6 which I have come to depend on for Malware. SpyBot has the useful option of using a file of recent updates rather than updating over the internet which I took advantage of. Avast took out enough in its initial runs to make progress but many of the files and viruses identified by it and SpyBot could not be removed as they were locked or in use and the startup removal options had been blocked.
I therefore loaded a LiveCd version of Ubuntu 9.10, again it was flaky which I now realise was because of the MBR contamination as even the LiveCDs access disks to determine the disk structure and allow for mounting when it is running. I also used a LiveCD of Parted Magic which does not read the disks during set up to delete a number of the files and step forwards a little.
The big step forwards was when, after a lot of internet research, I found that another and less well known command line tool gpedit.msc was still active and this enabled me to get to a 'policy editor ' which is used to enable and inhibit the use of the registry editor and File Options. Such a tool was needed as the use of the registry editor is controlled by the registry which is bad news.
At this point I could insert a USB memory stick and use it to transfer programs. These included ImgBurn to allow me to write and verify a CD/DVD with the all important .pst file from Outlook with all the Emails, Address book, Tasks and Calendar although I could not be totally sure they were virus free.
I am not quite sure what was the critical activity which restored a lot more functionality. Up to now I could download some files but only in short stages and Firefox did not display the expected download windows and installing usually failed. Windows Update had started working and I think my install of Internet Explorer 8 overwrote or reinstalled some system files and restored full Internet Access, program downloading and program installation. This could equally have been the removal of one of the viruses.
I had by now given up most hopes of saving the machine without a complete re-install from a disk image so I now looked to a long term solution and partitioned the drive to give a NTSF Data partition and three partitions for a Linux root (/) ext3 file system, a separate ext3 partition for /home and swap file. I then installed Ubuntu 9.10 Kalmic Koala with the Grub2 bootloader to dual boot Windows and Ubuntu Linux system. I has some difficulties with the install when it came to the inbuilt partition editor which I now believe was the result of the MBR being virus contaminated but once I had got past that the MBR was overwritten as part of the install of the Grub2 bootloader. I reloaded the Grub2 loader and the MBR again after I discovered the trojan.mebroot virus as a precaution.
This enabled me to nail a few more viruses either directly or by deleting files by booting into Ubuntu. I now loaded the free version of PC Tools Spyware Doctor which is very good at finding viruses and spyware and found 11 more which I removed by registry edits and file deletions from Ubuntu - that brought the total up to just over 100 (one hundred) infections which I removed I had removed and operation was almost back to usual. I also found that the wininit file was reloading 4 of them and that had to be deleted.
PC Tools Spyware Doctor then briefly flagged a file being accessed as a trojan as Avast was doing a full scan (but not flagging it as a virus) and that turned out to be the first hint about the MBR Virus Trojan.mebroot being present. To quote F-Secure "This MBR rootkit Trojan.mebroot is very advanced and probably the stealthiest malware we have seen so far. It keeps the amount of system modifications to a minimum and is very challenging to detect from within the infected system." The first detections by the virus firms seems to have been on November 28th with one detection in Mexico and one in the USA, both 5 days after it hit the machine I was working on so no wonder I had a hard job. I located the file and did a search for files with the the same date and found 10 more which I deleted in from the dual booted Ubuntu sytem. The location of these files also revealed that the remote control facilities allowing complete control of the machine were turned on.
Sophos seemed to have also picked it up on Trojan.mebroot and had a free for 30 days Malware and AntiVirus trial so I downloaded both there rootkit Detection and AV packages, registered and installed them. The Rootkit tool detected nothing more but the AV scan found three more viruses (4 files) which were still in the Linux Trash folder from my own search which implies there were a few more they do not yet detect or have knowledge of. The machine is now very slow to startup (lots of AV checks going on??) and log off.
Another example of the Viruses found is Mal/EncPK-LT which is a banking trojan that disables firewall, steals sensitive financial data (credit card numbers, online banking login details), makes screen snapshots, downloads additional components, and provides a hacker with the remote access to the compromised system. It has the typical stealth-mode characteristics common to Rootkits and downloads/requests other files from Internet to keep updated and introduce new threats.
Contains characteristics of an identified security risk. All of the Zbot viruses found have similar characteristics.
A bluescreen problem during shutdown turned out to be unrelated and due to a Sony DVD Video camera driver conflicting with the updated SP3 version of XP.
Actual independent viruses found have been:
Also many tacking cookies and adware examples removed
Also 20 Tracking cookies and adware examples still in place.
During the period the machine was infected it had been kept up to date as far as Microsoft Automatic Updates to Windows XP including Service Pack 3 but not the Office Updates so that was turned on and Office 2003 and One-Note 2003 were updatedand Windows Defender was also loaded.
The final state is, I believe a safe but somewhat damaged machine which needs to be reloaded once all the information is out of date and an audit trail is no longer required. Any bank account passwords and other passwords need to be changed and it is possible that the machine was under remote control at periods. Several of the various pieces of software used will need to be removed or replaced if the nmachine is ever used again under Windows as a business rather than home computer.
In parallel the machine can be safely used under Ubuntu Linux as it now has a dual booted system. The viruses do not affect Linux but I am wondering if it may be worth adding a Windows Virus checker to protect others from forwarded problem files and emails.
The total time taken was circa 62 hours spread over two weeks.
Final thoughts: the task took much longer than I had imagined would be the case and the infections far worse. It is another indication of the problems in using any Windows system however much care you take over security - once one virus is in then you have little hope of recovery other than a lot of time in the hands of an experienced professional. In this case there would have been almost no chance of recovery without running much of the recovery from a LiveCd then dual booted Linux system which raises the question of why one should not run with a refined version of Linux like Ubuntu from the start. I had to do almost everything in by Article "The Road to Freedom - A progressive migration from Windows to Ubuntu for Safety, Security and Savings in Home Computing" just to rescue a system for a while until it is compromised again. There is an old saying "You can not make a silk purse out of a pigs ear"
Times in Minutes:Seconds
Ubuntu 9.10 Kalmic Koala Switch on to full desktop and disk activity finished 1:07 and shutdown from empty desktop 0:11 Total Cycle 1:18
Windows XP Professional SP3 Switch on to full desktop and disk activity finished 18:00 and shutdown from empty desktop 2:35 Total Cycle 20:35
This needs a little breakdown and Windows took 3:30 to a visible desktop, 9:40 to a website loaded in Firefox and 12:07 to the Firewall being active - note the gap.
The times are so slow under Windows because the machine was a state of the art tablet computer 4 years ago but with only 512 Mbytes memory which is totally inadequate for XP Pro Service Pack 3 with redundant Virus Protection, Spyware Protection and a full Firewall whilst Ubuntu will run in 256 Mbytes and is reasonably nimble with 512 Mbytes and, of course, needs no virus protection and is designed without open ports.
We have been setting up our Xmas mailing lists using the Mailing List facility in Thunderbird. We found there were a number of unexpected features:
There are a number of Virus checkers which run under Linux which are for detection of Windows Viruses. A major reason for there existence is to check for viruses on Email Servers running under Linux but most will also run checks on files and folders. It is therefore possible to check a Windows filesystem on a dual boot machine or from a live CD when it is not running. Many of the virus checkers are from the same providers who make them available for Windows and are, of course, proprietor. These include free versions of my two favourites for Windows, Avast and AVG. Instead I am using the open Source Virus checker ClamAV and its Gui interface ClamTk, they have a good reputation and have a huge virus database which is updated rapidly when new viruses and are detected. The next paragraph has a brief description of ClamAV and its associated packages.
Clam AntiVirus is an anti-virus toolkit for Unix. The main purpose of this software is the integration with mail servers (attachment scanning). The package provides a flexible and scalable multi-threaded daemon in the clamav-daemon package, a command-line scanner in the clamav package, and a tool for automatic updating via the Internet in the clamav-freshclam package. It features built-in support for various archive formats, including Zip, Tar, Gzip, Bzip2, OLE2, Cabinet, CHM, BinHex, SIS and others. It has built-in support for almost all mail file formats and for popular document formats including Microsoft Office and Mac Office files, HTML, RTF and PDF. The virus database is normally updated by the clamav-freshclam package which is automatically loaded alongside ClamAV for Internet updating. An update package package can also be created on an Internet-connected computer and run as a .deb. We do not plan to do any on-access checking or automatic checking of incoming/outgoing email at this point in time so we are not running the daemon and have no overheads other than a small amount of hard disk space unless we are running a specific check.
Ubuntu has all the various packages in the repositories so they can be installed by the Synaptic Package Manager or it can be installed along with a simple but adequate GUI for file and folder testing called clamtk from Add/Remove programs as 'Virus Checker' . The standard install also brings in a package to add virus checking as a right click option in the file browser. I installed it and used the Gui (ClamTk) to check a Windows system on a dual boot computer - you first need to mount the partition with the Windows 'C: Drive' which needs administrative privileges and a password. It automatically updates and can give a comprehensive check - it was slow on an entire 'C: Drive' but found the 'test' files I expected. It also had a number of false alarms detected as PUA.Packed.aspack212, mostly in my Irfanview .dll files which I have been using for years so they were obviously false - two more detections needed a little more investigation but again I concluded they were false but will watch them anyway. The false alarms only occurred when checking using the advanced mode for Potentially Unwanted Applications PUAs which seems to use a heuristic method - this is a known anomaly which I am sure will be solved.
We will ClamAV it to check files from students and others where we want to avoid any chance of passing existing problems on to other users.
There is an updated version of the ClamTk GUI which I downloaded as a .deb and installed as it seemed to offer some better features but I would not regard that as essential. Unless you are very knowledgeable keep to the version in the repository.