Home Uniquely NZ Travel Howto Pauline Small Firms Search
Diary of System and Website Development
Part 11 (May 2007 - September 2007)

24th May 2007

Summary of Changes since October 2003

It has been many years since the diary has been kept up so I will start with a brief summary of the major changes which have occurred in the period since it was last written up.

Desirable Changes to Web Site

This is a list of changes being considered.

Where possible these have been incorporated in this page

29th May 2007

Ubuntu installation on a Legacy Desktop

This is the first real entry into the Diary for many years. When I make changes to my, or friends, systems I make notes as I proceed. The following are from the notes I made at the time and will be edited further before adding to the Fun with Ubuntu Linux page.

This section covers the installation of Ubuntu onto a friend, Dugald's machine. It comes into the legacy class as it does not have the latest hardware such as USB 2.0 or a network connection although a Wifi 802.11b adapter has been added and extra memory. It has an AMD Athlon 1.4 Ghz Processor, 256 Mbytes memory, Matrox MGA G200 AGP graphics, Netgear MA311 802.11b wireless adapter, 2 USB 1.1 ports and two EIDA drives of 40 and 10 Gbytes, Ricoh CD-R/RW MP7200A, LPT and two TTY ports and Creative Audio compatible sound. This is a more than adequate specification to make installing Ubuntu Linux worthwhile and will give a new lease of life to a machine which was floundering under Windows xp and Norton. Tests with the LiveCD showed that most of the the hardware, including surprisingly the Wifi card was recognised so we proceeded to install Ubuntu Dapper Drake from the version 6.06.1 LiveCD.

The built in disk partitioner on the Ubuntu CD had problems with making changes to the NTFS formatted 40 Ghz primary drive so the decision was made to leave the primary drive alone and make the Linux installation onto the second drive of 10 Ghz capacity previously the Windows D: Drive and that was emptied. The intention was to leave Windows XP on the old C: drive and create a new partition formatted with FAT32 which could be shared at a latter stage. The install was the straight forwards and the secondary drive could be reformatted completely and Ubuntu worked OK but the Windows XP failed to boot - searches indicate this is probably because it had Norton Go-Back installed and makes changes to the Master Boot Record and Windows start up sequence. We concentrated on Ubuntu initially but Windows XP was latter reinstalled giving an interesting comparison in time and difficulty.

Ubuntu Wifi Internet and Network Access: The only network connection is by Wifi on Dugald's machine which has a Netgear MA311 802.11b wireless adapter which is based on a Prism 2.5 Wavelan chipset. Before installing we had checked using the LiveCD that it had been recognised by Ubuntu and shown to work in a basic way with an unencrypted link. This gave us confidence to install Ubuntu. Once Ubuntu was installed it was found that it could not be set up for WEP or to get an IP etc by use of Network Manager alone. I had experienced this before with several other Wifi cards and had spent months getting some understanding of the problems but this was the first time I achieved a satisfactory long term solution, at least for a single network.

There were several steps needed to get WEP going:

Firstly use System -> Network Manager where it had found the Wifi and to set up it up as eth0. I set up the properties to the following:

SSID (This was found and available for selection on the drop down menu)
Static IP address
IP address of 192.168.1.8 (where x is well above any that might be issued ie 8)
Subnet mask (filled in automatically) to 255.255.255.0
Key type Hex
WEP key to abcd1234ghjk4567liuy5678as (seemed to be ignored!)
Gateway of 192.168.1.1

DNS of 192.168.1.1 and 217.115.138.24 added on DNS tabs - the second is a open community run DNS server added on a belt and braces basis but probably redundant.

Secondly open a terminal and type

sudo ifconfig eth0 up
sudo iwconfig eth0 key abcd1234ghjk4567liuy5678as

This enables the card and sets the WEP string and the card then worked fine but the settings are lost when the system is rebooted

My final stage was a bit of a bodge to set up a script to implement the above commands and run it at startup. The system runs many system scripts at startup and on Ubuntu they are accessed by symbolic links in the /etc/rc2.d folder (for normal startup at run level 2) to scripts generally kept in the /etc/init.d folder

I therefore did a quick and dirty fix by adding an extra symbolic link to my own script by opening a terminal window and typing

sudo ln -s /home/dugald/bin/StartWifi /etc/rc2.d/S99wifi

The script file StartWifi, needs to be owned by root and with permissions set up for everyone, I found the easiest way to create this was to use the file browser nautilus but opened as root by typing

sudo nautilus

in a terminal window. I then right clicked in the home directory and created the folder bin, opened the folder and then created the file StartWifi and immediately right clicked to properties and the permissions tab where I set read write and execute for everyone. I then double clicked to open it and selected edit to add the contents namely.

#!/bin/bash
ifconfig eth0 up
iwconfig eth0 key abcd1234ghjk4567liuy5678as

and saved it. Note that this is going to be run by root at startup so the sudo is not needed.

I chose ~/bin as the home as this was already a gross bodge and ~/bin is the normal home for user scripts. I therefore also took the step of adding ~/bin to the search path for scripts by modifying ~/.bashrc by appending the following lines:

# Additions to the end of the standard ~/.bashrc file
# to set up path to /bin directory in home folder
PATH=$PATH:~/bin

to the end of .bashrc for the future use of scripts

Accessing the Windows partitions: The next stage in configuration was to provide access to the Windows partitions by modifying the /etc/fstab file. The changes below are for the final configuration with two FAT32 partitions on the primary drive and the second was to some extent determined by trial and error as the formatting had been done after the install had set everything up - normally they would be more predictable and as in my main Fun with Ubuntu Liux page. I need to investigate more fully why the second needed to be idedisk and nothing else as a folder name - perhaps the folder has to exist and have permissions set?? This supposition has not been tested but one of my books confirms it is the likely cause so the procedure for mounting a new Windows (or any other new partition) should also include making the directory in /media - again the trick of using sudo nautilus gives a GUI interface to do it.

sudo cp /etc/fstab /etc/fstab_backup
sudo gedit /etc/fstab

and appended the following lines at the end of file

/dev/hda1 /media/hda1 vfat iocharset=utf8,umask=000 0 0
/dev/hda5 /media/idedisk vfat iocharset=utf8,umask=000 0 0

Network Printers were detected and installed with no problems using Administration -> Printers and following the wizard. Drivers were built in for both printers.

Restoring Windows: As I said above the Windows installation had been damaged so Windows XP was reinstalled. This overwrites the Master Boot Record which meant that Grub was no longer called and needed to be re-enabled. As this is a useful procedure I am giving it a separate section.

29th May 2007

A quick and easy way to re-enable or install Grub

You can restore GRUB after a Windows installation by following the steps below which are an amalgamation of information from the Ubuntu site and other places:

This is all you need to do if you had initially installed Ubuntu into a Windows system. If you are adding Windows then Ubuntu will not have anything in the grub configuration for Windows and you’ll have to edit the grub boot menu file.

Open the file /boot/grub/menu.lst with the following command having backed up the original by:

sudo cp /boot/grub/menu.lst /boot/grub/menu.lst_bak_1
sudo gedit /boot/grub/menu.lst

You’ll see a sample section for Windows, which you’ll want to un-comment and add to the boot menu list in whatever position you want it in. (un-comment by removing the #’s) so, for example:

title Windows XP
root (hd0,0)
makeactive
chainloader +1

The (hd0,0) assumes that Windows is installed on the primary drive and the first partition. If you had installed Windows somewhere different then it should reflect that.

30th May 2007

Comparisons between installing Windows XP and Ubuntu Linux onto a legacy machine.

Having had problems which resulted in the lose of the Windows XP operating system from Dugald's machine it was only fair to reinstall it. The first step was to replace the Windows XP bootloader to the Master Boot Record (MBR) to see if that was the problem. It booted Windows to the point where Norton Go-Back used to kick in and asked what recover mode one wanted then rebooted automatically in an everlasting loop just as it did with Grub. The procedure above was tested at that point to ensure that the Ubuntu system still remained and could be accessed by reloading the MBR and Ubuntu was available. This was then the point to decide on whether to repartition the primary drive before reloading Windows. I decided to repartition the drive using a LiveCD version of QtParted which would not change NTSF partitions but allowed me to delete the existing partition which was replaced by a Primary Partition for the system and an extended partition containing a partition for Data Both were formatted as FAT32 so they could be also be mounted and read and written from Ubuntu Linux - NTSF partition can be read without difficulty but writing can be problematical. The partition arrangement was different from that when Ubuntu was installed and caused some extra configuration as discussed above.

The original Windows XP OEM CD was still available and used to avoid any problems with the CD code for installing and for registration (The CD code was still on the side of the box). The CD was for the very original version of XP prior to service packs. This was installed following the screen instructions in about 45 minutes and the system booted and ran very fast compared to latter versions. Regrettably it is no longer supported so Service pack 2 needed to be applied.

The first task was to get internet access and the CD for the Wifi card was fortunately still available. The drivers were installed by Control Panel -> System -> Device manager -> Properties for the card and finding the reload drivers button. The install including search CDs was used after the CD was inserted and went smoothly with full networking being available immediately and the WEP code was requested at the first attempt to connect. This was much easier than Ubuntu because a driver disk was available in this case and the Ubuntu drivers did not support WEP with bugs being sorted - it had been just as fast on my laptop where the card had better support in Ubuntu. Overall about par in time and complexity so far.

Now came updates - in the case of Ubuntu all the updates were downloaded and installed in one go, about 200 Mbytes of them but that included all the installed and uninstalled programs forming part of the distribution. The time was largely that of the download. The system had to be rebooted at the end as the kernel had been updated. For Windows XP, the first activity was to get the service pack - I prefer to download it and then run it and once it is in ones hands and blow to a CD it can be used many times. It was a again a download of just over 200 Mbytes which took longer than the whole Ubuntu Distribution updates to install. I then let the automatic updating take over and it took about 5 stages, each involving some interaction and complexity in confirming it was a genuine copy and signing up to long sets of terms and conditions. The total of over 80 critical updates and updates of updates took many hours and 5 reboots.

Once the system was updated it still needed a virus checker and firewall and had of course been vulnerable for many hours. I downloaded the free version of ZoneAlarm which I prefer to other firewalls (35 Mbytes) and the free version of the AVG virus checker (25 Mbytes) which is easy to install, updates automatically and does not add a huge processing overhead. I would normally also install Microsoft Windows Defender (10 Mbytes) but that does impose an addition overhead on a legacy machine.

In Ubuntu it took all of a minute to start to print a test page on each of the Networked printers on Windows machines - I did not succeed under Windows in the limited time available but I have done it many times before.

Under Ubuntu I could quickly read shared files on all the Windows machines on the Network and read and write to Windows partitions in FAT32 (read only under NTSF) on the dual booted machine and 5 minutes on th command line set up automatic mounting when the machine booted.

In conclusion the reinstalling of Windows XP went very smoothly, all be it slowly, and the end result was a much faster set up than before because it had been a clean install and Norton which is an unbelievable performance killer was replaced by free software for the firewall and virus checker. Ubuntu of course does not need either. The initial install of Ubuntu took longer because of the time testing with a LiveCD and sorting the Wifi out but a reinstall and update would take about half the time of Windows XP and updating (might be two thirds if you have a Windows XP service pack 2 CD). One must also remember that all the Office Software is also included in the time whilst Microsoft Office or preferably Open Office was still to be installed and updated. I did not do timings or booting of Windows XP and Ubuntu but I think XP was slower by the time the firewall and virus checker were up and running. The reinstall was however much faster than before I (or more likely Norton Go-Back wrecked the previous installation). Overall on this legacy machine I think Ubuntu won out in ease and speed of installation and configuration once the Wifi card bug had been cracked. The performance is better and it does not need perpetual firewall and virus checker updates and has far less critical security updates. A worrying aspect is that neither could have been updated without broadband and Windows can not be kept secure without broadband.

What lessons have been learnt here and earlier on setting up dual boot systems:

22th June 2007

Updates and more Updates to Windows: Back home after a month away. All Windows XP systems had extensive updates downloaded which wasted about three hours. Microsoft system updates needed restarts and both virus checkers and firewall updates needed interaction and restarts. In addition two of the McAfee virus checkers had run out their year and would have needed to be changed so I loaded the free AVG virus checker which I have found works very well and is free of interactions - full system scan found one dialer in a old backed up install program which McAfee had missed.

Ubuntu Updates: In contrast I checked for Ubuntu updates - there was a fair number but they only took about 6 minutes to download and another two or three installing. It involved a kernel update so it needed a restart which is very unusual, even so it took about 10% of the time and interaction of the Windows systems!

23th June 2007

GPSBabel: Looked at GPSBabel web site - recommended in an email about our GPS page - runs under Windows and Linux so well worth a further look and trial when we get time.

24th June 2007

A few additions made to the Cascading Script File to define some blocks for displaying Code, Terminal Input and a general block for emphasis. The additions follow displayed in the new 'Code Block'. The dotted border is to show it is an excerpt.

.terminal {margin-right: 5%; margin-left: 5%; display: block; border-style: solid; border-width: 1px; border-color: #b0b0b0; font-family: "Courier New", Courier, mono; font-weight: bolder; font-size: .87em; overflow: auto; background-color: #d8d8d8; padding: 10px; }
.code {margin-right: 5%; margin-left: 5%; display: block; border: 1px dashed; font-family: "Courier New", Courier, mono; font-weight: bolder; font-size: .87em; overflow: auto; background-color: #e8e8e8; padding: 10px; }
.generalblock {margin-right: 5%; margin-left: 5%; overflow: auto; border-style: solid; border-width: 1px; border-color: #b0b0b0; background-color: white; padding: 10px; }

Terminal Input

General blocks of text which needs to stand out

The code above has been added to the general.css file for the site and they have been used to update the sections above.

26th July 2007

Web Authoring

I have just been contacted by a friend who has a large and well known web site with some questions which have made me think it is time to put down some thoughts on Web (HTML) Authoring.

The emailed comments and questions which have set me off were: " Initially, I constructed my web pages using basic html instructions. Then I discovered, first, the Netscape editor and then MS Frontpage. I was aware that these introduced extra coding but had not realised how much. I began to look at one page that had got out of hand. The size was around 460 Kbytes. I have got it back to about 60 Kbytes and still have a bit to go. Is there an editor that does not add extraneous instructions? Or am I really stuck with hand coding html?"

This sounds a simple question but one that needs a fairly long answer to enable one to match what is required to what is available OpenSource, free or commercially. My reply was written in a way that I hope will form the basis of a new web page.

HTML Authoring and Tools: I will deliberately try not talk about just editing in the case of HTML as what one is interested in is the result as viewed in a browser rather than the HTML 'code' and all the associated scripts (such as JavaScript or PHP) and layout such as Cascading Style Sheets (CSS). It has, in fact, become common to talk about an HTML 'toolbox' which includes all the rest of the tools needed to produce, web pages and sites and optimise, test and maintain them. This may include software for producing/optimising graphics, checking the code is compliant with various HTML/XHTML standards and the browsers in use, optimising and tidy the code, checking links and for orphan pages etc etc. A lot of HTML 'editors' include some of the these tools as well as other useful/essential functions such as spell checking.

For a long time I think you were coding your pages in the most basic and arguably the best way by using a text editor and for an example I will continue to use making a phrase bold by typing in a <b> at the start and a </b> at the end. The advantages are you know exactly what you are doing and will get the shortest length possible.

The next level of HTML editing is to use software which knows about all the common HTML functions and parameters so when you want a bit of bold text you select it and click on a button, use CTRL B or use a menu to make it bold just as in a word processor. These code editors will also allow you to set up links, add pictures, etc just as easily and will have file selection available. You again get efficient code and can see exactly what the code is that has been generated. Most of them will also parse you code and show tags, attributes etc., it in different colours so if you get mismatched quotes it will be very obvious but not some of the more subtle errors. Most of these 'HTML Code Editors' also allow you to include scripts and edit script files with colouring to show how they parse and also include Cascading Style Sheets and edit CSS files. They will often include some form of spell checking.

This is the minimum level of HTML 'editor' I would contemplate, there are no disadvantages in terms of coding efficiency or extraneous code and they enable one to find/avoid some common errors. You will also be able to work with several files open in a tabbed view. A Freeware example that does this and a lot more is Arachnophilia which I have used on and off for many years in its various versions. It also runs under Linux!

Some of these authoring aids which work with the code visible also allow you to see the result as rendered in a browser in a split screen or by switching mode. You will probably have to manually refresh it after changes but, because they use a existing browser on your system, they will render scripts and CSS. Even if the do not have this function you can have one or more different browsers open and just save the page you are working on and refresh the browser window.

If you are prepared to pay then the HTML Code Editors become more sophisticated and will have contain many of the tools you need in the package. For example there are often tools to lay out the code neatly to aid visibility and updating in the future. Good layout with indenting may add a little to the code but makes life much easier and errors become more obvious. hey will show you all the parameters and attributes available in a tag and help use of Style Sheets when you need them. There will often be an ability to manage and check a whole site for missing links and unlinked files. Another very useful tool is one for global searching and replacement of open files and over the whole site. I have bought and used Hotdog for many years and the version I have currently (6.6) is a bit sort on support for scripting and some of the latest gizmos but has been very reliable.

Where difficulties really start is when you use a WYSIWYG (What You See Is What You Get) HTML editor. Few generate good HTML code and some are atrocious such as Microsoft FrontPage and the new and innocent never realise until they have complete web sites committed. I suspect that Bill has already been responsible for setting back progress by several years by his bug ridden, insecure, inefficiently and non standards compliant software which most of the world uses because of his monopoly. I will not even start on by views on what may happen as he expands from training to education, already a University degree is of less importance than a Microsoft certificate in the view of many deluded employers. Microsoft Front Page is an example, just look at the resulting code after a period of development and think of the inefficiency in the web and information interchange when the code is several time longer than necessary.

However many people do not want and should not need to work in HTML code just because of one firms apparent dominance and near monopoly in other areas. There are still alternatives both free and commercial but most important one should look at requirements. I believe that the WSYIWIG mode should be an addition to a code view, however good it is there will always be a need for the advanced user to inspect and occasionally still do some hand crafting. I, for example, use Javascript to generate some code 'on the fly' to save repeatedly re-entering a piece of code/text which has several repeats for my popups where there are title and alt attributes for the icon and the popup which are the same. I have not found any WSYIWIG HTML editor yet which will render them or easily enter them - this is an extreme example and if I had started with a WSYIWIG editor I might have done things differently.

The unchallenged leader for HTML authoring is Dreamweaver. I have used limited life copies of Dreamweaver MX 2004 fairly extensively and it is very good. It is better than anything else I have used in code view but you rarely need code anyway as it is so good in its WSYIWIG mode and it offers good support for CSS so when you click bold you can have it set up to use a style instead so if you ever decide bold should not only be dark but in pink a single change in a Style sheet will do it for the whole site! I have not used it extensively for active pages using PHP but know even the 2004 version offers support. So what is the catch? - cost is the problem, you pay about £400 for basic Dreamweaver and up to a £1000 if you include the Firestarter graphics and other accessories. If you are a big user or have a commercial site it is worth it and it is so good I regard it as one of the only valid reasons to continue using a machine under Windows. One way to get cheaper copies used to be offers associated with training courses but that may no longer be an option.

There is an alternative which is also very good for most purposes called NVu, pronounced New View which was originally written for Linux but also has a Windows Version. It has a code view as well as a WSYIWIG display and also a WSYIWIG display which also displays the HTML tag positions. What are the downsides, it can be a little idiosyncratic, for example, the code view allows one to edit but seems to display a tidied up and formatted version whilst the real code is different and can gain a lot of extra white space without you realising although there are tricks to clean it up or you can run a separate compliance checking and code formatting program occasionally. NVu is my choice under Linux, which I use most of the time, with an occasional use of the Tidy program provided by the World Wide Web Consortium which checks compliance as well as formatting the source for maximum efficiency, Dreamweaver has the edge but not one worth £400 for a personal site.

17th August 2007

Pauline now has permission from DTI to publish her book so I have converted it to a PDF file using OpenOffice and uploaded it for her author page. I have found after some searching that on can easily specify links to a particular page number in a PDF file although there seems to be no mechanism for linking to internal bookmarks. The link to a page, 13 in this case, takes the form of

<href= "document.pdf#page=13">description text</a>.

Most chapters of a book start on a new page so it works fine provided the pdf file is not updated and page numbers change.

18th August 2007

I have put in some major work on the web site, if you recall I laid out a list of changes I was considering which I repeat below with the ones I implemented highlighted

This involved changes to some 330 pages and was mostly done by global edits with a little hand crafting to pick up changes which were missed.

I found in checking that my <script> statements ought to have at least a type="text/javascript" attribute to satisfy the latest standards and I also implemented that as a global edit but inspecting and agreeing each change in case some were unexpected.

24th - 25th August 2007

BT Total Broadband and Authenticated Email

This is a good example of why I maintain my Diary series of pages, they pre-date the great rush to provide Blogs and give me somewhere to keep a record of what I have done whilst I am deciding whether a new web page is required or whether the information should find a home in existing page(s). In this case it all started with a request to help set up BT Total Broadband for one of my customers who needed to do a lot of work from home but ended up with a lot of information about authenticated email and how to get round the restrictions of many ISPs when on the move.

BT Total Broadband is a good, if somewhat expensive solution for the home, and is potentially much easier to set up for a series of machines than a classic ADSL, firewall, router and Wifi point. It has many additional 'features' such as a built in Internet Phone offering an extra number. It is however very much a home system and security is compromised severely for ease of setup on the Wifi side. It is based very much round the assumption that you will give up use of existing mail systems and use the BT Yahoo email provided. It assumes you will configure your machines by loading their CD which, from previous experience with BT Internet risks turning your machine from a professional interface to one much more suited to the inexperienced home user with a family. What I also rapidly discovered reading about the system and setting it up is that it makes it very difficult for someone who is bringing work home to be able to send any email through the system.

Mobile Email: This brings up the subject of sending email whilst on the move as using BT Total Broadband is just one example of how restrictions on email sending can be applied. Firstly lets consider how is email sent and then why do ISPs want to restrict the use of the Simple Mail Transport Protocol (SMTP) which is the universal way email is sent around the world. Every standard email program takes the message you want to send and transfers it to a SMTP server which then sends it onto the Internet where it eventually ends up in a mailbox. The mail server is usually provided by the ISP and has a 'normal' internet address and mail which is to be sent out is sent by convention to port 25. For those who do not have any idea what a port is then an analogy is that an internet address say mail.fasthosts.com is like a telephone exchange and the port is like an extension. The use of many ports is historically well defined, web servers are on port 80 and SMTP servers on 25 whilst ports around 139 are to do with file transfers.

Spam: The restrictions introduced by ISPs mainly result from coping with spam which is now one of the major problems of the internet. One of the responses to the increase in spam email is that ISPs try to ensure that all email sent from their domain can at least be traced back to the sender. This way if a user of an ISP starts spamming, the ISP can trace the sender and remove their account. If this is not done and an ISP gets known as a source of spam he will be blacklisted and many ISPs will block all email from them which is a disaster. New laws also mean that ISPs also have to keep a record of emails which are sent. So how do ISPs trace email that originates from their domain? In the days of dial up the ISPs used to check the caller ID of the phone and only allow access to their SMTP server for their customers calling in on one of their access numbers. The technique is similar for broadband which can be traced to the particular telephone and account.

This stops spam but only allows for users who accesses their ISP from a single place via broadband or via a single dial up telephone connection . It however prevents any sending of email when using a GSM data connection or broadband connection from any other site using your own ISPs SMTP server. So what do you do? In many cases you can use the SMTP server of the ISP who is providing the connection – at home by broadband connection is provided by Tiscali but my web sites and email by Freezone and Fasthosts. I send an email with an address of say peter@myfasthostsdomain.co.uk via smtp.tiscali.co.uk and it works fine for my desktops at home as a solution. However when I use a laptop via the mobile phone and a Vodafone GSM data connection I have to swap to the vodafone SMTP server smtp.vodafone.net and so on. Worse still the average WiFi hotspot does not provide a SMTP server at all as it would be a magnet for spammers. Even use of the SMTP server of the Service you are using does not always work as some ISPs limit use of their SMTP servers to messages from their own domain by checking the 'From:' address field and will not automatically relay messages that don't come from their domain. If you get an error message saying "relaying denied" or "relaying not allowed", then your ISP has put these measures in place.

Authenticated SMTP servers: Authenticated SMTP servers are one answer to access from many places. Many ISPs SMTP servers for their clients to access via other other networks where you are Authenticated by a username and password rather than your access route. This allows you to send emails from home, WiFi access points, mobile connections and other broadband connections without having to change the SMTP server address in your email package. These are often not well known or publicised but they do exist for the services that interest me here namely; Freezone, Btconnect (the BT business service) and Fasthosts (but only for their Advanced mailboxes where it is called roaming). They may exist for Btinternet, the home service as well but I have looked for or found it yet. The following give links from which the information can be gleaned.

Limited SMTP access problems: This is however not the end of the story as many ISPs and network providers do not like you to use their networks to send email which could be spam even if it is not through their SMTP servers and try to prevent it. I have started to come across this problem whilst traveling and examples are Xtra in New Zealand and many of the mobile operators including parts of the Vodafone worldwide system. So how do they control email whilst you are using their network? The easiest (and only really practical) method is to just force all emails to go via their SMTP server or as it is perhaps more correctly known MTA (Mail Transport Agents, the underlying programs that transfer emails round the world). Now as discussed above MTAs have traditionally listened on port 25, and that is the industry standard. So what most ISPs do is prevent connections to port 25 to propagating freely on the internet and either block it, divert any traffic to port 25 to their own SMTP server or in some cases check the email addresses used to verify it against a list of those which have been authorised in advance. An example of an ISP that allows pre-authorisation on their SMTP servers is BTconnect, the commercial side of BT.

Firms providing SMTP services and how to get round Port 25 blocking: A number of firms have grown up which provide an authorised SMTP service such as AuthSMTP and SMTP.com for those whose ISPs do not offer a service and they have worked out a solution to the problem - they set up their servers to also allow access on other port numbers. The common ports which are used are 225 and 2525 because they are easy to remember and they also use ports such as 23 and 26 which are normally reserved for other services such as Telnet which can not be globally blocked. They both offer diagnostic utilities which will check which ports are blocked and open or suggest how to check yourself so you can chose a suitable port. for example telnet smtp.com 2525 will check if port 2525 is open and gets through your firewall. It looks as if AuthSMTP offers more ports and is marginally cheaper and offers a fuller service but I have not tried either. Fasthosts Roaming (authenticated) SMTP also offers the ability to select port 225 which is less common and less likely to be blocked. I contacted Freezone and they suggested using 587 which is an AOL email port which their SMTP server will accept.

What if I have a PDA with a simple email set up which does not support changing the port? If you do not have a field in which to enter the Port number, such as on many PDA email applications, try entering something like "smtp.mailserver.com:2525" (without the quotation marks), into the "Outgoing Mail" field. This way, the colon specifies that 2525 is the port you wish to use without the explicit use of a field in many applications.

The final BT Total Broadband setup. It is time to get back to the original problem. The BT Total Broadband setup I was looking to achieve involves use by one person of a WiFi laptop with the 'master copy' of their personal email on BTconnect (actually a business system) and duplicate access to the main business email address which was with Fasthosts. In addition another member of the household would be operating a desktop connected by WiFi with an account with Freezone. In addition a number of other people would want to connect occasionally via Wifi. Despite my addition worries, not helped by BT themselves saying use of the BTconnect account via the BT Total Broadband would be impossible it all worked out. I found, as I expected, that one could not use their SMPT servers for 'relaying' ie sending email from addresses other than the btinternet.com domain.

What was unexpected in view of what had been said and written in some web postings there seems to be no problem with use of an authenticated SMPT server and there was no port blocking. I quickly had the system working using a Freezone SMTP server, which I was familiar with, whilst I did the searching around for the information for Fasthosts and BTconnect and set them up as in the table below. The BTconnect account is the master and emails are not left on the server whilst the business account with Fasthosts is set to leave the original on the machine to be stored at the office – unfortunately Outlook 2003 does not have a facility to automatically bcc (blind copy) any outgoing mail to another account which is what I do with Thunderbird email myself whilst using the laptop away from base.

Some Authenticated SMTP Server Information
  Fasthosts Freezone BTconnect (Business) BTinternet (home)
SMTP Server smtp.fasthosts.co.uk [or mail.fasthosts.co.uk] auth-smtp.freezone.co.uk smtp.btconnect.com [or mail.btconnect.com ?]

mail.btinternet.com [or smtp.btinternet.com]

Username Same as pop account fzxxxxxxx if you have an old account or auth-smtp@yourdomain.co.uk Same as pop account BT Yahoo Username
Password Same as pop account Your fzxxxx password if you have an old account or your admin account password if you have a domain Same as pop account BT Yahoo Password
Ports available 25 and 255 25 and 587 25 25
Use for other email accounts (Email relay) No - only the email address of the Advanced Mailbox can be used on mail sent No limitations at present With telephone registration of each address on other domains With Internet registration of each address on other domains

What can I do if I do not have any access to an Authenticated SMTP server from my ISP and have BT Total Broadband? If you've got a non-BT e-mail address which you want to keep using from an ISP which does not have an Authenticated SMTP server then you your choices are:

The first two have the advantage that they will allow you to and send email almost anywhere if you use a laptop whilst the third only solves the problem for a desktop on BT Total Broadband.

Registering other email address you wish to use with BT Total Broadband: The need for pre-registration with BT Broadband's outgoing mail servers comes about because, even when authenticated themselves, they will not normally allow you to send e-mail which looks as if it comes from a non-BT e-mail address as they check your FROM: headers. They do however allow you to specify in advance what non-BT addresses you want BT to allow. This is poorly documented and the way forwards I offer originates from information I located at http://www.pdoc.co.uk/btbroadband.shtml .

First specify the emails addresses the want the BT Broadband SMTP server to allow:

  1. Log in to BT Yahoo! and go to the Mail section, choose Options then Mail Settings then Add.
  2. In the Account Name box type a name of your choice . Click Continue.
  3. In Name type the name you use when sending mail from your existing account.
  4. In Email type your existing account e-mail address. Click Continue.
  5. On the next screen ("Setup Mail Server") click Skip this Step.
  6. On the next screen, click Send verification.
  7. Type your BT Yahoo! password if requested.
  8. Stay logged in to BT Yahoo! and check for an email in your old account using Outlook or whatever. You should receive an email from "Yahoo!Member Services" - open it and either click on the big link provided or note the verification code provided.
  9. Back in BT Yahoo!, check the verification code has been filled in, or fill it in yourself. Click on Verify Account.
  10. If asked for your BT Yahoo! password, fill it in and click Sign In. You've now finished with the BT Yahoo! web pages.
Secondly set up the BT Broadband SMTP settings in your mail program.
  1. SMTP server mail.btinternet.com or smtp.btinternet.com
  2. Specify that the SMTP server requires authentication
  3. use the BT Yahoo! e-mail address and password as the username and password for authentication.
  4. The SMTP server can use either port 25 or port 587.

Now check it all works! Microsoft Outlook has a useful facility to check the email settings and provide feedback whilst you are still in the account setup or editing.

A valid alternative - get an email [only] account with a provider offering Authenticated SMTP such as Fasthosts. This is likely to cost no more than using a dedicated service such as AuthSMTP and has many advantages including unlimited numbers of emails. Fasthosts email plus package is £14.99 a year (plus VAT) and offers two Advanced mailboxes with separate Authorised (Roaming) SMTP access plus many other features namely

You do not have to use the new addresses immediately because you can set the reply to address of your old email in your outgoing mail settings and set up a forwarder on the mailbox to divert any email to that account to your old one.

Freezone Internet where this site (pcurtis.com) is hosted offer similar facilities for £0.99 a month (plus VAT?) with a FREE domain name but no alternative SMTP port other than 587 .

Both need a Domain name to be registered but that gives a fixed address for one whatever provider you use in the future. Domain registration is included with Freezone and works out at £5.95 every two years (plus VAT?) for Fasthosts through their subsidiaryUKReg . Setting up an email only account is my longer term recommendation to my client.

A BT Yahoo Mail Anomaly: All the communications for the BT Broadband is routed through the address they set up for you on BT Yahoo! I accessed that initially by the web mail interface as will many home users then set up an account in Outlook along with the other two accounts. Interestingly I found that once I had opened it and read it using webmail it would not download – this might be because I had specified leaving a copy on the server but I have not had this with any other webmail system.

What do I think of BT Total Broadband overall? A number of nice touches like the Internet Phone built in and very easy to set up if you are an unsophisticated used only interested in home use for the family. I like the fact that it can automatically update in the box without the machine even being on. It is almost trivial to let visitors have Wifi access as they just have to enter a short WEP key. If you want to have a network between machines then with file and print sharing then security becomes a concern as you are limited to WEP with a fixed 40 bit key which can be cracked by anyone who spends a few hours on the internet and has 5 or ten minutes to spare outside your house or reads it off the back of the unit. I like to set up a second level of authentication so only machines with a MAC address I have authorised - a MAC address is unique to each network interface but it is difficult to access the internal firewall and router settings and MAC address filtering does not feature in the options one can set up as far as I could tell.

What do I intend to do with this Diary Entry? I think the best way forwards is to turn it round and create a new page on Sending Email on the move with BT Total Broadband a special case/example of the many ways round the problems with entry points in Global Communications and Computing and perhaps Ubuntu Linux on the Move

24th - 25th August 2007

Network Install of Ubuntu Dapper Drake on a Toshiba Portege 3440

The Toshiba Portege is marginal in specification for installing Ubuntu Dapper Drake 6.06 in several ways, it has a maximum of 192 Mbytes of RAM and has no built in CD. I did the initial installs using the special CD/port replicator however I discovered that all the partitions move when you un-dock and Grub has to be modified. You can not do an install from the DirectCD in any case due to lack of memory, only from the Install CD and preferably after a swap partition has been created.

This all led me to look at the possibilities of a network install as the Portege 3440 is new enough to have a Boot From Network option in the Bios (accessed by F2 during boot-up). I chose to boot from a Windows machine using Tftpd32.exe (a freeware Windows TFTP/DHCP server program with PXE boot support built in). The standard method for an Ubuntu Network Boot involves loading a certain amount of software (a few Mbytes) from the machine acting as the server and then the rest is loaded from the Internet. I did a bit of Googling and found nothing specifying what one did about software firewalls and the DHCP/Firewalls server in my ADSL Router Firewall box, which was clearly still needed to allow the Internet access. I did a number of tests turning various areas on and off and found the the DCHP server in the 'box' is still needed and the software and router firewalls had no impact so can be left enabled.

The remainder of this is based closely on the Ubuntu Community Help page at http://help.ubuntu.com/community/Installation/WindowsServerNetboot . The proceedure is:

 

  1. Download the zip archive version of Tftp32.exe from the author Philip Jounin's web site at http://tftpd32.jounin.net/
  2. On the Windows machine, make a new folder which I called "cb" – I made it at top level on C: but that is probably not necessary.
  3. Extract Tftpd32.exe and its two support files into "cb" (it does not need to be installed as long as the two support files are in the same directory)
  4. Download the Ubuntu Network Boot files from http://archive.ubuntu.com/ubuntu/dists/dapper/main/installer-i386/current/images/netboot/netboot.tar.gz – you can change dapper to feisty etc
  5. Create a folder in “cb” called “netboot”
  6. Extract the contents of netboot.tar.gz into your new folder "netboot" – I got errors saying the end of the file was garbage using Winzip. You should have several files and folders from the extraction.
  7. Open the folder ../cb/netboot/ubuntu-installer/i386/ and copy "pxelinux.cfg", "linux", and "pxelinux.0" into ../cb/netboot/
  8. Run Tftpd32.exe and click on the DHCP tab
  9. In the default router box, put the address of your internet gateway (usually 192.168.0.1 or 192.168.1.1 for standard ADSL routers, mine is 192.168.1.1)
  10. In the ip pool starting address box put your gateway's address + 1 to the last number (ie: 192.168.1.2)
  11. Size of pool = 20 (for example)
  12. Network mask should be 255.255.255.0
  13. Set the Bootfile field to the location of the pxelinux.0 file ( \netboot\pxelinux.0 )
  14. Set the WINS/DNS server address to the address of your ADSL router (192.168.1.1)
  15. Leave the domain name blank.
  16. Hit the "Save" button and then click "Settings"
  17. On the Settings menu tick the boxes for the following and leave the others how they are.
    • PXE Compatibility
    • translate unix file names
    • allow / as virtual root
  18. click "ok",
  19. Boot up the Portege 3440 whilst tapping the F2 key.
  20. Select N for network boot and you should see it looking for the DCHP source then the installer should start and present the usual menus.
  21. Close Tftp32 on the server machine.

 

 

Valid HTML 4.01 Copyright © Peter & Pauline Curtis
Content revised: 2nd August, 2020