Home Uniquely NZ Travel Howto Pauline Small Firms Search
Diary of System and Website Development
Part 25 (January 2014 -> December 2014)

January - May 2014

Most of the activities have been concerned with applet development and debuging and have been recorded directly in Cinnamon Spices

8th May 2014

Use of GIT

In order to make this self contained I am starting with a duplicate of what I wrote earlier although the emphasis will rapidly change to my use of GIT in applet development.

GIT is the distributed version control sysytem that Cinnamon uses which was written by Linus Torvalds for the Linux Kernel.

The links and documents I have found useful are:

Wikipedia GIT Software - a good introduction to GIT

The Pro GIT book, written by Scott Chacon in PDF form

GIT Magic - Amore informal but usefull approach - you may need to google for alternative downloads.

https://help.github.com/articles/set-up-gi - Github guide. Github.com is the server used for all the cinnamon system software which is free for open source software.

http://www.kernel.org/pub/software/scm/git/docs/user-manual.html - from the Linux Kernel Team

Use gitk to understand GIT - useful tutorial for an essential tools part 1

Use gitk to understand GIT merge and rebase useful tutorial for an essential tools part 2

 

Building Cinnamon, Muffin etc from GIT on github

My first use of GIT was to be able to build Cinnamon on my own machine and test out various development patches.

And I spent quite a lot of time reading the PDF version of the Pro Git book, written by Scott Chacon - the link takes you to an online version and there is a download link for the PDF at the left. I put a copy on the phone so I could dip into it and after a number of passes through the early part the more specific instructions started to make sense - I do not like doing things which I do not understand at all. One should also look at http://www.kernel.org/pub/software/scm/git/docs/user-manual.html for more information.

Once I had read The Pro Git book, written by Scott Chacon in PDF form I switched to the Github help files at https://help.github.com/articles/set-up-git to set up GIT on the server. I also set up new email account especially for use with github and changed to it by following instructions at https://help.github.com/articles/how-do-i-change-my-primary-email-address

Install GIT

Use Synaptic or in a terminal:

sudo apt-get install git

Carry out configuration of the GIT you have installed

Using information at https://help.github.com/articles/set-up-git I found I should

git config --global user.name "pdcurtis"
git config --global user.email "gitmail_at_pcurtis.com"
git config --global credential.helper cache
git config --global credential.helper 'cache --timeout=3600'

The first two will need to be done on every machine. I have not yet done the last two as I already seem to be set up in a way that kept me logged in permanently.

Fork

Using information at https://help.github.com/articles/fork-a-repo

Click fork when logged in and viewing Cinnamon Repository then

git clone https://github.com/pdcurtis/Cinnamon.git
# Clones your fork of the repo into the current directory in terminal

then move to Muffin Repository and repeat while you are online if you intend to compile muffin.

Needed once only however many machines etc you use

Configure remotes

When a repo is cloned, it has a default remote called origin that points to your fork on GitHub, not the original repo it was forked from. To keep track of the original repo, you need to add another remote named upstream:

cd ~/Cinnamon
# Changes the active directory in the prompt to the newly cloned "Cinnamon" directory

git remote add upstream https://github.com/linuxmint/Cinnamon.git
# Assigns the original repo to a remote called "upstream"

Needed once only however many machines etc you use

Repeat for muffin if you are going to build it as well.

Keep your local copy up to date

cd ~/Cinnamon
git fetch upstream
# Fetches any new changes from the original repository, without modifying your files [you are working on ???]

git merge upstream/master
# Merges any changes fetched into your working files

I was interested to find the size of the Cinnamon directory

du -ch | grep total
Cinnamon was much smaller than I expected at about 28 Mbytes whilst muffin was 50 Mbytes. Interestingly du gave a much higher figure than the Properties of the folder and these figures rise after compiling to 50 and 70 Mbytes. I suggest allowing 500 Mbytes total as they will grow with the GIT repositories.

[ Click here to expand to show what I had in the terminal when I did all the steps above on the first machine for Cinnamon.  ]

Building your version of Cinnamon

This is now getting to the interesting part where we can do a test to see if we can compile from our own local copy, we can then start to make our own changes or pull others test changes before they are accepted into the Master branch and see the effects. The instructions can be found at https://github.com/linuxmint/Cinnamon/wiki/Building and http://community.linuxmint.com/tutorial/view/955 which has some additional comments. I started by only building Cinnamon as I am using the nightly updates so I was unlikely to get seriously out of step. I have built muffin since to check all the proceedures work.

One-off set up of repositories and dependencies before building your version of Cinnamon

The next stage is to make sure all the repositories required to compile cinnamon (and muffin) are available. They are usually present in Ubuntu but you can check by looking at the following file /etc/apt/sources.list by

sudo gedit /etc/apt/sources.list

and making sure that the deb-source files are present as well as the others. If you are not using Mint you should ignore the first lines. This is for Ubuntu 12.04 precise and Mint Maya maya and precise will change with to nadia and quantal in 12.10 and so on.

deb http://packages.linuxmint.com maya main upstream import
deb-src http://packages.linuxmint.com maya main upstream import

deb http://archive.ubuntu.com/ubuntu/ precise main restricted universe multiverse
deb-src http://archive.ubuntu.com/ubuntu/ precise main restricted universe multiverse

deb http://archive.ubuntu.com/ubuntu/ precise-updates main restricted universe multiverse
deb-src http://archive.ubuntu.com/ubuntu/ precise-updates main restricted universe multiverse

deb http://extras.ubuntu.com/ubuntu precise main
deb-src http://extras.ubuntu.com/ubuntu precise main

You now need to find out all the required dependencies for cinnamon and muffin and install them by:

sudo apt-get install dpkg-dev
sudo apt-get build-dep cinnamon
sudo apt-get build-dep muffin

Note: dpkg may be installed already. The dependencies needs a 30 Mbyte download and occupies an additional 125 Mbytes for Cinnamon and Muffin together.

Compile and install Muffin

I have only once carried out these stages as I am using the muffin nightly but if you are working from scratch you will need to check the instructions in one of the links above noting you need to use sudo with dpkg -i for all the packages created. It is possible the order matters if you do not have a version of muffin installed already. The version will change but the following should work if you change the version numbers to match.

cd ~/muffin
dpkg-buildpackage

cd~/
sudo dpkg -i muffin-common_1.1.2_all.deb libmuffin0_1.1.2_i386.deb muffin_1.1.2_i386.deb gir1.2-muffin-3.0_1.1.2_i386.deb libmuffin-dev_1.1.2_i386.deb

Compile and Install the new Cinnamon

NOTE: You will need a version of muffin before compiling Cinnamon. Installing the nightly from the PPA at the very start seems the most sensible way to make sure you have no unresolved dependencies from doing things in the wrong order even if you then overwrite it by installing custom versions of muffin compiled as above.

We are now ready to Compile the new Cinnamon by:

cd ~/Cinnamon
./autogen.sh
dpkg-buildpackage

This is a slow activity but it eventually produces a cinnamon deb file in the parent directory, which you can install with gdebi or dpkg -i.

I had to use dpkg -i as I had the Nightly installed which showed up as a more recent version and gdebi did not allow be to overwrite a newer version so

cd ~/
sudo dpkg -i ~/cinnamon_1.6.7_i386.deb

Note: Version number may need to be updated - check in home folder.

Make sure all your programs are closed as you will need to immediately logout and back in again with the new version - use Alt Ctrl Backspace to force a logout if required

I then found I had the latest version of Cinnamon - without any changes by me of course. This was real progress.

Merging other peoples Pull requests.

One of the main reasons for installing GIT and Cinnamon/muffin on my machine is so I can try out the various solutions which are being put forwards for testing. Most use the pull request mechanism even when they are still work in progress so they never get 'pulled' into Upstream for me to access. I tryed this out on a suitable Pull request by running the code provided - in this case:

git pull https://github.com/autarkper/Cinnamon alttab-multi-ws

In this case it turned out to contain 4 commits and had some problems so I wanted to get rid of it. At this point I realised that I should have created a branch for testing it first and it was now in my master branch and very difficult to get shot of. This is because the writers of GIT are almost paranoid about protecting one from ever losing any information which is what I wanted to do. In the end I found http://nakkaya.com/2009/09/24/git-delete-last-commit/ and then http://git-scm.com/2011/07/11/reset.html which is by the writer of GIT Pro which is my current reference book (PDF) and used.

git reset --hard HEAD~4

HEAD~nn is a shorthand for commit nn before head. Alternatively you can refer to the SHA-1 of the hash you want to reset to. --soft option will delete the commit but it will leave all your changed files "Changes to be committed", as git status would put it. I wanted to get rid of any changes to tracked files in the working tree since before the commits so use --hard instead.

gitk - a graphic viewer for GIT

This program helps enourmously in helping one see what one is doing.

sudo apt-get install gitk

Run it in a terminal from within the Cinnamon or muffin folder. There are a nuber of command line options I have not yet explored

The following is a typical output after my second attempt at merging in a pull request having created a new branch for my testing and a extra branch for a particular test. At this time there have been no additional changes in 'upstream' to fetch and merge.


Switching between branches (git checkout )

When you change to a different Branch (or location on a branch) the current contents of the working folder (~/Cinnamon or ~/muffin in this case) reflects the changes which have been made and committed at that point (plus changes made but not committed??) . The branch you have changed to “the current branch” is “the branch that will advance when one performs a commit”. The location of the snapshot is refered to as the HEAD??

You change branch by

git checkout existingbranchname

The simplest way to creat a new branch for example when you need to do some testing is by

git checkout -b newtestbranch

This creates a new branch from where the HEAD pointed and switches to it.

If this is all becoming a bit of a mystery then have a look at http://lostechies.com/joshuaflanagan/2010/09/03/use-gitk-to-understand-git/ which goes through all the steps on how to create a branch to do some work, after it has been test merge it into the branch you are using for long term development (development) , delete the testing branch and how to push the extra work (commits) back to your branch on the remote server (origin) and takes one through what gitk shows after every step in a simple example.

Using GIT for applet development.

This needed me to create a new repository on the github server called cinnamon-applets and then clone it onto my machine and add my applets into it. I started to do that but got badly out of step when I was fixing the segfault problems so I had to do a retrospective addition of the various stages and commit them in turn. This did not always go well and I needed to do a couple of 'naughty' corrections and I just hope nobody pulled the repositories whilst this was going on!

Typical

pcurtis@defiant:~/cinnamon-applets$ git branch
master
* pad-test 
pcurtis@defiant:~$ cd cinnamon-applets
pcurtis@defiant:~/cinnamon-applets$ git status
On branch pad-test
Changes not staged for commit:
(use "git add <file>..." to update what will be committed)
(use "git checkout -- <file>..." to discard changes in working directory)

    modified: netusagemonitor@pdcurtis/applet.js
    modified: netusagemonitor@pdcurtis/settings-schema.json

no changes added to commit (use "git add" and/or "git commit -a")
pcurtis@defiant:~/cinnamon-applets$ git commit -a -m "Changes to arrangements for padding to get round removal of leading spaces, reduce jitter and shrink width"
[pad-test f9cb13f] Changes to arrangements for padding to get round removal of leading spaces, reduce jitter and shrink width
2 files changed, 10 insertions(+), 10 deletions(-)
pcurtis@defiant:~/cinnamon-applets$ git checkout master
Switched to branch 'master'
Your branch is up-to-date with 'origin/master'.
pcurtis@defiant:~/cinnamon-applets$ git branch
* master
pad-test
pcurtis@defiant:~/cinnamon-applets$ git checkout pad-test
Switched to branch 'pad-test'
pcurtis@defiant:~/cinnamon-applets$ git branch
master
* pad-test
pcurtis@defiant:~/cinnamon-applets$ git status
On branch pad-test
nothing to commit, working directory clean
pcurtis@defiant:~/cinnamon-applets$

Linking example

pcurtis@defiant:~/.local/share/cinnamon/applets/stopwatch@pdcurtis$ ln -s /home/pcurtis/cinnamon-applets/stopwatch@pdcurtis /home/pcurtis/.local/share/cinnamon/applets/stopwatch@pdcurtis
pcurtis@defiant:~/.local/share/cinnamon/applets/stopwatch@pdcurtis$

ANDROID

Generic Android 4.2.2 Network Media Player (copy from earlier diary)

My first Android device was a Generic Android 4.2.2 Network Media Player with Dual Core Processor, HDMI output, 4 USB sockets and a simple remote from 7Dayshop.com . This allows me to plug in a 640 Gbyte USB 3.5 inch drive which has all my music, all my pictures including 'print files' from the last 10 years and most of my old videos transferred from VHS tapes and some 200 Video8 tapes from my camcorders up to 2000 when I went digital. For under £45 this is an exception bit of kit, especially when used with an external mouse and preferably keyboard and plugs straight into a HDMI input on a monitor or TV. It has its own sound output but my HDMI monitor has a headphone output which seems to give less interference and better sound.

This was my first experience with Android and it was like going back to the dark ages prior to Windows 95 after using Linux for many years but it does allow me to do most of what I wanted. The simplicity of the Apps available was a real surprise when Android is used extensively on Tablets and even high level phones have far more computing power than my netbook. The biggest problem in this application is however that Android severely restricts the use of external storage by Applications and it is only available write only to applications so, for example, one sets up an application which is capable of lossless rotating of old images which are on their sides but then one finds it can only work on the very limited internal memory so one is faced with using the internal file manager to copy ones folder of pictures to the internal storage (4 Gbytes but only a part is available) viewing and rotating then copying back onto the hard drive. The App I have been using sorts the pictures on file creation time rather than the Exif data and suposedly has the ability to change the accessed dates to match the Exif data (camera shooting time) but can not write them to the hard drive unless one copies them to and from the local memory in the file manager - duh. I found a simple command line program which does it on my Linux machines in a couple of minutes so it is quicker to move the hard drive and do the whole 32 Gbytes than mess around for even a couple of folders. You can run several programs at a time so I can set some music playing then start the Pictures displaying but changing the volume even with the remote stops the slide show.

Android obviously handles email in phones and tablets but again it is a trip down memory lane. Few email apps in 2012 would even display html mail and it seems only two (which are not free) can write html mail. I suspect most people are driven to web mail. On the good side contacts do get automatically synchronised if you have gmail account which is almost manditory to even download your apps from 'Google Play', the source of 'approved' apps, even the name is a give away to the level of sophistication Google offers in Android. I have set up Inomail on the box which is not bad and handles html mail (even writes it if you pay £3.00) and the integrates and synchronises gmail contacts. The system does not properly synchronise calendars (google not box problem I believe so it may get fixed) so I have to use the web version of google contacts at present on the box but it is looking more like a simplistic but workable set up.

I have not got network file sharing working so far. I have installed Samba Filesharing but there there seems to be a fundamental problem as Android seems to block ports below 1000 so SSH and other file sharing requires non standard ports on unrooted machines.

Apps in use on the Network Media Player (NMP) box:

Apps tried which are likely to be uninstalled

To be investigated:

Samsung Galaxy Tab 3 7 inch (8 Gbyte)

This proved to be a very different story to the NMP box and almost everything we needed worked out of the box. Partly the basic built- in Android programs have improved in the last year but also a large number have been enhanced by Samsung for their phones and tablets. For example the built in Email, Calendar and Contacts apps are perfectly adequate and work out of the box for POP mail, Gmail, Google Contacts and Google Calendars which all syncronise with Thunderbird and my Blackberry. The Samsung Tab 3 is light (300 gms) and has exceptional battery life, it can be running in the background for days and has a good ten hours of use. There is a built in GPS which is not very sensitive and is slow to obtain lock but even so proved very useful during our time in Holland.

The Samsung Tab 3 however does have severe shortfalls when it comes to interfaces.

The major one is that it only has a single microUSB connection, primarily used for charging, which does not implement OTG.

USB On-The-Go, often abbreviated to just OTG, is a specification first used in late 2001, that allows USB devices such as digital audio players, mobile phones and tablets to act as a host, allowing other USB devices like a USB flash drive, digital camera, mouse, or keyboard to be attached to them. Use of USB OTG allows these devices to switch back and forth between the roles of host and client devices. For instance, a tablet may utilise an external USB keyboard or read from removable media when acting as the host device, yet present itself as a USB Mass Storage Device when connected to a computer. In other words, OTG introduces the concept that devices can perform both master and slave roles – whenever two USB devices are connected and one of them is an OTG device, they establish a communications link. Whichever device controls that link is called the master or host, while the other is called the slave or peripheral.

The microUSB connector on the Samsung has another problem in that it does not implement the USB Mass Storage file transfer system which is used by most devices such as cameras, media players and earlier Android devices. This is not unique to Samsung - since Honeycomb, Android devices have by default, starting using MTP (Media Transfer Protocol) instead of USB Mass Storage file transfer. Arguably there were good reasons to swap but the support in hosts has been so patchy that it is still a major problem.

The problem with a USB mass-storage device is that it operates at the granularity of a mass storage device block, rather than at the logical file level. In other words, the USB mass storage class gives a host computer undifferentiated access to bulk mass storage, such as compact flash, rather than to a file system, which might be safely shared with the target device. In practice, therefore, when a USB host computer has mounted an MSC partition, it assumes absolute control of the storage, which then may not be safely modified by the device without risk of data corruption until the host computer has disconnected.

In contrast USB Mass Storage uses a block-level protocol which means is that a partition can not be mounted on two different systems at the same time. Each partition that needs to be mounted on a host computer must first be unmounted from the device. This is impossible on a device with only a single partition, since if the system partition was unmounted, the device would crash. To work around this issue, the first Android phones either had two partitions in the internal memory or had a microSD slot. Phones with microSD slots would just provide access to the microSD card partitions to the connected computer. When MTP is used this restriction is lifted with the associated advantages of a single partition allowing flexible distribution between application and other data.

However MTP access is much less flexible as the main purpose of this protocol was to only allow the transfer of media files and associated metadata to and from portable devices, one transfer function, in or out, at a time. Productive activities such as open, edit and modify require copying the whole file to the hosting system for these functions then copying back in their entirety to change the data.

We are primarily interested in access from Linux and versions of Ubuntu and Linux Mint have some built in MTP support - see webupd8.org for earlier versions of Ubuntu/Mint. On connecting to my machines running Mint 17 I get the occasional error message but generally it mounts. On the pad I get a little USB symbol in the top right corner of the notification area and if one swipes down there is a notification one can tap to chose the USB interface options. Chose Media Device (MTP) details and screenshots follow.

screenshot

wipe down to

screenshot

tap to get to

screenshot

and tap the select and a Nemo (file manager) window should open on the computer if it has not already showing an additional device 'Samsung Android' with folders 'Tablet' and 'Card' which correspond to Sdcard0 and extSdCard.

Screenshots

The screenshots taken above use the built in Android function activated by the combination of Home and Power keys until thee screen shakes. On other machines the Volume Down and Power key combination is more common.

Bluetooth on the Samsung

Bluetooth seems to work very easily on the Samsung. I have only tested with the BTR006 audio receiver which paired easily and the music output switches automatically when Bluetooth is on.

Sumvision Cyclone Voyager II Quad Core processor (16 Gbytes).

The Sumvision Cyclone Voyager II Quad Core bought from 7dayshop is a much less refined machine than the Samsung Galaxy Tab 3 7" in many ways but is also much cheaper especially from (£69 v £116), has more flexible interfaces, has a much more powerful processor, but all at the expense of battery life. It is also significantly heavier (355 v 300 gms) and has a smaller display area on the screen as it uses software 'buttons' at the bottom of the screen. The software buttons are however much more flexible and easier to use and as they re-orient with the device rotation and are reprogrammed with context.

Much of my time using the Sumvision has been taken in restoring the level of functionality and refinement to match the Samsung Galixy Tab 3 7 inch I bought for Pauline. This has involved installing several additional Apps, each of which needs configuration to get the best out of them. The important ones are;

USB Keyboard

I bought a Sumvision keyboard and case at the same time as the Cyclone Voyager II which works well with it, however I also like some of the features of the inbuilt keyboard and to be able to rotate to a portrait view so it is often unplugged. The keyboard, as expected does not work with the Samsung because of the lack of OTG covered above.

Bluetooth on the Sumvision Cyclone

Despite the Sumvision web site stating Bluetooth is only available on Cyclone Voyage II 8" and larger screen pads it is present and works well with the BTR006 audio receiver which paired easily and the music output switches automatically when Bluetooth is on. I have found that Music sometimes stops a few minutes after the screen timeout - changing settings in Battery Doctor seems to solve it although I advise not using the automatic functions to close down background programs can again cuse Bluetooth problems.

GPS on the Sumvision Cyclone

There is an GPS receiver (undocumented in the sales bumf) - the sensitivity does not seem high but neither is the one in the Samsung Galaxy Tab 3. I installed:

Connection of Cyclone to Computer (USB Mass storage Mode)

Connecting to my Linux Mint computer is very easy. When the tablet is pluged in the tablet shows a screen asking if you want to connect and warning that programs using the plugged in microSD and the internal sdCard may have problems (because they will not be accessible whilst the connection is active). Two drives showed up called CYCLONE and Kingston ( tmy 32 Gbyte microSD is made by Kingstone)

I checked both the Samsung and the Sumvision had good connections by transfering most of my music collection (10 Gbytes) directly from one to the other. The average speed showed as 3.8 Mbytes/sec and the transfer took ~40 minutes. The battery did not seem to go down much (max 3%) on the Cyclone so it is possible it keeps topped up although it did not show as charging - otherwise it must use very little with the screen off. The Samsung gained 5% charge whilst on the cable as one would expect a slow rate. .

Power Pack and other extras for Sumvision Cyclone

I have purchased a 7dayshop RPB52 Universal Portable Backup Power Pack 5000mah and a 7dayshop DC 2.5mm to USB 2.0 High Speed Charging Cable for Bluetooth Receiver / Transmitter & More so I can recharge the Sumvision when away from a mains supply. I also bought a LMS Data Gold Plated HDMI to Mini HDMI Digital / Audio Cable - 1.8m to test the HDMI out put which worked perfectly for Sound and Picture on my LG Monitor. I finally bought and fitted a 32 Gbyte Kingston microSD card.

Encryption - Truecrypt and Android implementations.

We use Truecrypt on all our computers which:

We use the Truecrypt facility to create a Virtual Disk with the contents encrypted into a single file or onto a disk partition or removable media such as a USB stick. The encryption is all on the fly so you have a file, you mount it as a disk and from then on it is used just like a real disk and everything is decrypted and re-encrypted invisibly in real time. The virtual Drive is unmounted automatically at close down and one should have closed all the open documents using the Virtual Drive by that point just like when you shut down normally. The advantage is that you never have the files copied onto a real disk so there are no shadows or temporary files left behind and one does not have to do a secure delete. I have had it loaded it onto all my Windows and Linux systems. Truecrypt obviously installs deep into the operating system in order to encrypt decrypt invisibly on the fly. Making new volumes (encrypted containers) is now trivial – just use the wizard. One has to have administrative privileges to mount ones volumes and under Linux this means that one is asked for ones administrative password on occasions as well as the volume password when the Truecrypt volume is mounted. Truecrypt has now been withdrawn under rather odd circumstances by its original writers and its future is ill determined allthough the formats will no doubt continue to be supported and developed. At present the work and downloads of existing versions is concentrated at truecrypt.ch.

Opening Truecrypt Containers (Vaults or Files) in Android using EDS Lite

There are a couple of options to open TrueCrypt containers on the go using an Android device. Cryptonite is in some ways the more elegant solution but unfortunately it requires the device to be rooted in order to handle TrueCrypt files. For those who don’t want root their devices, there is EDS (Encrypted Data Store).

EDS Lite is a free and open source app that works with containers (vaults) that are stored locally. These can be transferred using USB, or downloaded from a cloud service to an Android device. Of course, once modified locally they can be uploaded to the cloud again for cross device and platform syncing. This is known as ‘normal’ mode of operation, and is the only mode supported by the free version of EDS. For TrueCrypt containers to be compatible with EDS, they must:

EDS can create Truecrypt vaults (containers), display the 'linked' containers, open them and display the contents in a File explorer which can also display normal files and launch applications.

This mode of operation is not as good as full 'on-the-fly' encryption and decryption as there will be times when the temporary file is availabe unencrypted and it also depends on how well it is overwritten, it should involve multiple overwrites with random data for complete security. The full EDS can do 'on-the-fly' encryption and decryption but only on 'rooted' machines.

In practice the process of viewing and editing encrypted files from a Truecrypt Vault is quite practical and is done within EDS Lite. The interface is however not completely intuitive or consistent with other apps, but few are!

Syncronising data between computers and mobile machines - Combining Dropbox for Android and Dropsync

Dropbox works somewhat differently on a mobile device to on a computer. On computers one has a folder (Called Dropbox in your home folder under Linux) which is syncronised to the Dropbox in the Cloud and then syncronised with similar folders on PCs running Linux, Windows or on Apple computers. As soon as a file has been changed it is uploaded and other machines have it available and are downloading almost immediately. This would provide an unacceptably large data flow for mobile devices so only the file names are available on mobile devices and they are downloaded for use on demand when an appropriate application is available. They can also be stored locally if they are marked as favourites for offline use. The only major exception is with ones pictures which can beuploaded automatically with an option to hold upload until a Wifi connection is available. This means that it is difficult to keep even small files continuously and automatically in syncronisation as is desirable for a Truecrypt Container.

This is where an app called Dropsync comes in. It uses the Dropbox api (application progamming interface) to syncronise a folder (or multiple folders in the paid version) between the Android device and your cloud based Dropbox in the same way that Dropbox does on real machines. Upward syncronisation is immediate and the download syncronisation frequency can be set up to reflect data restrictions as well as obviously being available on demand. It is very easy to set up the Folder correlation after which it is all seamless. The free version has restrictions on the number of folders and the size of each file which can be uploaded (8 mbytes). To summarise Dropsync provides:

Dropsync, once set up gives us a home for one or more Truecrypt type containers (Vaults) which can be used to keep encrypted information in sync between many computers and mobile devices. In the free version these are limited to 8 Mbytes each. There may be the occassional conflict if there are several users who change files frequently and do not close the Vaults when they are not in use but no more than with Dropbox. It is prudent to do a manual sync if you have chosen a long autosync interval or if the automatic upload conditions are not met (eg a low battery).

Summary of accessing and modifying a shared encrypted document.

This is what I have writen for Pauline on how to access and possibly edit an encrypted document on an Android pad.

That sounds long but it takes me under a minute in and to get to view a file with the main time being carefully entering a pass phrase on a small onscreen keypad.

Why have a pad?

We first looked at Pads when Pauline started complaining that her old MSI Wind U100 Netbook was getting very slow but could not see any way they could be a replacement - however we began to see other roles that they could play. This was followed by Pete having to help set up and explain the use of an iPad to John which again gave some more ideas. Finally it came down to a spur of the moment purchase in Guernsey, the Samsung Galaxy 3 7" was on a very good special offer and well under our duty free limit so it seemed a good way to really see if one could be useful. We had already laid out a list of things which would be needed to make a contribution to our existing collection of 'tools' for round the house and whilst travelling and the Galaxy seemed to make a good shot at many of them. The applications seemed to have come a long way since we got the android media box, certainly the preloaded and enhanced versions from Samsung and we could see a gap where it fitted between the excellent, always available email, contacts and diary through the Blackberry and the internet and office functions on the Netbook

What was needed was an enhancement of the functions of the Blackberry onto a screen capable of handling simple web browsing and basic office functions yet light to carry and cabable of lasting for long periods between charging. The netbooks/laptop also handles our music played through bluetooth, web authoring and as an image organizer and viewer and editor for digital photos from our cameras. A sensible target would be to extend the time away from a day with a phone to a long weekend (or even a week) with a pad with a saving of several kgs.

 

Frequently Used Apps


Basic System - Mostly Preloaded Apps

Downloaded Extras
BBC News
Fearsum UK TV
Evernote
Windup (Not needed on Samsung)
Docs to Go (or Preloaded Polaris Office on Samsung)
Evernote
920 Text
Maps
GPS Essentials

Downloaded Utilities
SwiftKey Keyboard (Not needed on Samsung)
Battery Doctor (Not needed on Samsung)
Dropbox
Dropsync
EDS Lite
Clipper
ES3 File Browser
Epson iPrint

Tethering via Bluetooth to an Android Phone

This proved to be very easy and provides access to the internet via an Android smart phone - in my case a Samsung S3 Mini (GT-I8200N) with Android 4.2.2 although this should work with most phones and equal or higher versions of android. The phone needs an active internet connection which can be a mobile phone connection with Data enabled or a Wifi connection. One might ask why bother in the case of a wifi connection, why not connect directly? It has obvious advantages if you are on a 'pay by time' connection as you can connect several machines as well as the phone and it also gives extra security in an internet cafe. In may cases the phone can be set up to log in automatically to a Sky or similar Wifi connection.

First we need to set up and pair the Android Phone to the Computer

screenshot

That is basically it - when you click on the network manager applet in the panel you should have an extra heading Mobile Broadband with the new connection GT-I8200N under it.

screenshot

You only need to pair once after which you just need to click on the GT-I8200 connection after ensuring the following are satisfied:

To disconnect or after disconnection

I have found it is possible to make duplicate connections which show up in the network Manager Applet and there is ne simple way to remove them. The only way seems to be to do to the folder that contains them /etc/NetworkManager/system-connections/ and delete the duplicates - you need root privaledges so use

gksu /etc/NetworkManager/system-connections/

There is no quick links in the notification panel on the Android phone for switching tethering on and off but you can add a Settings widget to the home screen to take you straight to the tethering settings.

Note that you can use data very fast from a computer especially if there are automatic upgrades etc. Experiment with a Wifi connection on the phone first and use a data monitoring applet such as NUMA. Check data use frequently on the Phone.

More about the connections used by Android for Bluetooth Tethering - PAN, NAP and BNEP

The Android Bluetooth tethered connection uses a Personal Area Network (PAN) which is a computer network used for data transmission among devices such as computers, telephones and personal digital assistants. PANs can be used for communication among the personal devices themselves or for connecting to a higher level network and the Internet. A Bluetooth PAN is composed of up to 8 active devices in a master-slave relationship (a very large number of devices can be connected in "parked" mode). The first Bluetooth device is the master or in this case the Network Access Point (NAP), and all other devices are slaves that communicate with the master. The communication protocol is the Bluetooth Network Encapsulation Protocol (BNEP) which is used to transport common networking protocols over the Bluetooth media such as IPv4 and IPv6. The packet format is based on Ethernet.

The Bluetooth manager detects that the Android device supports PAN/NAP and, by ticking the box, creates a connection for the Network Manager. The Network Manager (in the form of the applet) makes this available but the support is incomplete. As mentioned above one easily ends up with duplicate connection which have to be deleted from the terminal as root. The connection is also identified by the Bluetooth address and it delivers this rather than the actual connection interface which is bnep0 and is displayed as a Mobile Broadband Connection which is an approximation to the truth! This causes a problem with my NUMA applet which depends on the interface delivered by the Network Manager so I have had to add an exception as I have with the ppp0 interfaces used by Mobile Broadband connection.

 

Blackberry - Sorting out Multiple Contacts and Calendars

Contacts:

1. Go into your address book
2. Hit the Option key and choose options
3. Type "rset"(will not show on screen)
4. Each entry will come up in turn and give an opportunity to clear all the address book data with a YES.

Background from Crackberry:

You can use "RSET" with both calendar and contacts. Its the same process for both, so I'll use the calendar application as an example. Pull up your CALENDAR application, go to OPTIONS, and type "RSET", just like that, out of no where, a dialogue box will appear asking you if you want to wipe-out the calendar (don't worry about erasing all calendars, a separate dialogue box will appear for each calendar item in your berry). If you only want to wipe out a particular calendar, simply hit "NO" to the calendar(s) you want to keep as is and hit "YES" to the one(s) you want wiped. Same exact process for contacts applies. For both, if you do decide to wipe out a CALENDAR and/or CONTACTS profile, and it is linked to an OTA-sync account, as soon as the wipe-out is complete, the berry will try to OTA sync again.

Calendars (cical enties)

Each calendar has a cical entry which can be removed to prevent future syncs by:

go from home screen -> options -> advanced options - > service book find and delete CICAL for calenders you do not want to sync

Video Capture Device - EasyCAP USBTV007 version by Fushical

There are a number of USB devices available cheaply and branded EasyCAP which contain a number of different chips. Few of the sellers give any indications which. All have some support under Linux and the sources of information include:

LinuxTV has a good reference article on the various versions of EasyCAP, how to identify them, and what drivers are available under Linux

TV viewing and recording solutions for Linux is a blog with a lot of useful suggestions, although written for the STK1160 device based EasyCAP much is applicable to capturing from all the devices - especially the page on the mtvcgui program.

The USBTV007 version is arguably the least desirable as the drivers currently available do not support sound capture although all that means in practice is that one needs an extra cable from the video source to the sound input on the computer. You also need a very up to date Kernel to get out the latest usbtv module to provide of the box support - the 3.13.0-24 kernel in Ubuntu 14.04 Trusty and Mint 17 Quiana is fine and supports PAL video unlike earlier versions but not audio. There is also a module under development which you can build which adds audio which I have not tried.

I have done most of the testing so far using VLC which is a very good player but also has transcoding and capture built in. There are a few 'tricks' to using it for capture

DVD Video - Kino and Handbrake

Kino to Capture the DV Video

I usually capture video from the MiniDV Video Recorders using Kino as DV AV Type 2 with Auto Split Files and Add Time Stamp set. The video files in all the DV formats are huge (12 Gbytes/Hour) so I have been looking at converting to use a MP4 container with an efficient H.264 video stream and AAC audio which reduces it down to circa 2.5 Gbytes/Hour.

Handbrake to Convert the DV AV Video to H.264

This can be done directly by FFMpeg but it is difficult to get optimum parameters, some of which may need to be optimised for the form of video, especially if one wants to deinterlace at the same time. It is capable of converting a folder full of DV captures at one time - Kino is set up to put each recording into a separate file with a timestamp in my default set up so this is essential as an hour tape often contains over 200 scenes.

Unfortunately the latest versions of Handbrake in the Ubuntu repositories do not handle MP4 because Ubuntu do not want to distribute software with MP4 support because it is patent-encumbered. Installing the "restricted-extras" package solves the problem for many proprietary formats, but this alone is not sufficient for Handbrake and furthermore HandBrake does not support external encoders so it is not possible to add an encoder to HandBrake at runtime. All the encoder libraries are built in and not dynamically linked/loaded at runtime and Ubuntu has unhelpfully recompiled its version of Handbrake for the repositories without the MP4 support. The last time Handbrake was available from a PPA rather than the Ubuntu Repositories was for Ubuntu 13.04 raring but the appropriate stebbins handbrake-releases PPA (from the author) has been updated to a reasonably recent version of Handbrake. So how do we fudge things to install from a PPA for an older version of Handbrake?

Fudges to install a Linux version of Handbrake which will convert to MP4.

deb http://ppa.launchpad.net/stebbins/handbrake-releases/ubuntu raring main
deb-src http://ppa.launchpad.net/stebbins/handbrake-releases/ubuntu raring main

sudo apt-get update
sudo apt-get install handbrake-gtk

and you will have a version of Handbrake with MP4 support.

Converting a folder with Handbrake.

The commands are not immediately obvious and one uses Source to select a Folder then Queue -> Add All Queue then Queue to check and start.

Deinterlacing

Deinterlacing is required to remove the comb artifacts from interlaced video when there is any motion in the scene. Deinterlacing however reduces the quality so it's not good to run a deinterlacer all the time and it can really slow down an encode. Often content is mixed, cutting back and forth between interlaced and progressive video. Even in interlaced content, filtering is only necessary when either something in the scene or the camera moves, producing those combing artifacts.

Handbrake has a clever alternative called decomb. The decomb filter looks at each pixel of each frame of a video. It then only deinterlaces frames that show visible amounts of combing. This means you never have to check if a video you're encoding is interlaced -- just run the decomb filter all the time and it'll take care of everything.

The Deinterlacing is set up from Picture Settings -> Filters and Decomb and Default should be used unless you know that the video was recorded entirely with progressive video. The setting however does not seem to be sticky so you need to remember to set it.

Other Changed Settings

I have been using the Normal preset but have tried increasing the x264 preset to Medium which increases the encoding time considerably.

Changing from a .m4v to .mp4 extension.

By default Handbrake adds a .m4v extension to MP4 containers. That is the only difference and it is perfectly acceptable to just change the extension of the files. The following command line string will change all files in the folder where it is run and starting with DV with a .m4v extension to .mp4. The DV start matches my MiniDV capture files and acts as a protection in case one calls it from the wrong folder!

pcurtis@xxxxxx:/media/DATA/Video Transcodes$ for i in DV*.m4v;do eval mv \"$i\" \"$(basename "$i" m4v)mp4\";done
Concatenating all the video clips in a folder.

FFMpeg has a utility to concatenate video files fast and losslessly but Ubuntu has switched to a brach from FFMpeg called AVconv which does not include this.

I have installed the original FFMpeg from a PPA which may cause problems as the two overlap and also a number of libraries were updated which avconv uses but this did allow me to test out the concatenation. The two stages are to create a file containing the list of files to concatenate and then use that. If the files all have the same containers and streams this is fast and lossless. See https://trac.ffmpeg.org/wiki/How%20to%20concatenate%20(join,%20merge)%20media%20files

printf "file '%s'\n" ./*.m4v > DVlist.txt ; ffmpeg -f concat -i DVlist.txt -c copy AllClips.mp4

Note that I have used an intermediate file rather than piping as I found it gave problems and also it is useful to have the file list. Also note that the output file has a different extension for the same container to avoid duplicating if the command is run a second time.

12 November 2014

Back to GIT

The following is a log of what I did to try and backfill in GIT after I had made one or two significant changes to my three applets and wanted to be able to track them properly. It had been many months since I had used GIT so I wanted to avoid damaging what I already had whilst it came back to me. So I cloned the existing repository onto another machine so I could 'practice' until I was happy to push it back to the server. I then progressively copied the changed files from the first update across and did a commit, then the next set of updates and commited and so on. I had a few mistakes and the log shows the brute force way I used to backtrack to start again from before the mistake. The most common mistake is to have the wrong message with a commit and I found the git commit --amend did not actually make any changes hence the forced reset. See below for why --amend did not work.

In the end it all worked out well so I pushed to the server from the second machine and then cloned back to my normal machine after backing up the original. One should have been able to just fetch but I wanted an easy life! I then added a few tags to the Uploaded versions which were available to users.

It starts with installing git and the git viewer gitk which helps enormously to visualise where one has got to.

I have added a few comments on what each step was.

pcurtis@matrix:~$ sudo apt-get install git gitk

// See if we have any previous git configuration otherwise we would have needed to set user.name and user.email

pcurtis@matrix:~$ git config --list
user.name=pdcurtis
user.email=gitmail@pcurtis.com

// OK so we can clone

pcurtis@matrix:~$ git clone https://github.com/pdcurtis/cinnamon-applets
Cloning into 'cinnamon-applets'...
remote: Counting objects: 190, done.
remote: Compressing objects: 100% (88/88), done.
remote: Total 190 (delta 105), reused 181 (delta 97)
Receiving objects: 100% (190/190), 81.08 KiB | 0 bytes/s, done.
Resolving deltas: 100% (105/105), done.
Checking connectivity... done.

// Change into working folder

pcurtis@matrix:~$ cd cinnamon-applets

// run gitk to see whats going on (use & so spawned process and we still have terminal available)

pcurtis@matrix:~/cinnamon-applets$ gitk &

pcurtis@matrix:~/cinnamon-applets$ git status

// Replace the changed files for Bumblebee from latest applet

curtis@matrix:~/cinnamon-applets$ git commit -a -m "Stopwatch v20_0.9.8 Check we have a valid temperature returned from nvidia-settings"

// Have made a mistake in message so go back one step to previous state - BRUTE FORCE -there are better ways

pcurtis@matrix:~/cinnamon-applets$ git reset --hard HEAD~1

// Needed to copy back the changed files again as the reset removed the changes

pcurtis@matrix:~/cinnamon-applets$ git commit -a -m "Bumblebee 0.9.8 Check valid temperature returned from nvidia-settings - UPLOADED 11 November 2014"
[master 5ecbc4b] Bumblebee 0.9.8 Check valid temperature returned from nvidia-settings - UPLOADED 11 November 2014
2 files changed, 19 insertions(+), 17 deletions(-)

====All the changed files in NUMA 2.5.0 copied over existing ones and then commit ======================

pcurtis@matrix:~/cinnamon-applets$ git commit -a -m "NUMA - 2.5.0 Multiple Instance capability but set to 1"
[master ba64e78] NUMA - 2.5.0 Multiple Instance capability but set to 1
4 files changed, 19 insertions(+), 6 deletions(-)

====All the changed files in 2.6.0 copied over existing ones======================
// Now commit the changes

pcurtis@matrix:~/cinnamon-applets$ git commit -a -m "NUMA 2.6.0 Android Bluetooth via bnep0 added. FormatSentReceived improved UPLOAD CANDIDATE"
[master d28539b] NUMA 2.6.0 Android Bluetooth via bnep0 added. FormatSentReceived improved UPLOAD CANDIDATE
2 files changed, 40 insertions(+), 13 deletions(-)

// push all of the changes which have been committed up to the server

pcurtis@matrix:~/cinnamon-applets$ git push origin
warning: push.default is unset; its implicit value is changing in
Git 2.0 from 'matching' to 'simple'. To squelch this message
and maintain the current behavior after the default changes, use:

git config --global push.default matching

To squelch this message and adopt the new behaviour now, use:

git config --global push.default simple

When push.default is set to 'matching', git will push local branches
to the remote branches that already exist with the same name.

In Git 2.0, Git will default to the more conservative 'simple'
behavior, which only pushes the current branch to the corresponding
remote branch that 'git pull' uses to update the current branch.

See 'git help config' and search for 'push.default' for further information.
(the 'simple' mode was introduced in Git 1.7.11. Use the similar mode
'current' instead of 'simple' if you sometimes use older versions of Git)

Username for 'https://github.com': pdcurtis
Password for 'https://pdcurtis@github.com':
Counting objects: 24, done.
Delta compression using up to 2 threads.
Compressing objects: 100% (17/17), done.
Writing objects: 100% (17/17), 17.84 KiB | 0 bytes/s, done.
Total 17 (delta 11), reused 0 (delta 0)
To https://github.com/pdcurtis/cinnamon-applets
3983078..d28539b master -> master

========== Now add Tags =====================
// What tags have we got already?
pcurtis@defiant:~/cinnamon-applets$ git tag
Bumblebee-0.9.5
Bumblebee-0.9.7
NUMA-2.3.17
NUMA-2.3.7
NUMA-2.4.2
Stopwatch-1.2.2

// Find out id of recent commits Ctrl-Z to exit

pcurtis@defiant:~/cinnamon-applets$ git log --pretty=oneline
d28539b88745be0b623a7fb1d5b02bffe7bff84a NUMA 2.6.0 Android Bluetooth via bnep0 added. FormatSentReceived improved UPLOA
ba64e7898def3c4ee4b90c8e534d2aedd56a3e70 NUMA - 2.5.0 Multiple Instance capability but set to 1
5ecbc4bfba802b1fcf633422fc4a1d38fd08b3d4 Bumblebee 0.9.8 Check valid temperature returned from nvidia-settings - UPLOADE
39830788231d9fa3eaee7ecd51d6e6c78224baeb Changes in padding and in width calculation for compact display and modified fu
f9cb13fa486a8258848c2f410d1d871eb5bf7fe4 Changes to arrangements for padding to get round removal of leading spaces, red
9298ec601eb6ddcfc4c65b55ea2d210e999ff5f1 NUMA 2.4.4 Removal of some commented out text and change log updated
27f567b7ec313ef8d5c7a85399182f67dc060127 NUMA 2.4.3 Addition of offsets to cumulative monitoring, reduced size applet wh
716cffa0bc97a59e379dde8f864754aca35d5fb1 Stopwatch - edit comments
e8c0660ee63ee6c8ac812b3bd4f7bf21131692c9 Update gitignore
fc9ee1d0414a8969f9c6738b2c313c182ed43fab Stopwatch version 1.2.3
cf84b6ae7e20712c78bd4d1b84f9806ffba0612a Bumblebee version 0.9.7 Uploaded 2 April 2014
8fc32db89cb3a8b22be25c7a228816d3286ec149 Bumblebee version 0.9.5 Uploaded January 7 2014
4d4344f56de8008006f3063d673b099f7f569177 Stowatch version 1.2.2 Uploaded 2 April 2014

// Add first simple tag

pcurtis@defiant:~/cinnamon-applets$ git tag Bumblebee-0.9.8 5ecbc4bfba802b1fcf633422fc4a1d38fd08b3d

// and so on

// Tags have to be pushed separately. --tags adds all not yet present

pcurtis@defiant:~/cinnamon-applets$ git push origin --tags
Username for 'https://github.com': pdcurtis
Password for 'https://pdcurtis@github.com':
Total 0 (delta 0), reused 0 (delta 0)
To https://github.com/pdcurtis/cinnamon-applets
* [new tag] Bumblebee-0.9.8 -> Bumblebee-0.9.8

// Check again

pcurtis@defiant:~/cinnamon-applets$ git tag
Bumblebee-0.9.5
Bumblebee-0.9.7
Bumblebee-0.9.8
NUMA-2.3.17
NUMA-2.3.7
NUMA-2.4.2
Stopwatch-1.2.2
Stopwatch-1.2.3
pcurtis@defiant:~/cinnamon-applets$

// Backfill done - now we can carry on in proper manner!

==================== Useful Commands ================

git diff - shows the differences from current to staged
pcurtis@defiant:~$ git difftool --tool-help
'git difftool --tool=<tool>' may be set to one of the following:
meld
The following tools are valid, but not currently available:
araxis
bc3
codecompare
deltawalker
diffmerge
diffuse
ecmerge
emerge
gvimdiff
gvimdiff2
kdiff3
kompare
opendiff
p4merge
tkdiff
vimdiff
vimdiff2
xxdiff
Some of the tools listed above only work in a windowed
environment. If run in a terminal-only session, they will fail.

pcurtis@defiant:~$ git difftool --tool=meld
Not a git repository
To compare two paths outside a working tree:
usage: git diff [--no-index] <path> <path>
pcurtis@defiant:~$ cd cinnamon-applets
pcurtis@defiant:~/cinnamon-applets$ git difftool --tool=meld

-------------------------------------------------------------

pcurtis@defiant:~/cinnamon-applets$ sudo apt-get install leafpad
....
pcurtis@defiant:~/cinnamon-applets$ git config --list
user.name=pdcurtis
user.email=gitmail@pcurtis.com
merge.tool=meld
core.repositoryformatversion=0
core.filemode=true
core.bare=false
core.logallrefupdates=true
remote.origin.url=https://github.com/pdcurtis/cinnamon-applets
remote.origin.fetch=+refs/heads/*:refs/remotes/origin/*
branch.master.remote=origin
branch.master.merge=refs/heads/master
pcurtis@defiant:~/cinnamon-applets$ git config --global core.editor leafpad
pcurtis@defiant:~/cinnamon-applets$ git config --list
user.name=pdcurtis
user.email=gitmail@pcurtis.com
merge.tool=meld
core.editor=leafpad
core.repositoryformatversion=0
core.filemode=true
core.bare=false
core.logallrefupdates=true
remote.origin.url=https://github.com/pdcurtis/cinnamon-applets
remote.origin.fetch=+refs/heads/*:refs/remotes/origin/*
branch.master.remote=origin
branch.master.merge=refs/heads/master
pcurtis@defiant:~/cinnamon-applets$

 

Using an alternative/external diff tool in GIT

There is a very good tool for looking at differences between files which is called meld. Once this is installed one can tell git to use by git difftool --tool=meld when looking at difference after which one can use git difftool instead of git diff

Bug in GIT when using Gedit

A number of GIT commands automatically open a copy of your default editor (gedit on Ubuntu and Mint and load text into it for editing including git commit and git commit --amend if a copy of the gedit editor is already open. Unfortunately although it appears you have saved the temporary file and closed it the text is not used. Most of the time you will be editing your text using gedit so this is a bad situation as you do not want to have to remember to close gedit every time you commit. I want to use a GUI editor for the commit text so I came up with the idea of also loading leafpad, the lightweight editor used by Lubuntu and setting that to be the default for GIT and that works perfectly and leafpad is a perfectly adequate editor for the simple jobs withing GIT. So we need to install leafpad and then configure git by

sudo apt-get install leafpad
git config --global core.editor leafpad

Putting all my standard git configuration actions together when working on a fresh machine.

If we add together all the various configuration which I have applied to git, the follow is required if I set up on another machine including obtaining the local copy.

sudo apt-get install git gitk leafpad meld
git clone https://github.com/pdcurtis/cinnamon-applets
cd cinnamon-applets
git config --global user.name "pdcurtis"
git config --global user.email "gitmail_at_pcurtis.com"
git config --global core.editor leafpad
git difftool --tool=meld

Testing Configurations

We now have the ability to develop the applets but for test they need to be within ~/.local/share/cinnamon/applets.

Linking

I have currently linked the applets I have been working on rather than make copies but this has the disadvantage that an an accidental update from the Cinnamon Spices web site could lose a lot of updates unless git had been used to commit frequently and I wonder if a copy might be better

ln -s ~/cinnamon-applets/bumblebee@pdcurtis ~/.local/share/cinnamon/applets/bumblebee@pdcurtis

 

Link to W3C HTML5 Validator Copyright © Peter & Pauline Curtis
Content revised: 8th July, 2020