Home Uniquely NZ Travel Howto Pauline Small Firms Search
Diary of System and Website Development
Part 27 (January 2016 - October 2016)

10th January 2016

Videos - FlashPlayer, Youtube and BBC iPlayer

I have found that a number of videos I am interested in have disappeared so I have been looking into various ways to download them to ensure they are available for my reference and future research. What started this off was that I found a video relating to the Broken Hills Historic Mine made by TVNZ (http://tvnz.co.nz/national-news/file-video-inside-broken-hills-gold-mine-4728919) disappeared almost whilst I was watching it and it contained valuable information used as background in my write up of the highlight of our trip into the Broken Hills Historic Mine in the Coromandel in New Zealand.

I have there fore been investigating ways of saving such videos on my local machine for my own personal research use. Two obvious places to start were Youtube and BBC iPlayer as examples of different but up-to-date formats. During my initial work I also found that I was running into problems with Adobe Flash under Linux. Adobe no longer support Flash updates for Linux and Firefox has now completely blocked any access. Youtube and the BBC until recently used Flash based technology but now Youtube use the inbuilt video facilities with HTML5. As far as I can see the BBC is still using the outdated and highly insecure Adobe Flash for both the BBC News site and iPlayer and I will have to look to a way to enable an updated version to be used.

I did my usual web searches and located ways to get round the the Flash problems and to be able to download videos and not surprisingly some of the best information and solutions were on Webup8 and the most of the required software was already within one of their PPAs which I already have installed.

Updating Flash Player using Fresh Player: Lets look at Flash first as it is a showstopper, or soon will be, for all Linux systems. The articles of interest are http://www.webupd8.org/2015/06/fresh-player-plugin-gets-hardware.html and http://www.webupd8.org/2014/05/install-fresh-player-plugin-in-ubuntu.html. The first and most recent article describes the latest ability to use hardware acceleration which is not possible under ubuntu 14.04 Trusty which Mint 17.3 is based on but it makes interesting reading for when Mint 18 comes on line.

First we need to install Fresh Player Plugin in Ubuntu via the webup8 PPA, by using the following commands:

sudo add-apt-repository ppa:nilarimogard/webupd8
sudo apt-get update
sudo apt-get install freshplayerplugin gdebi

Fresh Player Plugin is however just a wrapper that allows Linux users to use Pepper Flash . This is useful because the latest Flash Player is only available for Google Chrome (it comes bundled with it) on Linux, while other browsers, like Firefox, are stuck with an old Adobe Flash Player version (11.2)

Fresh Player Plugin is however just a wrapper to enable Firefox and other NPAPI (Netscape Plugin Application Programming Interface) compatible browsers to use the Adobe Flash Player that's bundled with Google Chrome which in the form of a PPAPI (Pepper Plugin API) plugin because Mozilla isn't interested in adding support for the Pepper Plugin API and only supports the old NPAPI format.

So how do we get hold of the new plugin now we have the capability to use it despite Firefoxes best efforts to prevent it? This is actually quite easy provided one is prepared to compromise and load a another piece of non-opensource software namely Google Chrome as well or instead of the OpenSource Chromium Browser. Once that is loaded then Firefox will automatically find the new plugin which is bundled with Google Chrome and use it. I found I ended up with two versions of Flash Player but the newer one seemed to be used although I found disabling the old one prevented both from working. I think the answer is to remove the old version first. The latest version of Google Chrome can be downloaded from here - save the appropriate file and install using gdebi. gdebi may not be installed already but you will note I included installing it above in preparation.

At a minimum you will need to restart Firefox and may have to look at Tools -> Add-ons -> Plugins. If you still have two versions both need to be enabled but the new one will be used.

Downloading YouTube Videos: Now we have dealt with Flash lets have a look at Youtube videos. You tube videos are streamed to you machine and have to be buffered and saved in an appropriate format by your own software as Youtube no longer provide there own support. Downloading gives security that the video will not be removed and, of course, removes the huge overhead of continuous streaming of video.

Nor surprisingly some very good open source software has been written for Linux and the favourite is youtube-dl is a command line tool that allows downloading videos from YouTube and many other similar websites. This is available from the same webup8 PPA as Fresh Player so the instructions are already in place above. There is a version in the Ubuntu/Mint Repositories but it is out of date and no longer works.

The latest youtube-dl supports: YouTube (besides single videos it also supports playlists, searches and user videos), metacafe.com, Google Video (including searches), Pinsthotobucket, Dailymotion, DepositFiles, blip.tv, Vimeo, myvideo.de, The Daily Show / Colbert Nation, The Escapist, CollegeHumor, arte.tv, Soundcloud, xvideos, infoq, mixcloud, Stanford Open Content, Youku, MTV, Xnxx, Google Plus, hotnewhiphop.com, auengine.com, gamespot.com, RingTV, wat.tv, traileraddict.com, tu.tv, instagram.com, Statigr.am, break.com, tudou.com, Jukebox, Wimp.com, CSpan, 3sat, brightcove.com and archive.org and there's also a generic downloader that works with some other sites.

The application comes with many useful features, including options to specify the video quality/format, add the original title to the downloaded video filename, proxy support, automatically extract audio from downloaded videos and more. It can be installed by the following but note that you may already have the PPA from above.

sudo add-apt-repository ppa:nilarimogard/webupd8
sudo apt-get update
sudo apt-get install youtube-dl

To download a video in what is regarded as the best format you just have to use the following command:

youtube-dl http://VIDEO_URL

where "VIDEO_URL" is the url of the video you want to download.

Normally you will want to see the available formats first by

youtube-dl -F http://VIDEO_URL

And then pick a suitable format number and download by

youtube-dl -f 123 http://VIDEO_URL

Where 123 should be replaced by your chosen format - note that options include audio only and video only formats, I found one video was very quiet until I realised!

The following shows one of my own videos which is at the bottom of this page that I downloaded as a test - see how easy it was and how well it worked. Note I deliberately used a medium -f 18 resolution to avoid a huge download.

$ youtube-dl -F https://www.youtube.com/watch?v=nVwqw8CWfaE
[youtube] nVwqw8CWfaE: Downloading webpage
[youtube] nVwqw8CWfaE: Downloading video info webpage
[youtube] nVwqw8CWfaE: Extracting video information
[youtube] nVwqw8CWfaE: Downloading DASH manifest
[youtube] nVwqw8CWfaE: Downloading DASH manifest
[info] Available formats for nVwqw8CWfaE:
format code extension resolution note
140 m4a audio only DASH audio 128k , m4a_dash container, aac @128k (44100Hz), 8.44MiB
160 mp4 256x144 DASH video 114k , avc1.4d400c, 25fps, video only, 7.39MiB
133 mp4 426x240 DASH video 251k , avc1.4d4015, 25fps, video only, 16.36MiB
134 mp4 640x360 DASH video 611k , avc1.4d401e, 25fps, video only, 33.64MiB
135 mp4 854x480 DASH video 1117k , avc1.4d401e, 25fps, video only, 65.87MiB
136 mp4 1280x720 DASH video 2220k , avc1.4d401f, 25fps, video only, 134.98MiB
17 3gp 176x144 small , mp4a.40.2, mp4v.20.3
36 3gp 320x240 small , mp4a.40.2, mp4v.20.3
5 flv 400x240 small
43 webm 640x360 medium , vorbis, vp8.0
18 mp4 640x360 medium , mp4a.40.2, avc1.42001E
22 mp4 1280x720 hd720 , mp4a.40.2, avc1.64001F (best)
$ youtube-dl -f 18 https://www.youtube.com/watch?v=nVwqw8CWfaE
[youtube] nVwqw8CWfaE: Downloading webpage
[youtube] nVwqw8CWfaE: Downloading video info webpage
[youtube] nVwqw8CWfaE: Extracting video information
[youtube] nVwqw8CWfaE: Downloading DASH manifest
[youtube] nVwqw8CWfaE: Downloading DASH manifest
[download] Destination: Madeira New Year's Eve Fireworks 2015_2016-nVwqw8CWfaE.mp4
[download] 100% of 42.63MiB in 00:33
$

Note the download is into the Folder from which youtube-dl is run but a different folder can be specified using the -o option. Do man youtube-dl if you have plenty of time to find out all its facilities.

Downloading BBC iPlayer Videos: This is not quite so easy as the BBC try to restrict the life of the iPlayer replays to 30 days and then sell the videos. Unfortunately there is no way to download or buy for Linux or even Android at present so again one has to use dedicated software and it is suggested that the 30 days is adhered to unless for uses such as research or the video has been bought (note you will probably not be able to play your bought version under Linux) .

Again the software in the repositories is out of date and we need to access the latest version via a PPA:

sudo apt-add-repository ppa:jon-hedgerows/get-iplayer
sudo apt-get update
sudo apt-get install get-iplayer rtmpdump

There are a number of conflicting instructions on how to search for and download a iPlayer Video. The only way I have made to work so far is to get the Address in iPlayer when it is being played and use that as below using the --pid http//IPLAYER_URL option

get_iplayer --pid http://www.bbc.co.uk/iplayer/episode/bumffumb/title

The file is downloaded to ones home folder as an H264 MPEG (.mp4) resolution 832x468 and 25 frame rate.

11th January 2016

YouTube Videos

It is actually very easy to upload videos to YouTube provided you already have gmail account as you need to do nothing more than click an upload button on virtually any youtube or google page when you are logged in. You will have to confirm the names which are displayed and then you are ready. Most formats can be converted but mp4 are probably the best. The upload can take quite a long time - perhaps 5 times the running time of the video and during that time you can enter a title, summary etc and look at the advanced screens. There are a good set of instructions at https://support.google.com/youtube/topic/2888648?hl=en

You can easily get back to your Videos in what is called My Channel by going to the youtube home page and when you are logged in you will see a panel on the left with an option called My Channel which gets you to your list of videos.

Once the video is uploaded and you have got to it via My Channels you can do a number of basic enhancements by clicking on Enhancements below the video. The most basic but very useful is to trim the video for start and finish times. I have so far uploaded two videos of the Fireworks and have truncated the end of both to the point where the fireworks stopped and in the case of the 2013/2014 I also took off some of the initial time waiting until Big Ben struck. You can also do a stabilisation to remove shake, I tried that on one of them. I do not think that colour corrections are useful for fireworks but there are times they could be.

There is also a comprehensive Video Editor but I see little point in using it rather than working offline with kdenlive especially as I noticed a very small warning that advertising material may be added.

One important consideration is that it is easy to delete material. In My Channels click Manage Videos and you can tick a video and you will find delete in the drop down actions box as well as a number of other useful things like changing the licence type.

The embedding of the video is very simple and the embedding code for any youtube video can be found from the share link when you look at the video On The Youtube Site where you will find it offers three tabs the middle being embed on which you find a show more option which provides code for different sizes etc you can embed.

Making Embedded YouTube and other Videos Responsive

Videos have a reputation of being difficult to add to responsive pages. I have however found a reasonably simple way to make the size respond to different sizes of screen. This is done by wrapping the code which embeds the video in a way that it shrinks with the page simply by by using css. The ideas came from https://css-tricks.com/NetMag/FluidWidthVideo/Article-FluidWidthVideo.php and http://stackoverflow.com/questions/15844500/shrink-a-youtube-video-to-responsive-width but I have extended it to also include centring the video and adding a small margin. This technique should work with most modern browsers

The following is added to the end of the script .css file (responsive.css)

.videoWrapper {
position: relative;
padding-bottom: 56.25%; /* 16:9 */
padding-top: 25px;
height: 0;
}
.videoWrapper iframe {
position: absolute;
top: 5%;
left: 5%;
width: 90%;
height: 90%;
}

The 25 pixels is for the 'candy' on the player (controls) and the margins centre the player in the responsive 'box'

And the video code is wrapped like this:

<div class="videoWrapper">
   <iframe width="832" height="468" src="https://www.youtube.com/embed/nVwqw8CWfaE?rel=0" frameborder="0" allowfullscreen></iframe>
</div>

Note the <iframe .... > is all one line obtained from the youtube site. I have however modified the width and height to match my 930 pixel html body width when using a 5% margin either side.

12th January 2016

HTML Tidy (Tidy-HTML5)

I use tidy to check the html - the original tidy did not handle html5 so Tidy-html5 was developed, abondened, forked and now exists again at https://github.com/htacg/tidy-html5

There are binaries at http://binaries.html-tidy.org/.

I currently have HTML Tidy version 5.0.0.RC1 installed by:

sudo apt-get remove libtidy-0.99-0 tidy -y
sudo wget -O tidy-5.0.0-64bit.deb http://www.htacg.org/binaries/binaries/tidy-5.0.0.RC1/tidy-5.0.0-64bit.deb
sudo gdebi tidy-5.0.0-64bit.deb

Now installed as tidy not tidy5 or tidy-html5 so example of use on this file is:

$ cd "/media/DATA/My Web Site" && tidy -e diary27.htm

line 138 column 46 - Warning: adjacent hyphens within comment
Info: Document content looks like HTML5
1 warning, 0 errors were found!

About HTML Tidy: https://github.com/htacg/tidy-html5
Bug reports and comments: https://github.com/htacg/tidy-html5/issues
Or send questions and comments to: https://lists.w3.org/Archives/Public/public-htacg/
Latest HTML specification: http://dev.w3.org/html5/spec-author-view/
Validate your HTML documents: http://validator.w3.org/nu/
Lobby your company to join the W3C: http://www.w3.org/Consortium
/media/DATA/My Web Site$

13th January 2016

Sharing programs

I have just realised that I have not documented a number of programs that I use to share information between our Laptops, Phones and Pads. Several are dependent on Dropbox being present. The main ones are:

Dropbox/Dropsync

The Android version of Dropbox does not carry out any automatic syncronisation, Dropsync is an App that enables automatic sync on selected files/folders to allow other programs to keep up-t0-date

Evernote/Nixnote

Evernote is a popular note-taking service that supports saving text, full webpages, voice memos, video notes and more with a lot of useful features. There are official Evernote clients available for Windows, OS X, web (but it lacks many features) and mobile platforms including Android but not for Linux. Evernote supplies a limited amount of storage in its free version.

NixNote is an unofficial Evernote client for Linux. The application is writen in C++ using the Qt framework, having good performance and a smalll memory footprin. The application currently uses Java for encrypting and decrypting text, but that's optional.

NixNote does not have a PPA yet so one needs to download the .deb install package and occassionally update manually - it is so good it is worth the extra work. Download link for NixNote beta ( deb, rpm and ebuild packages available).

To enable syncing with Evernote, from the NixNote 2 menu select Tools > Synchronize and authenticate NixNote 2 to access the Evernote Cloud storage.

Keepass2/KeePass2Android (Requires Dropbox and Dropsync)

Todo.txt/jdotxt (Requires Dropbox and Dropsync)

Google Calendar/Thunderbird-Lightning via Provider for Google Calendar (2.7) Extension

Google Contacts/Thunderbird-Lightning via gContactSync (3.0.0) Extension

Useful Mobile Facts

Find SIM number *#100#

14th January 2016

Linking an Android Phone GPS to OpenCPN Nautical Mapping Program

See http://opencpn.org/ocpn/Start_gps-setup for background

Install Share GPS App on Phone

Pair Phone and Computer if not already done so

Find out Address of Phone in terminal by:

$ sudo hcitool scan
Scanning ...
00:0E:07:2C:E8:82 T610
08:3D:88:DA:A7:A5 Peter Curtis (GT-I9301

Find out the Channel corresponding to GPS - below is part of long list of channels

$ sdptool records 08:3D:88:DA:A7:A5

............................
Service Name: ShareGPS

Service RecHandle: 0x1000e
Service Class ID List:
"Serial Port" (0x1101)
Protocol Descriptor List:
"L2CAP" (0x0100)
"RFCOMM" (0x0003)
Channel: 5

Edit /etc/bluetooth/rfcomm.conf as root to add the binding

$ sudo gedit /etc/bluetooth/rfcomm.conf

# RFCOMM configuration file /etc/bluetooth/rfcomm.conf

rfcomm0 {
bind yes;
# Bluetooth address of T610
device 00:0E:07:2C:E8:82 ;
# RFCOMM channel for the T610
channel 1;
# Description of the connection
comment "T610 Dial-up Networking via Bluetooth";
}

# ....................

rfcomm4 {
bind yes;
# Bluetooth address of Samsung S3 Evo
device 08:3D:88:DA:A7:A5 ;
# RFCOMM channel for the GPS
channel 5;
# Description of the connection
comment "Samsung S3 GPS";
}

Save and restart machine or restart Bluetooth in a terminal:

~$ sudo service bluetooth restart
bluetooth stop/waiting
bluetooth start/running, process 9577
$

Now add a connection in OpenCPN to look like the following:

screenshot

screenshot

You should now be able to run Bluetooth Share Note: Make sure GPS and BT are on before running it and connect OpenCPN. You can setup/use the Dashboard to monitor the number of Satellites.

I used to use Bluetooth GPS Output by Meowsbox with equal success but needs a licence (£1.00) to keep it working more than 10 minutes and more importantly it showed as Virus Infected when I ran Defenx so I uninstalled it and lost my licence.

18th January 2016

UPDATE: Installing Googleearth 64 bit on Ubuntu 14.04 and Mint 17.x

If you just download the Google Earth package and try to install it, it fails to install because it depends on ia32-libs (on Ubuntu 64bit).

The ia32-libs package is no longer available in Ubuntu, starting with Ubuntu 13.10. The package was superseded by multiarch support so you don't need it any more, but some 64bit packages like Googleearth (which are actually 32bit applications) still depend on this package and because of this, they can't be installed in Ubuntu 14.04 or 13.10, 64bit.

The following 'correct'method of handling this is described on Webupd8 at http://www.webupd8.org/2014/04/install-google-earth-in-ubuntu-1404.html . It is to simply install the 32bit package on Ubuntu 64bit. Of course, that will install quite a few 32bit packages, but that's how multiarch works.

The extra problem with Google Earth is that the 32bit package doesn't support multiarch so it doesn't install all the 32bit dependencies it needs to run on Ubuntu 64bit. Below you'll find instructions on how to properly install Google Earth and its 32bit dependencies on Ubuntu 64bit.

So Webupd8 tells us the way to properly install Google Earth (along with the required 32bit dependencies) in Ubuntu 14.04 (or 13.10) 64bit, use the following commands:

sudo apt-get install libfontconfig1:i386 libx11-6:i386 libxrender1:i386 libxext6:i386 libgl1-mesa-glx:i386 libglu1-mesa:i386 libglib2.0-0:i386 libsm6:i386
cd /tmp && wget http://dl.google.com/dl/earth/client/current/google-earth-stable_current_i386.deb
sudo dpkg -i google-earth-stable_current_i386.deb
sudo apt-get install -f

Note some of these are long lines so make sureyou cut and paste the whole line into a terminal.

You will see errors after the first command but the final apt-get install -f seesm to sort all the dependencies out

 

3rd August 2016

Introduction to Git

This section has moved to An Introduction to Git part 1 - Applicaions in Cinnamon Applet Development what follows is an early draft.

 

This is the start of the new draft

This is the draft of a new page drawing on the existing Git page with new input. It is intended to help others who want to make use of my applet deelopment on github, contribute to that deveolment or set up their own Git repository for applet development or other puposes.

The initial input has come from the terminal output from my own work and some explanitory text and needs considerable integrating and formating before it become useful to others

Git is a distributed version control systems and every folder with a Git repository on every computer becomes a full-fledged independent repository with complete history and full version-tracking capabilities, without requiring online network access or a central server for its use.

I must starting by saying that using Git is not easy but why it is so useful that it is worth the learning curve. Firstly everything is local, you can work for long periods and then push the details to a remote server or get updates from it once and at your convenience. The second major advantage is that there is a simple mechanism to break away from the main development path to try things out simultaneously in many different areas and them bring them all back together through the branching and merging In fact many projects have a stable and devopment braches always running in parallel with the development branch periodically merged into the stable branch. If fact this can be done by a large team all working in parallel yet able to bring their work together. One can start a new branches from the current point (referred to as the head) or from points in the past and then switch between branches very easily and commit a series of changes on each branch. Every time you change (checkout) branch your working directory reflects whats in that branch and its history. If you are not successful you can just delete the branch or if you have solved the problem you can merge it back along with changes pulled from the server and changes in other branches. This is all very fine in theory but what happens if the same file has been changed in different places? The system detects that and forces you to resolve the differences. In its most basic form Git marks the conflicts within the files so you can edit them. It also allows you to use meld or other GUI utilities to help by showing you the files side by side and move, delete or otherwise resolve by editing the differences.

There are some GUI interfaces to use Git but you really do need to understand Git by running it from the terminal before using them. The gitk visualiser is however very helpful, if not essential. But before we start to look at the important commands we need a basic understanding of the principles behind how Git works.I took a long time to use git effectively and that was because I do not think I really understood the fundamentals of Git.

Firstly Git has a working area where you edit files and a repository where a series of snapshots of the working area are stored very efficiently. Note we are talking about snapshots not differences. You can get a snapshot into your working area by a checkout (as in supermarket!) To start with you probably have the simplest case with a single path (branch) by default called master and you are checked out at its HEAD. When you have made a series of edits on the files in your working area you need to take another snapshot and add it into the Git repository - this is called a commit and once that is done it is very difficult to lose data especially if your local repository is connected to a remote repository on a different machine. Git is also very efficient, for example the complete Git repository for the linux kernel back to 2005 when it was switched into Git is 2 Gbytes and that has every commit (snapshot) made since then. I suspect there are few larger and more complex projects.

Security is guaranteed as each snapshot (commit) is identified by it's checksum. The mechanism that Git uses for this checksum is called a SHA–1 hash. This is a 40-character string composed of hexadecimal characters (0-9 and a-f) and calculated based on the contents of a file or directory structure in Git. This is a unique identifier which also confirms that the contents have not been changed. These checksums are the fundamental way you identify a particular snapshot to, for example, return to and reload your working area (Checkout) or start a branch from. Each commit also has information attached to it including its parent(s) a Message and details of who made the commit.

The above are the most vital things you have to understand but are a simplification as the adding of the snapshot to the repository is actually two steps, the first is that individual files are added to a staging area and then those in the staging area are in the list to be committed. This unique “staging area” concept does allow you to determine exactly which changes shall be included in your next commits, even down to single lines. It is possible to modify the file further but the modifications will not be in the next commit unless it is restaged. Once a snapshot has been committed to the repository is very difficult to change. I have so far never needed to separately stage files and use a simplified command which stages and commits all changed files and adds the message. Even so you must be aware that it is actually two steps and eventually you may need the flexibility.

So far I have only been covering local activity and have ignored the fact that in most cases you will be working collaboratively or making your work openly available through a remote Git repository which you (or others) will push you commits to and pull or fetch the changes back to your machine to merge into your work. so it is time to look at how you might want to use Git.

  1. Take an existing 'project folder' such as an applet you are developing and add Git to it so you have an audit trail for your changes and backup for mistakes.
  2. Combine above with a hosted Git service like Github so you can make the work available to others and you can work on several machines as well as having a remote backup. This is what I have done with the applets.
  3. Start with a copy of an existing repository to make modifications for your own purposes, possibly adding patches which have not been included in the master at that time or testing patches with the intention of providing feedback or to get round problems. This is what I did to test and incorporate a patch to the kernel I was using on the Helios to get round some problems with the Skylake architecture used without waiting 6 months it to reach a kernel I could use.
  4. Fork a repository so you can contribute to a major project such as Cinnamon. Cinnamon has close to 1 million lines of code, over 5000 commits, 122 releases, 373 forks and 143 contributors. Only a handful of people can make changes to the master, in the case of Cinnamon 95% of commits come from just two people, it is the only way to keep control. In the case of the Linux kernel almost everything is finally approved by Linus Torvalds himself. A fork starts as a copy of the main repository on the server which is then cloned to your machine after which it can be updated with your contributions. You then create a pull request which, if and when approved, can be pulled into the master and merged. This allows total control over the process.

This time I am going to be concentrating on 3. and 4.

I have been looking for a good place to add some reference information and this seems as good a place as any before we get on to the real subject of this document.

Wikipedia - GIT Software - a good introduction to GIT history

The Pro GIT book, written by Scott Chacon in PDF form and now as a Second Edition pdf . This is the best of the references to me but be warned it takes about 150 pages to cover the most important areas (Chapters 1 - 3 and start of 6)

GIT Magic - A more informal but usefull approach - you may need to google for alternative downloads - last checked 7 Aug 2016

https://help.github.com/articles/set-up-gi - Github guide. Github.com is the server used for all the cinnamon system software which is free for open source software.

http://www.kernel.org/pub/software/scm/git/docs/user-manual.html - from the Linux Kernel Team

Use gitk to understand GIT - a very useful tutorial on use of gitk part 1

Use gitk to understand GIT merge and rebase useful tutorial for an essential tool part 2

We can now get to the main business of this document.

Using Git to manage Applet development

I had several applets already published before I decided it was time to bring them under proper control by using Git. They were already sub folders within an existing folder which I turned into a Git repository ( I will come to details of how latter) - this was kept entirely local to start with and for the period where I was trying to put earlier versions together to form a history to start everything going. Once I had worked back through some critical parts of the development I added a remote repository one of of the Git repositories hosting services. I had already working with GitHub when I was looking at Cinnamon. Github is the single largest host for Git repositories, and is the central point of collaboration for millions of developers and projects. A large percentage of all open source Git repositories are hosted on GitHub including Mint and Cinnamon. So while it’s not a direct part of the Git open source project, there’s a good chance that you’ll want or need to interact with GitHub at some point which is why I chose it. You do not need an account initially to clone my repository from Github.

What this mean, without going into details yet is that you can 'clone' my git repository which will give you a folder (usually within your home folder) containing all my current applet folders as well as a hidden folder called .git which contains all the information to go back through the development history and see every change made between 'commits' which can be thought of as a sort of snapshot. You can also go back ('checkout' in GitSpeak) to any of the 'commits' and every file in the working area will change to what it was at that time. The important points are when a version was uploaded so if my latest version does not work on your machine you can go back and collect the old version. All these important points have a 'tag' make them easy to find. As time goes on you can 'pull' my latest version down without having to clone again.

Git is however not the most user friendly piece of software and is mostly used from the command line. What I have described above is not difficult and there is a first class graphical visualisation program called gitk which makes it easy to see what the commands have done and in the limit you can just clone another time!

That is only a tiny part of what git can do. You can edit the files for your own purposes and can 'merge' my updates in automatically provided there are no conflicts. Even if there are there are ways to handle the conflicts manually. One of the most powerful features is that you can create 'branches' to make your modifications and test them and then merge them back to the master.

Git is now the most widely used source code management tool, with over 40% of professional software developers using Git as their primary source control system. I have therefore decided that my instructions for installing and setting up git will allow you to do a little than access the various versions (including those more recent than those I upload) but also provide the tools to effective make your own changes and merge tham with any I am making. In theory you could set up your own repository on github and provide your enhancements or bug fixes to me to 'pull' into my version although I am not sure how I do that yet! Before you worry the whole installation and setup is done in about 8 terminal commands cut and pasted from below.

sudo apt-get install git gitk leafpad meld
git clone https://github.com/pdcurtis/cinnamon-applets
cd cinnamon-applets
git config --global core.editor leafpad
git config --global diff.tool meld
git config --global merge.tool meld

git config --global user.name "your_username_on_github"
git config --global user.email "yourname@your_git_email"

The use of a simple text editor called leafpad avoids all sorts of problems in using the same editor for your editing of files and within git. meld is a really good difference tool and is also set to be the default for hand crafting conflicts in a merge.

The last two pieces of configuration are to provide a suitable name and email as it is added to every commit you make. If you intend to use github there are advantages to having then the same and using your github username rather than your full name

Well that was not difficult and you can now see all my applets in ~/cinnamon-applets but do not try editing them yet as there is another stage, namely creating a branch for your tests, to avoid changing the original.

gitk - a graphic viewer for Git. Now have a look at what you have with gitk which is a graphic viewer for Git. gitk is run in a terminal from within the folder containing the Git repository and helps enormously in helping one see what one is doing.

cd ~/cinnamon-applets
git gitk &

This changes to the cinnamon-applets folder in home from wherever you were then calls gitk. The & at the end makes gitk run a separate process so the terminal is available to continue.

The following is a typical output from when I was doing some work on Cinnamon. I have deliberately chosen to show something a little more complex than you will see with my applets to show its power - you should not worry about the details! It was actually after a second attempt at merging in a pull request from somebody elses repository having created a new branch for my testing (development) and a extra branch for a particular test when I was investigating a menu problem with Cinnamon (menu-issue).

You will note that gitk provides a lot of information:

At the top left panel is a list of all the commits with their commit messages and the two panels to the right identify the author and the date. The commits are linked to their parents and children. The position of master, the remote and branches are shown with the active branch in bold. The tags are also shown.

Below is full information on the highlighted commit at the top and the changes in every file is normally shown below although it can easily be switched to show a tree on the right with view to the left where every file can be examined.

I am not going into gitk more here as there are two excellent pages of explanation already

Finding a list of tags. Now lets go back to NUMA version 2.4.2 which was the one used for 18 months with Cinnamon versions up to 2.8. First we look and see exactly what the tag is before by git tag we call checkout:

pcurtis@defiant:~/cinnamon-applets$ git tag
BAMS-1.1.5
Bumblebee-0.9.5
Bumblebee-0.9.7
Bumblebee-0.9.8
Bumblebee-3.0.0
NUMA-2.3.17
NUMA-2.3.7
NUMA-2.4.2
NUMA-2.6.0
NUMA-3.0.0
Stopwatch-1.2.2
Stopwatch-1.2.3
Stopwatch-3.0.0
nvidiaprime-3.0.2
[1]+ Done
pcurtis@defiant:~/cinnamon-applets$

Using checkout to load an earlier snapshot into the working folder. We can now checkout NUMA-2.4.2

pcurtis@defiant:~/cinnamon-applets$ git checkout NUMA-2.4.2
Note: checking out 'NUMA-2.4.2'.

You are in 'detached HEAD' state. You can look around, make experimental
changes and commit them, and you can discard any commits you make in this
state without impacting any branches by performing another checkout.

If you want to create a new branch to retain commits you create, you may
do so (now or later) by using -b with the checkout command again. Example:

git checkout -b <new-branch-name>

HEAD is now at bc5b1e3... 2.4.2 Changes to automatically select first active interface at start-up Uploaded 6 May 2014
pcurtis@defiant:~/cinnamon-applets$

And you can now see the change with gitk or look at the files in an editor. It is sensible to use gitk to follow even the simple changes here to get a feeling of what is going on and to be able to see the features of each individual commit (remember to refresh gitk from the file menu). I have found it very useful, arguably essential for me, to understand Git although it could be considered an overkill at this point

Using checkout to return to the current version. Time to go back to the current version which is master using checkout

pcurtis@defiant:~/cinnamon-applets$ git checkout master
Previous HEAD position was bc5b1e3... 2.4.2 Changes to automatically select first active interface at start-up Uploaded 6 May 2014
Switched to branch 'master'
Your branch is up-to-date with 'origin/master'.
pcurtis@defiant:~/cinnamon-applets$

Have a look at the files and use gitk and you will find everything is as it was when you cloned it - easy wasn't it?

Exploring setting up branches and merging conflicts: The following is left here until I can get something better to show setting up two branches, editing a file called git_applet_intro.txt in both untill they are in conflict and merging then cleaning up.

pcurtis@matrix:~$ cd cinnamon-applets
pcurtis@matrix:~/cinnamon-applets$ git status
On branch master
Your branch is ahead of 'origin/master' by 3 commits.
(use "git push" to publish your local commits)

Changes to be committed:
(use "git reset HEAD <file>..." to unstage)

new file: git_applet_intro.txt

pcurtis@matrix:~/cinnamon-applets$ gitk &
[1] 6049

pcurtis@matrix:~/cinnamon-applets$ git commit -a -m "Add help file"
[master 85638c3] Add help file
1 file changed, 74 insertions(+)
create mode 100644 git_applet_intro.txt
pcurtis@matrix:~/cinnamon-applets$ git branch test1
pcurtis@matrix:~/cinnamon-applets$ git branch test2
pcurtis@matrix:~/cinnamon-applets$ git checkout test1
Switched to branch 'test1'
pcurtis@matrix:~/cinnamon-applets$ git checkout test2
M git_applet_intro.txt
Switched to branch 'test2'
// Edits to git_applet_intro.txt
pcurtis@matrix:~/cinnamon-applets$ git commit -a -m "Changes to test2 branch"
[test2 771aa7b] Changes to test2 branch
1 file changed, 4 insertions(+), 1 deletion(-)
pcurtis@matrix:~/cinnamon-applets$ git checkout test1
Switched to branch 'test1'
// Conflicting Edits to git_applet_intro.txt
pcurtis@matrix:~/cinnamon-applets$ git commit -a -m "Changes to test1 branch"
[test1 20e307c] Changes to test1 branch
1 file changed, 1 insertion(+), 1 deletion(-)
pcurtis@matrix:~/cinnamon-applets$ git checkout test2
Switched to branch 'test2'
pcurtis@matrix:~/cinnamon-applets$ git merge test1
Auto-merging git_applet_intro.txt
CONFLICT (content): Merge conflict in git_applet_intro.txt
Automatic merge failed; fix conflicts and then commit the result.
pcurtis@matrix:~/cinnamon-applets$ git status
On branch test2
You have unmerged paths.
(fix conflicts and run "git commit")

Unmerged paths:
(use "git add <file>..." to mark resolution)

both modified: git_applet_intro.txt

no changes added to commit (use "git add" and/or "git commit -a")
pcurtis@matrix:~/cinnamon-applets$ git mergetool

Merging:
git_applet_intro.txt

Normal merge conflict for 'git_applet_intro.txt':
{local}: modified file
{remote}: modified file
Hit return to start merge resolution tool (meld):
// Resolved conflicts in meld which was opened at theis points. Changes saved before closing it now need the commit to complete
pcurtis@matrix:~/cinnamon-applets$ git commit -a
[test2 a6960b1] Merge branch 'test1' into test2
pcurtis@matrix:~/cinnamon-applets$ git checkout master
Switched to branch 'master'
Your branch is ahead of 'origin/master' by 4 commits.
(use "git push" to publish your local commits)
pcurtis@matrix:~/cinnamon-applets$ git merge test2
Updating 85638c3..a6960b1
Fast-forward
git_applet_intro.txt | 5 ++++-
1 file changed, 4 insertions(+), 1 deletion(-)
pcurtis@matrix:~/cinnamon-applets$ git branch --merged
* master
test1
test2
pcurtis@matrix:~/cinnamon-applets$ git branch --no-merged
pcurtis@matrix:~/cinnamon-applets$ git branch -d test1
Deleted branch test1 (was 20e307c).
pcurtis@matrix:~/cinnamon-applets$ git status
On branch master
Your branch is ahead of 'origin/master' by 7 commits.
(use "git push" to publish your local commits)

Untracked files:
(use "git add <file>..." to include in what will be committed)

git_applet_intro.txt.orig

nothing added to commit but untracked files present (use "git add" to track)
// Deleted git_applet_intro.txt.orig

pcurtis@matrix:~/cinnamon-applets$ git status
On branch master
Your branch is ahead of 'origin/master' by 7 commits.
(use "git push" to publish your local commits)

nothing to commit, working directory clean
pcurtis@matrix:~/cinnamon-applets$ git branch -d test2
Deleted branch test2 (was a6960b1).
pcurtis@matrix:~/cinnamon-applets$
pcurtis@matrix:~/cinnamon-applets$ git status
On branch master
Your branch is ahead of 'origin/master' by 7 commits.
(use "git push" to publish your local commits)

nothing to commit, working directory clean
pcurtis@matrix:~/cinnamon-applets$

Merging conflicting files: Looking at the above it is obvious that another very important feature which Git has to have is the ability to merge changes from different places such as the remote files on the server or your own branches. This can be done from the command line and it automatic identifies the common root and does a three way merge into the merged file provided there are no conflicts (ie the same lines changed in both files). If there are conflicts they are marked in the resulting file ready for you to edit.

Use meld for resolving conflicts during merging - highly recommended: Git also offers the ability to specify and use a graphic merge tool, one of the best being meld. Once the conflict has been identified you just run meld as below (for fictitious conflict in README). I have handled all my conflicts with meld and never done a manual edit.

$ git mergetool -t meld
Merging the files: README

Normal merge conflict for 'README':
{local}: modified
{remote}: modified

Hit return to start merge resolution tool (meld):

When meld opens meld, three areas are shown, and left to right are:

  1. The local branch (usually the master branch)
  2. The current version that you are merging into
  3. The other (or “remote”) version (from the a test branch in example above)

To choose the text from the master branch, click on the black arrow in the left-most window at the top right.

So far we have covered how to install and set up Git, clone my repository, examine it with the gitk graphic visualiser, create a branch to make your own modifications and commit them, keep it up to date with my repository with pull and merge in any conflicts btween what I have done and your modification.

Sharing your changes with others.

Let us now look at what to do if you want to share some of your enhancements with others. There are several ways to collaborate but one of the oldest and still one of the easiestif not best is via patches. Small files which contain all the differences which can be applied to another unchanged file or one with, at a minimum, no conflicting changes. This is a basic mechanism used whether you are using Git or not but Git has a few enhancements and makes it all very easy.

First there is git diff which is very like an ordinary linux diff and takes SHA1 IDs or tags or positions relative to the HEAD for the two parameters. There is one caution and that is that Gitis often set up to provide a coloured output to a terminal and it is best to use the --no-color option. If one parameter is missing it assumes the diff id relative to HEAD and both gives the diff from the staging area.

git diff --no-color HEAD~2 > LastTwoCommits.patch // Makes a patch file for the last two commits and saves in LastTwoCommits.patch in current folder

The patchcan be applied easily by

git apply < /path/to/file LastTwoCommits.patch

There is still a lot of planning needed for successful collaboration to work. You should, at a minimum:

If the patchis complex and has multiple commits a better way to create itmay be use git format-patch to generate mbox-formatted files. This was originally intended as a way to email to a list by turning each commit into an email message with the first line of the commit message as the subject and the rest of the message plus the patch that
the commit introduces as the body. The nice thing about this is that applying a patch from an email generated with format-patch preserves all the commit information properly. It generates a set of patch files, one for each commit which look rather like the information available in the lower panel of gitk. The name is created from the start of the commit message and when called it prints out the names of the patch files it creates.

pcurtis@defiant:~/cinnamon-applets$ git format-patch nvidiaprime-3.1.4
0001-nVP-Added-Optional-Additions-to-Context-Menu.patch
0002-nVP-NUMA-BAMS-BB-Corrected-icons.png-in-Applet-folde.patch
0003-BAMS-Updated-Changelog.patch
0004-Bumblebee-gputempscript.sh-corrected-back-to-origina.patch
0005-Ignore-.patch-files.patch
pcurtis@defiant:~/cinnamon-applets$

These days it is easier to send the files as attachments as many of email packags do not preserve formating etc information correctly.I have listed the last very simple change here so you can see how full the information is:

cat 0005-Ignore-.patch-files.patch
From 38e74fa5c9e019715b55a859fd48046c50b2305e Mon Sep 17 00:00:00 2001
From: pdcurtis <gitmail@pcurtis.com>
Date: Tue, 9 Aug 2016 04:11:38 +0100
Subject: [PATCH 5/5] Ignore .patch files

---
.gitignore | 1 +
1 file changed, 1 insertion(+)

diff --git a/.gitignore b/.gitignore
index b1e5d66..9015710 100644
--- a/.gitignore
+++ b/.gitignore
@@ -1,4 +1,5 @@
*~
netusagemonitor@pdcurtis/vnstatImage.png
*.zip
+*.patch


--
2.7.4

pcurtis@defiant:~/cinnamon-applets$

This patch file can be applied by:

git am 0005-Ignore-.patch-files.patch

This should automatically created the new commit. The author information is taken from the email’s From and Date headers, and the message of the commit is taken from the Subject and body (before the patch) of the email.

Again when applying a patch it should be to a branch starting at the origin of the incoming patch series. I fneed be the current master can be merged in for testing.

NOTE: I have not tried out git format-patch and git am to check the above out whilst I have used the usual git diff and git apply

Creating your own Git Repository with possible use of GitHub as a central Repository.

We have contrated so far on using, modifying and contributing to an existing repository. However you may wish to take your own applet or other project and create a Git repository to bring it under version control and eventually look at making it available to others as I have done. I will start from scratch but you may have done most of the preliminary setting up already if you have cloned an existing repository.

First we need to install and configure Git. If you thing you have done it before then git config --list will show you what you have (best done outside of a Git repository (folder) so you only see the global settings.

pcurtis@defiant:~$ git config --list
user.name=pdcurtis
user.email=xxxxx@pcurtis.com
merge.tool=meld
core.editor=leafpad
color.ui=auto
diff.tool=meld
pcurtis@defiant:~$

If you have nothing or are only partially set up then do the appropriate parts of:

sudo apt-get install git gitk leafpad meld

git config --global user.name "your_username_on_github_is_best"
git config --global user.email "yourname@your_git_email_is_best"

git config --global core.editor leafpad
git config --global diff.tool meld
git config --global merge.tool meld
git config --global color.ui auto

You have at this point two choices depending on what you eventually expect to do.

  1. If you are serious about making your work available you may want to set up a GitHub account from the start, create your repository within GitHub, clone it as an empty repository, set up .gitignore, add all your current files and make the first commit which you then push back to the repository on GitHub. This is what I did as I already had a GitHub account from working on Cinnamon and Git was already set up for me.
  2. The second is to add Git to an existing folder turning it into a repository by git init, setting up .gitignore and then making the first commit. It is easiest if the folder is in home.

.gitignore In both cases you need to go through the Git setup as above and you also need a specific .gitignore file. This file contains a list of files/filetypes etc that git should ignore such as temporary files, patch files, zip files - anything you do not want in every commit as they are transient. Mine is very simple for the applet development but if you are working on code you will want to ignore all sorts of log files, intermediate files and compilation results:

*~
*.zip
*.patch

Linking an existing Git repository: If you have an existing Git repository you can link it to GitHub at a latter stage. First get a GitHub account and create a new repository with the same name as the Git repository (folder) on your machine which should be in home. You then create a remote called origin (by convention) and push your existing work to it. I would always back up the folder first!

cd new-repo
git remote add origin https://github.com/pdcurtis/new-repo
git push -u origin master

The git push -u origin master makes sure that the first time you have all the tracking set up and is recommended on GitHub

The following goes one further and shows by trial after setting up a new repository on GitHub called git-documents. The are a couple of calls to git status so you can see the effects better.

pcurtis@defiant:~$ cd git-documents
pcurtis@defiant:~/git-documents$ ls -A
.gitignore README.md status.txt
pcurtis@defiant:~/git-documents$ git init
Initialised empty Git repository in /home/pcurtis/git-documents/.git/
pcurtis@defiant:~/git-documents$ git status
On branch master

Initial commit

Untracked files:
(use "git add <file>..." to include in what will be committed)

.gitignore
README.md
status.txt

nothing added to commit but untracked files present (use "git add" to track)
pcurtis@defiant:~/git-documents$ git add .
pcurtis@defiant:~/git-documents$ git status
On branch master

Initial commit

Changes to be committed:
(use "git rm --cached <file>..." to unstage)

new file: .gitignore
new file: README.md
new file: status.txt

pcurtis@defiant:~/git-documents$ git commit -a -m "first commit"
[master (root-commit) c7a06db] first commit
3 files changed, 36 insertions(+)
create mode 100644 .gitignore
create mode 100644 README.md
create mode 100644 status.txt
pcurtis@defiant:~/git-documents$ gitk &
[1] 14048
pcurtis@defiant:~/git-documents$ git remote add origin https://github.com/pdcurtis/git-documents.git
pcurtis@defiant:~/git-documents$ git push -u origin master
Username for 'https://github.com': pdcurtis
Password for 'https://pdcurtis@github.com':
Counting objects: 5, done.
Delta compression using up to 8 threads.
Compressing objects: 100% (4/4), done.
Writing objects: 100% (5/5), 1.36 KiB | 0 bytes/s, done.
Total 5 (delta 0), reused 0 (delta 0)
To https://github.com/pdcurtis/git-documents.git
* [new branch] master -> master
Branch master set up to track remote branch master from origin.
pcurtis@defiant:~/git-documents$

 

The following is left over from the section in kernel.htm

Back to GIT

The following is a log of what I did to try and backfill in GIT after I had made one or two significant changes to my three applets and wanted to be able to track them properly. It had been many months since I had used GIT so I wanted to avoid damaging what I already had whilst it came back to me. So I cloned the existing repository onto another machine so I could 'practice' until I was happy to push it back to the server. I then progressively copied the changed files from the first update across and did a commit, then the next set of updates and commited and so on. I had a few mistakes and the log shows the brute force way I used to backtrack to start again from before the mistake. The most common mistake is to have the wrong message with a commit and I found the git commit --amend did not actually make any changes hence the forced reset. See below for why --amend did not work.

In the end it all worked out well so I pushed to the server from the second machine and then cloned back to my normal machine after backing up the original. One should have been able to just fetch but I wanted an easy life! I the added a few tags to the Uploaded versions which were available to users.

It starts with installing git and the git viewer gitk which helps enormously to visualise where one has got to.

I have added a few comments on what each step was.

pcurtis@matrix:~$ sudo apt-get install git gitk

// See if we have any previous git configuration otherwise we would have needed to set user.name and user.email

pcurtis@matrix:~$ git config --list
user.name=pdcurtis
user.email=gitmail@pcurtis.com

// OK so we can clone

pcurtis@matrix:~$ git clone https://github.com/pdcurtis/cinnamon-applets
Cloning into 'cinnamon-applets'...
remote: Counting objects: 190, done.
remote: Compressing objects: 100% (88/88), done.
remote: Total 190 (delta 105), reused 181 (delta 97)
Receiving objects: 100% (190/190), 81.08 KiB | 0 bytes/s, done.
Resolving deltas: 100% (105/105), done.
Checking connectivity... done.

// Change into working folder

pcurtis@matrix:~$ cd cinnamon-applets

// run gitk to see whats going on (use & so spawned process and we still have terminal available)

pcurtis@matrix:~/cinnamon-applets$ gitk &

pcurtis@matrix:~/cinnamon-applets$ git status

// Replace the changed files for Bumblebee from latest applet

curtis@matrix:~/cinnamon-applets$ git commit -a -m "Stopwatch v20_0.9.8 Check we have a valid temperature returned from nvidia-settings"

// Have made a mistake in message so go back one step to previous state - BRUTE FORCE -there are better ways

pcurtis@matrix:~/cinnamon-applets$ git reset --hard HEAD~1

// Needed to copy back the changed files again as the reset removed the changes

pcurtis@matrix:~/cinnamon-applets$ git commit -a -m "Bumblebee 0.9.8 Check valid temperature returned from nvidia-settings - UPLOADED 11 November 2014"
[master 5ecbc4b] Bumblebee 0.9.8 Check valid temperature returned from nvidia-settings - UPLOADED 11 November 2014
2 files changed, 19 insertions(+), 17 deletions(-)

====All the changed files in NUMA 2.5.0 copied over existing ones and then commit ======================

pcurtis@matrix:~/cinnamon-applets$ git commit -a -m "NUMA - 2.5.0 Multiple Instance capability but set to 1"
[master ba64e78] NUMA - 2.5.0 Multiple Instance capability but set to 1
4 files changed, 19 insertions(+), 6 deletions(-)

====All the changed files in 2.6.0 copied over existing ones======================
// Now commit the changes

pcurtis@matrix:~/cinnamon-applets$ git commit -a -m "NUMA 2.6.0 Android Bluetooth via bnep0 added. FormatSentReceived improved UPLOAD CANDIDATE"
[master d28539b] NUMA 2.6.0 Android Bluetooth via bnep0 added. FormatSentReceived improved UPLOAD CANDIDATE
2 files changed, 40 insertions(+), 13 deletions(-)

// push all of the changes which have been committed up to the server

pcurtis@matrix:~/cinnamon-applets$ git push origin
warning: push.default is unset; its implicit value is changing in
Git 2.0 from 'matching' to 'simple'. To squelch this message
and maintain the current behavior after the default changes, use:

git config --global push.default matching

To squelch this message and adopt the new behaviour now, use:

git config --global push.default simple

When push.default is set to 'matching', git will push local branches
to the remote branches that already exist with the same name.

In Git 2.0, Git will default to the more conservative 'simple'
behavior, which only pushes the current branch to the corresponding
remote branch that 'git pull' uses to update the current branch.

See 'git help config' and search for 'push.default' for further information.
(the 'simple' mode was introduced in Git 1.7.11. Use the similar mode
'current' instead of 'simple' if you sometimes use older versions of Git)

Username for 'https://github.com': pdcurtis
Password for 'https://pdcurtis@github.com':
Counting objects: 24, done.
Delta compression using up to 2 threads.
Compressing objects: 100% (17/17), done.
Writing objects: 100% (17/17), 17.84 KiB | 0 bytes/s, done.
Total 17 (delta 11), reused 0 (delta 0)
To https://github.com/pdcurtis/cinnamon-applets
3983078..d28539b master -> master

========== Now add Tags =====================
// What tags have we got already?
pcurtis@defiant:~/cinnamon-applets$ git tag
Bumblebee-0.9.5
Bumblebee-0.9.7
NUMA-2.3.17
NUMA-2.3.7
NUMA-2.4.2
Stopwatch-1.2.2

// Find out id of recent commits Ctrl-Z to exit

pcurtis@defiant:~/cinnamon-applets$ git log --pretty=oneline
d28539b88745be0b623a7fb1d5b02bffe7bff84a NUMA 2.6.0 Android Bluetooth via bnep0 added. FormatSentReceived improved UPLOA
ba64e7898def3c4ee4b90c8e534d2aedd56a3e70 NUMA - 2.5.0 Multiple Instance capability but set to 1
5ecbc4bfba802b1fcf633422fc4a1d38fd08b3d4 Bumblebee 0.9.8 Check valid temperature returned from nvidia-settings - UPLOADE
39830788231d9fa3eaee7ecd51d6e6c78224baeb Changes in padding and in width calculation for compact display and modified fu
f9cb13fa486a8258848c2f410d1d871eb5bf7fe4 Changes to arrangements for padding to get round removal of leading spaces, red
9298ec601eb6ddcfc4c65b55ea2d210e999ff5f1 NUMA 2.4.4 Removal of some commented out text and change log updated
27f567b7ec313ef8d5c7a85399182f67dc060127 NUMA 2.4.3 Addition of offsets to cumulative monitoring, reduced size applet wh
716cffa0bc97a59e379dde8f864754aca35d5fb1 Stopwatch - edit comments
e8c0660ee63ee6c8ac812b3bd4f7bf21131692c9 Update gitignore
fc9ee1d0414a8969f9c6738b2c313c182ed43fab Stopwatch version 1.2.3
cf84b6ae7e20712c78bd4d1b84f9806ffba0612a Bumblebee version 0.9.7 Uploaded 2 April 2014
8fc32db89cb3a8b22be25c7a228816d3286ec149 Bumblebee version 0.9.5 Uploaded January 7 2014
4d4344f56de8008006f3063d673b099f7f569177 Stowatch version 1.2.2 Uploaded 2 April 2014

// Add first simple tag

pcurtis@defiant:~/cinnamon-applets$ git tag Bumblebee-0.9.8 5ecbc4bfba802b1fcf633422fc4a1d38fd08b3d

// and so on

// Tags have to be pushed separately. --tags adds all not yet present

pcurtis@defiant:~/cinnamon-applets$ git push origin --tags
Username for 'https://github.com': pdcurtis
Password for 'https://pdcurtis@github.com':
Total 0 (delta 0), reused 0 (delta 0)
To https://github.com/pdcurtis/cinnamon-applets
* [new tag] Bumblebee-0.9.8 -> Bumblebee-0.9.8

// Check again

pcurtis@defiant:~/cinnamon-applets$ git tag
Bumblebee-0.9.5
Bumblebee-0.9.7
Bumblebee-0.9.8
NUMA-2.3.17
NUMA-2.3.7
NUMA-2.4.2
Stopwatch-1.2.2
Stopwatch-1.2.3
pcurtis@defiant:~/cinnamon-applets$

// Backfill done - now we can carry on in proper manner!

==================== Useful Commands ================

git difftool - shows the differences from current to staged
pcurtis@defiant:~/cinnamon-applets$ git difftool --tool-help
'git difftool --tool=<tool>' may be set to one of the following:
araxis
meld
The following tools are valid, but not currently available:
bc
bc3
codecompare
deltawalker
diffmerge
diffuse
ecmerge
emerge
gvimdiff
gvimdiff2
gvimdiff3
kdiff3
kompare
opendiff
p4merge
tkdiff
vimdiff
vimdiff2
vimdiff3
winmerge
xxdiff
Some of the tools listed above only work in a windowed
environment. If run in a terminal-only session, they will fail.

pcurtis@defiant:~/cinnamon-applets$ git config --global diff.tool meld

-------------------------------------------------------------

pcurtis@defiant:~/cinnamon-applets$ sudo apt-get install leafpad
....

pcurtis@defiant:~/cinnamon-applets$ git config --global core.editor leafpad
pcurtis@defiant:~/cinnamon-applets$ git config --list
user.name=pdcurtis
user.email=gitmail@pcurtis.com
merge.tool=meld
core.editor=leafpad
color.ui=auto
diff.tool=meld
core.repositoryformatversion=0
core.filemode=true
core.bare=false
core.logallrefupdates=true
remote.origin.url=https://github.com/pdcurtis/cinnamon-applets
remote.origin.fetch=+refs/heads/*:refs/remotes/origin/*
branch.master.remote=origin
branch.master.merge=refs/heads/master

pcurtis@defiant:~/cinnamon-applets$

Note The Git section has moved to An Introduction to Git part 1 - Applicaions in Cinnamon Applet Development

Using an alternative/external diff and merge tools in Git

There is a very good tool for looking at differences between files which is called meld. Once this is installed one can tell git to use by git difftool --tool=meld when looking at difference after which one can use git difftool instead of git diff or configure in advance so you can just use git difftool by

git config --global diff.tool meld

Likewise for merging one can tell git to use by git mergtool --tool=meld when looking at difference after which one can use git mergetool instead of git merge or configure in advance and just use git difftool by

git config --global merge.tool meld

Note: These were included in my standard configuration at the start of this page in the kernel patching section.

Bug when using the same editor within and outside Git

A number of Git commands automatically open a copy of your default editor (gedit on Ubuntu and Mint and load text into it for editing including git commit and git commit --amend if a copy of the gedit or xed editor is already open. Unfortunately although it appears you have saved the temporary file and closed it the text is not used. Most of the time you will be editing your text using gedit so this is a bad situation as you do not want to have to remember to close gedit every time you commit. I want to use a GUI editor for the commit text so I came up with the idea of also loading leafpad, the lightweight editor used by Lubuntu and setting that to be the default for Git and that works perfectly and leafpad is a perfectly adequate editor for the simple jobs within Git. So we need to install leafpad and then configure git by.

sudo apt-get install leafpad
git config --global core.editor leafpad

Note: The above was included in my standard configuration at the start of this page in the kernel patching section.

 

16th August 2016

Terminal Access to Linux from Android

I unfortunately managed to have an Applet which broke Cinnamon preventing any display on my laptop whilst I was away from home. I could hover see that I still had a wifi connection automatically started to my phone when put into tethering mode. If Tethering is enabled Android Settings displays the MAC Address and IP address assigned to every connected device so it looked possible to make ann SSH connection if I could find a suitable Android Client. An internet search found JuiceSSH which I downloaded and simply enabled me to login from the terminal. I was surprised to find that I could use the tethered connection from the phone - I had expected to need another phone to make a connection which I could use.

Once logged in I could run Git and checkout an earlier version of my applet and login again using Cinnamon. The only problem was that the font size was very small for a 1280 wide 4.5 inch screen but I found afterwards that the volume keys can be used to adjust it. Otherwise it was just the same as access from another computer using SSH.

The applet problem turned out to be a faulty edit of a settings file where a 200000 had lost the initial 2 giving an upper limit of zero. I would have expected that sort of error to be trapped in the Cinnamon Settings software - I will be be careful in future.

If the terminal access had not been available I would have needed to have another user with administration rights (sudo) to log into the machine or to use a LiveUSB to get to the applet neither of which were available to me on the boat. I have now set up another user just in case and the next section covers how to login from another user.

20th August 2016

Terminal access from a different user

Having been locked out of the cinnamon desktop manager a couple of times by malformed applets under development I have set up a spare user on my system with admin privileges so I can login to the problem user. There are several ways to login to a different user but the simplest are:

su - username
sudo -iu username
ssh username@localhost
ssh username@lanaddress

The ones using ssh need ssh to be installed and will work without sudo priviledges, ie for a non admin user. The last one allows access over a local network (ie wifi) . There is background at http://unix.stackexchange.com/questions/3568/how-to-switch-between-users-on-one-terminal

This enables one to run Git to checkout an earlier version of the applet to recover.

Visual Git

http://www.wei-wang.com/ExplainGitWithD3/#

Git Tips and Tricks

This is a collection of some of the various slightly non standard activities and experiments I have tried out

How to restore 'origin' to exactly the same as 'upstream' for a fork.

I came back to look at my Cinnamon 'Fork' after many years. I did some work at the stage of Cinnamon 1.6 to see the potential speed up on my slow Wind Netbook of various proposed Menu enhancement patches. I had a number of test branches and the whole situation was somewhat confused when I looked back at it 3 years later. The usual Fetch and Merge gave many conflicts and when I tried to push it. it seemed there had been circa 1200 commits on the 'upstream' repository. I decided the only way was a fresh start and get back to what I would have with a fresh Fork on GitHub and Clone. So the plan was to get back before I had made any changes and started any branches, delete all my braches which were for very out of date tests, fetch and merge all the changes from the cinnamon repository (upstream) and force a push of the result up to my fork.

# First Change Directory # and use gitk to find the first commit
cd ~/Cinnamon
gitk &
# Then copy the SHA1 ID to do a hard reset to it.
git --hard reset 4f16f79b3684dbba9b76168101965c53419ca38f
# We now just have just the first commit in our local repository
git branch
# Find out what local branches we have
git branch -D menu_test1
git branch -D menu_test2
# We Force Delete Branches as they had uncommited or unmerged changes
git fetch upstream master
git merge upstream/master
# We are now up to date with upstream (The Cinnamon repository)
# Next force a push to origin which get your fork to the same as your local repository
git push --force upstream/master
# Now we have an exact local and remote copies of the Cinnamon Repository (upstram master)

Who said it was near impossible to destroy information with Git! One should only think about these forced changes in extreme circumstances although to be fair we have only lost two test branches which we would have got rid of in any case.

How to look at remote repositories [before you merge or pull a request from them]

The common way to keep up to date is to do a pull or a fetch and merge but what if you want to have a look at what is on a remote repository before you do that. Or you might want to look a somebody elses fork before pulling in a patch.

There is a little exploited feature of Git and that is FETCH_HEAD. When you do a fetch you obtain a local copy of the remote after which you do not need to be online as you have to be for a pull. What fetch does varies slightly with Git versions and what I will be describing is true for Version 1.8.4 and latter but in all cases you have FETCH_HEAD pointing to the HEAD of the last fetch you have done.

So you can do a diff between your current HEAD and FETCH_HEAD to find out all the changes you will be merging in with a typical pull (or git merge upstream/master). Perhaps of more interest you can do the same for a single file to see if there will be a merge conflicts with it in a branch you are working on.

git checkout master
git remote show # Have a look at list of existing remotes
git fetch upstream master # fetches the whole repository after Git Version 1.8.4
git checkout branch_I_working_in
git diff HEAD FETCH_HEAD
# Or have a look using meld
git difftool HEAD FETCH_HEAD
# Just look at differences in all the commits in a particular file
git difftool HEAD FETCH_HEAD file_I_am_working_on

There are other ways of doing this but this does not involve setting up branches etc if you are thinking about a pull or just to have alook round a repository.

I'm working on the wrong branch

Have you ever realised you have done a big edit and realised you are still on the master rather than your topic branch. Rather than commit, checkout your topic_branch, merge and then reset your master to remove the changes from the working directory you can use git stash which takes your working directory, saves it all on a stack and does a reset so there is nothing left to commit so you can switch. Once there you can pop everything back into your working directory. That is just one of many applications for stash

# Lots of edits
git status
# Realisation you are on the master branch
git stash
git checkout topic_branch
git stash pop
# Carry on with your edits


Thoughts on remotes - conventions

One has various conventional names for remotes which are often set up automatically when one forks and clones. There is no mystery they are just aliases for the address of the remote repository. You can see this by

pcurtis@defiant:~/Cinnamon$ git remote --verbose
origin https://github.com/pdcurtis/Cinnamon.git (fetch)
origin https://github.com/pdcurtis/Cinnamon.git (push)
upstream https://github.com/linuxmint/Cinnamon.git (fetch)
upstream https://github.com/linuxmint/Cinnamon.git (push)
pcurtis@defiant:~/Cinnamon$

It is convention to have origin as your own remote repository and upstream if your repository is fork of another repository which you need to keep up to date with but they could be called anything else.

Individual branches may be tracking or non tracking branches. By default one does not pull or push remote branches other than master unless set up specifically to track or fully specified in the pull, fetch or push commands. The output above shows which branches are the defaults for push, fetch and pull.

So if we now look at our branches with various options

pcurtis@defiant:~/Cinnamon$ git branch
* master
pcurtis@defiant:~/Cinnamon$ git branch -v
* master 6b0ea0c Merge pull request #5608 from JosephMcc/window-list1
pcurtis@defiant:~/Cinnamon$ git branch -vv
* master 6b0ea0c [origin/master] Merge pull request #5608 from JosephMcc/window-list1
pcurtis@defiant:~/Cinnamon$

We can with the -vv option see what we are tracking as well where we are.

Setting up a remote branch with tracking

  1. Make sure everything has been commited on the branch you want to make a branch on your remote repository
  2. Set up the remote so it is tracked and you can push and pull to it.
  3. Have a look at what we have set up

pcurtis@defiant:~/git-documents$ git status
On branch snippets-dev
Changes not staged for commit:
(use "git add <file>..." to update what will be committed)
(use "git checkout -- <file>..." to discard changes in working directory)

     modified: snippets.txt

no changes added to commit (use "git add" and/or "git commit -a")
pcurtis@defiant:~/git-documents$ git commit -a -m "More about setting up on extra machine including tracking a remote branch"                                                         #1
[snippets-dev a0a2031] More about setting up on extra machine including tracking a remote branch
1 file changed, 55 insertions(+), 32 deletions(-)
pcurtis@defiant:~/git-documents$ git push --set-upstream origin snippets-dev      #2
Username for 'https://github.com': pdcurtis
Password for 'https://pdcurtis@github.com':
Counting objects: 16, done.
Delta compression using up to 8 threads.
Compressing objects: 100% (16/16), done.
Writing objects: 100% (16/16), 5.57 KiB | 0 bytes/s, done.
Total 16 (delta 7), reused 0 (delta 0)
remote: Resolving deltas: 100% (7/7), done.
To https://github.com/pdcurtis/git-documents.git
* [new branch] snippets-dev -> snippets-dev
Branch snippets-dev set up to track remote branch snippets-dev from origin.
pcurtis@defiant:~/git-documents$ git branch --all -vv                             #3
  cheatsheet-dev 9f7df6f Added new file cheatsheet.txt
  examples-dev aed55c0 Edited examples.txt
  master 448e980 [origin/master: ahead 4] More edits to cheatsheet.txt and snippets.txt
* snippets-dev a0a2031 [origin/snippets-dev] More about setting up on extra machine including tracking a remote branch
  remotes/origin/master c7a06db first commit
  remotes/origin/snippets-dev a0a2031 More about setting up on extra machine including tracking a   remote branch
pcurtis@defiant:~/git-documents$

Working on several machines - Cloning and set up remote tracking branches

In the last section we delt with how to have a remote tracking branch as well as your master branch on your remote repository (origin). One major use of ones remote repository is to allow one to work on several machines and keep in sync via ones remote repository and as an added advantage, the more repositories the better the backup.

So you have your Git repository on your desktop computer and a remote repository on GitHub and you want to work on your laptop whilst you are on holiday or on the way to work.

pcurtis@matrix:~$ git clone https://github.com/pdcurtis/git-documents          #2
Cloning into 'git-documents'...
remote: Counting objects: 27, done.
remote: Compressing objects: 100% (6/6), done.
remote: Total 27 (delta 2), reused 0 (delta 0), pack-reused 21
Unpacking objects: 100% (27/27), done.
Checking connectivity... done.
pcurtis@matrix:~$ cd git-documents
pcurtis@matrixt:~/git-documents$ git branch --all --verbose                    #3
* master 448e980 [ahead 4] More edits to cheatsheet.txt and snippets.txt
  remotes/origin/master c7a06db first commit
  remotes/origin/snippets-dev a0a2031 More about setting up on extra machine including tracking a remote branch
pcurtis@matrix:~/git-documents$ git checkout --track origin/snippets-dev      #4
  Branch snippets-dev set up to track remote branch snippets-dev from origin.
  Switched to a new branch 'snippets-dev'
pcurtis@matrix:~/git-documents$ git branch --all -vv                          #5
* master 448e980 [origin/master: ahead 4] More edits to cheatsheet.txt and snippets.txt
  snippets-dev a0a2031 [origin/snippets-dev] More about setting up on extra machine including tracking a remote branch
  remotes/origin/master c7a06db first commit
  remotes/origin/snippets-dev a0a2031 More about setting up on extra machine including tracking a   remote branch
pcurtis@matrix:~/git-documents$

Notes: If you add more branches you should note that the git branch --all --verbose uses local information so you need to do a fetch first if extra remote branches have been added since the clone.

Deleting remote branches

git push origin --delete cheatsheet-dev

Now do the following on other machine(s)

git fetch --all --prune # Propagate change onto other machine(s)
git branch --unset-upstream cheatsheet-dev # Remove tracking references on other machine(s)
git branch -d cheatsheet-dev # remove branch on other machine(s)

14th September 2016

How to deal with a Pull Request

I have just received a pull request through GitHub - this told me that it did not involve any merge conflicts and made the following suggestions on the way to handle by the pull request from ReliQ/cinnamon-applets master branch was as follows:

Step 1: From your project repository, check out a new branch and pull in the changes so one can test them.

git checkout -b ReliQ-master master
git pull https://github.com/ReliQ/cinnamon-applets.git master

Step 2: If one is satisfied one can then merge the changes and update ones own remote repository on GitHub.

git checkout master
git merge --no-ff ReliQ-master
git push origin master

The --no-ff means that a merge commit is made regardless of whether a fast forwards merge is posible so that an merge is visible andto give an audit trail.

GitHub provides a mechanism so a general discussion on the merge can be held with the proposer and anyone else who wants to get involved which is a very powerful way to collaborate before the merge is completed. The Pull Request I have received is is an example of the basic workflow that most GitHub projects use. Topic branches are created by someone who has forked a repository, Pull Requests are opened on them when the Topic is completed, a discussion ensues, possibly more work is done on the topic branch and eventually the request is either closed or merged.

In this case it was a simple change to the css style sheet file and I tired it in a new branch as suggested above but did not merge it as it was only suitable for a dark theme and gave white on light gray lettering on the void theme I use. It has however started me thinking about the mechanism I have for users to change the styling file where their changes are likely to be overwritten when the applet is updated.

18th September 2016

Renaming local and remote tags

I discovered I had two tags which were inconsistently named and found the following way to correct them in http://stackoverflow.com/questions/1028649/how-do-you-rename-a-git-tag which worked very well.

git tag new old
git tag -d old
git push origin :old
git push origin --tags

The colon in the push command removes the tag from the remote repository (origin is probably redundant as it is the default)

As the tags were simple tags have been renamed this should not cause major problems to anyone who has cloned my repository.

Dropbox Updates and System Tray Icon missing Context Menu

The following locates where the current dropbox version is located.

ps -ef | grep -i dropbox

The easy way to update is to stop dropbox by

dropbox stop

Delete that folder which is usually ~/dropbox-dist and also ~/.dropbox

Start dropbox with an option to go to the web site and (re)install dropbox

dropbox start -i

When it has completed you will be asked to login after which it will start to index all your files.

System Tray Icon missing Context Menu: There is a problem with the dropbox icon with some versions and it can be present but with no right click menu or status popup when you hover. This is a show-stopper at present as it is the only access to the Dropbox preferences and to a password change which Dropbox is bringin due to a hack. It seems to be trying to display it as an indicator when the Dropbox icon has a bug possibly due to Dropbox programming (unfinished or incorrect indicator code?). You can turn off support for indicator and force it to display as a status icon in the system tray if you have no other indicators in use.

You can try this solution by going to "System Settings"->"General", then turn OFF "support for indicators." Restart Cinnamon (right-click panel, select "Troubleshoot"->"Restart Cinnamon"). You'll likely have to restart Dropbox for the status icon (rather than indicator) to appear (open a terminal and type "dropbox stop" [Enter], then "dropbox start" [Enter] -- no quotation marks). See https://www.dropboxforum.com/hc/en-us/community/posts/204585553-Systray-menu-Cinnamon. There are reports that if indicators are already off you may have to toggle it.

There are other errors relating to starting up which I have not experienced - see https://github.com/linuxmint/Cinnamon/issues/4396

Before You Leave

I would be very pleased if visitors could spare a little time to give us some feedback - it is the only way we know who has visited the site, if it is useful and how we should develop it's content and the techniques used. I would be delighted if you could send comments or just let me know you have visited by sending a quick Message.

Link to W3C HTML5 Validator Copyright © Peter & Pauline Curtis
Content revised: 8th July, 2020