terminal – nick-web.co.uk

Just a quick one – I had to edit an old PHP script that I wrote a while back to create calendar events. At the start of this file is a large array containing a number of letters which correspond to specific durations. As usual – I stuck with my one of the well known XKCD comics and decided that to update this file it was easier to learn a few new tricks (teach a man to fish and all that). In summary, I had to add to the file a list of formatted letters like

"N", 
"N", 
"N", 
"L", 
"L"

Each letter was repeated 3 times on recurring lines, and there was a total of 189 letters which had to be wrapped in quotation marks and have a comma at the end.

To begin with, I created a text file and entered in each line one character that would eventually become 3 lines. I then wrapped this up in AWK using the following one liner;

awk '{ print "\"" $0 "\","; print "\"" $0 "\","; print "\"" $0 "\","}' input.txt > output.txt

I know that docent look pretty, however its a one liner and it works. To start with,

print $0

will print the current line, so by prepending and appending the characters I needed, properly escaped, around that

$0

and repeating that 3 times I turned

N
L
R

into

"N",
"N",
"N",
"L",
"L",
"L",
"R",
"R",
"R",

for a total of 189 lines. I then moved this file to where I run my PHP script, and started VIM opening the two files at once. A quick press of

to select Visual Mode where I select the whole file, followed by a

:b 1

to move to the PHP script (the b stands for buffer, and the 1 is because that was the first file opened. 2 would be the seconds etc.) in VIM and a quick paste and I was almost done – I just had to remove the final comma

from the last array entry. I’m not good enough to have a bit of AWK work the last bit out – and the time spent implementing it would not be worth it.

AWK one liners, and AWK itself has never really been something I’ve used. In the past its been presented to me as a way to solve problems, but the solutions have been handed to me. This is the first time I’ve actively went hunting for a solution without asking for help – and even though its simple, I’m quite happy I managed to figure it out – even if it did take about 20 minutes of searching…

As a final note I’m also quite happy I managed this completely in the command line. No mice were used in the editing of this script!

MAJOR UPDATE – USING PHP ONLY WITH NEW CODE! Please view this next post for more information!

So. Google Reader is closing down. I’m not going to get all high and mighty – It’s Google’s product. They do with it as they wish! There are several places that let you save your feeds from Google Reader, but I wanted to add all my Starred Items from Reader into Pocket. It turns out it wasn’t the easiest thing to do! After a few terminal commands and some PHP, this is what I came up with!

Firstly, head to Google Reader, and use the Data Takeout feature that Google provides to save your Reader Data only. The outputted ZIP file should contain a folder entitled “Reader”. Within that, there should be a file named “starred.json”.

*UPDATE* – I have saved these 2 files to GitHub! Go, Grab!
https://github.com/nickwebcouk/pocketimport

Now the fun begins! To make it easy (and quick) I used the Terminal on Mac and ran these commands within the above folder. I used two files (new.txt and newnew.txt) just to keep track of what was happening. There are easier ways of doing this! I ran the following commands from terminal:

grep -a1 "canonical" starred.json > new.txt
grep -v "^\--" new.txt > newnew.txt
grep -v "} ]," newnew.txt > new.txt
grep -v "\"canonical\" : \[ {" new.txt > newnew.txt
grep -v "\"updated" newnew.txt > new.txt
grep -v "} \]," new.txt > newnew.txt
cat newnew.txt | rev | cut -c 2- | rev > new.txt
cut -c 17- new.txt > newnew.txt
rm -r new.txt
mv newnew.txt url.txt

The “grep” command looks through text files for specific expressions. “cat” outputs a full file, “rev” reverses items, “cut” cuts text, “rm” removes files and “mv” moves files.

To make this easier, I created a Shell Script (Tested on OSX 10.8.2 only)

#!/bin/sh

clear
grep -a1 "canonical" starred.json > new.txt
grep -v "^\--" new.txt > newnew.txt
grep -v "} ]," newnew.txt > new.txt
grep -v "\"canonical\" : \[ {" new.txt > newnew.txt
grep -v "\"updated" newnew.txt > new.txt
grep -v "} \]," new.txt > newnew.txt
cat newnew.txt | rev | cut -c 2- | rev > new.txt
cut -c 17- new.txt > newnew.txt
rm -r new.txt
mv newnew.txt url.txt

You can run that file by saving it to the same location as the Google Reader folder, and running the following in terminal first:

chmod 755 script.sh
./script.sh

the first line tells the computer to allow script.sh to be executed, and the second line executes the script.

I then moved the url.txt file that had just been created to my www root folder (for me its under /users/~name/sites/), and created the following PHP/HTML code:

<!DOCTYPE html>
<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=UTF-8" />
<title>Instapaper: Export</title>
</head>
<body>
<h1>Unread</h1>
<ol>
<?php
    $file_handle = fopen("url.txt", "r");
    while (!feof($file_handle)) {
        $galbool = FALSE;
        $line = fgets($file_handle);
        $string = $line;
        $check = $string[strlen($string)-2];
        if ( $check == "/"){
            $string = rtrim($string);
            $string = rtrim($string, "/");
            $desc = $string;
        }
        $gallerycheck = str_replace("/gallery", "", $string, $count);
        if ($count == 1){
            $galbool = TRUE;
            $string = rtrim($string);
            $string = rtrim($string, "/gallery");
            $desc = $string;
        }
        $desc = strrchr($string, "/");
        $desc = str_replace("/", "", $desc);
        $desc = str_replace("-", " ", $desc);
        $desc = ucwords($desc);
        if ($galbool == TRUE){
        $formatted = '			<li><a href="' . $string . '">' . $desc . '[Gallery]</a></li>';
        } else {
        $formatted = '			<li><a href="' . $string . '">' . $desc . '</a></li>';
        }
        echo $formatted;
    }
    fclose($file_handle);
?>
</ol>
</body>
</html>

This provided me with a HTML page, which if saved as instapaper-export.html allowed me to head to getpocket.com and use the Instapaper import option.

584 starred articles and 2 seconds later, I received this wonderful little message!

*MAJOR UPDATE* – USING PHP ONLY WITH NEW CODE! Please view this next post for more information!

MAJOR UPDATE – USING PHP ONLY WITH NEW CODE! Please view this next post for more information!