Getting Tilestream Working on Ubuntu 15.10 (Wily Werewolf)

This configuration is suitable for a simple tilestream appliance that only serves tiles. It can be quickly configured and uses node.js from chris-lea repositories.

The basic concept:

• install latest Ubuntu 15.10 iso (default settings, I included openssh only), reboot
• install nodejs sym linking it as the node binary
• install npm and git
• clone the mapbox git repo
• edit the package.json file to allow the latest version of connect
• npm install
• run

The basic concept is to link apt-get to the chris-lea repo to install node, download the git repo of the tilestream app from mapbox and adjust the package.json dependency to get a more modern version of connect. Then it should fire up ok.

This list should get you up and running...

$ sudo apt-add-repository ppa:chris-lea/node.js

$ sudo apt-get update

$ sudo apt-get -y install nodejs git-core

$ sudo ln -s /usr/bin/nodejs /usr/bin/node # node binary is the nodejs binary! 

$ sudo apt-get -y install npm # installs lots of dependencies

$ git clone https://github.com/mapbox/tilestream.git && cd tilestream

$ vim package.json # edit the connect version to latest (replace the connect version with a *)

$ npm install

$ ./index.js --host 10.0.0.34

Bam!!

Check out my similar install instructions for Ubuntu 14.04

Check out the config instructions on mapboxes github site

Installing Tilestream on Ubuntu 14.04 (2015 update)

• Install a fresh Ubuntu 14.04 64bit, include only openssh in the setup wizard,
• reboot,
• log in to guest and check the ip address then login from mac host via ssh (so we can copy paste following commands)
• do this as a normal user not root!

1. Follow github instructions from mapbox:

sudo apt-add-repository ppa:chris-lea/node.js

sudo apt-get update

sudo apt-get install nodejs

sudo apt-get install git-core

sudo apt-get update

git clone https://github.com/mapbox/tilestream.git

cd tilestream/ && npm install

And the result is (drum roll)...

./index.js will complain about the stringify module - fail

so to fix this....

    2. Update the dependencies section of the package.json file and allow the latest connect version

    "dependencies": {

        "jsdom":"https://github.com/springmeyer/jsdom-nogyp/tarball/c501ee8b71d8",

        "xmlhttprequest": "1.5.0",

        "backbone":"0.3.3",

        "jquery":"1.6.3",

        "express":"2.5.11",

        "connect":"*",

        "bones": "1.3.29",

        "step": "0.0.5",

        "tilelive": "4.5.3",

        "mbtiles": "0.2.9",

        "sphericalmercator": "1.0.2",

        "underscore": "1.6.0",

        "wax": "6.4.2",

        "modestmaps": "3.3.5"

    },

    3. You have to remove the current node_modules directory, reinstall using the new package config and re launch ./index.js:

charlton@tileserver:~/tilestream$ rm -fR node_modules/ && npm install && ./index.js --host 10.0.0.34
npm WARN package.json tilestream@1.1.0 'repositories' (plural) Not supported. Please pick one as the 'repository' field
npm WARN engine hoek@2.16.3: wanted: {"node":">=0.10.40"} (current: {"node":"0.10.37","npm":"1.4.28"})
npm WARN engine boom@2.10.1: wanted: {"node":">=0.10.40"} (current: {"node":"0.10.37","npm":"1.4.28"})
npm WARN engine cryptiles@2.0.5: wanted: {"node":">=0.10.40"} (current: {"node":"0.10.37","npm":"1.4.28"})
npm WARN deprecated set-immediate@0.1.1: Use `setimmediate` instead
> sqlite3@2.2.7 install /home/charlton/tilestream/node_modules/mbtiles/node_modules/sqlite3
> node-pre-gyp install --fallback-to-build
[sqlite3] Success: "/home/charlton/tilestream/node_modules/mbtiles/node_modules/sqlite3/lib/binding/node-v11-linux-x64/node_sqlite3.node" is installed via remote
sphericalmercator@1.0.2 node_modules/sphericalmercator
step@0.0.5 node_modules/step
xmlhttprequest@1.5.0 node_modules/xmlhttprequest
underscore@1.6.0 node_modules/underscore
jquery@1.6.3 node_modules/jquery
└── htmlparser@1.7.7
express@2.5.11 node_modules/express
├── qs@0.4.2
├── mime@1.2.4
├── mkdirp@0.3.0
└── connect@1.9.2 (formidable@1.0.17)
connect@3.4.0 node_modules/connect
├── utils-merge@1.0.0
├── parseurl@1.3.0
├── debug@2.2.0 (ms@0.7.1)
└── finalhandler@0.4.0 (escape-html@1.0.2, unpipe@1.0.0, on-finished@2.3.0)
backbone@0.3.3 node_modules/backbone
tilelive@4.5.3 node_modules/tilelive
└── optimist@0.6.1 (wordwrap@0.0.3, minimist@0.0.10)
bones@1.3.29 node_modules/bones
├── underscore@1.1.7
├── optimist@0.6.1 (wordwrap@0.0.3, minimist@0.0.10)
└── mirror@0.3.3 (uglify-js@1.0.2)
modestmaps@3.3.5 node_modules/modestmaps
wax@6.4.2 node_modules/wax
jsdom@0.8.3 node_modules/jsdom
├── cssom@0.2.5
├── nwmatcher@1.3.6
├── htmlparser2@3.8.3 (domelementtype@1.3.0, entities@1.0.0, domhandler@2.3.0, readable-stream@1.1.13, domutils@1.5.1)
├── cssstyle@0.2.30 (cssom@0.3.0)
└── request@2.65.0 (aws-sign2@0.6.0, forever-agent@0.6.1, caseless@0.11.0, stringstream@0.0.5, tunnel-agent@0.4.1, oauth-sign@0.8.0, isstream@0.1.2, json-stringify-safe@5.0.1, extend@3.0.0, node-uuid@1.4.7, qs@5.2.0, combined-stream@1.0.5, tough-cookie@2.2.1, mime-types@2.1.7, form-data@1.0.0-rc3, http-signature@0.11.0, bl@1.0.0, hawk@3.1.1, har-validator@2.0.2)
mbtiles@0.2.9 node_modules/mbtiles
├── underscore@1.5.2
└── sqlite3@2.2.7 (set-immediate@0.1.1, nan@1.1.2)
Creating tiles dir /home/charlton/Documents/MapBox/tiles
Started [Server Tile].
Started [Server Core:8888].

Bam!!!

If you have a firewall secretly hiding - don't forget to allow connections from other machines:

charlton@ubuntu:~/tilestream$ sudo ufw allow 8888

[sudo] password for charlton: 

Rules updated

Rules updated (v6)

History

I couldn't find anything about the stringify error anywhere because it is indirectly related but by trial and error I asked for the latest version of each module in the package.json file and found the outdated connect module was the culprit.

Here's the history for proof!

charlton@tileserver:~$ uname -a

Linux tileserver 3.19.0-25-generic #26~14.04.1-Ubuntu SMP Fri Jul 24 21:16:20 UTC 2015 x86_64 x86_64 x86_64 GNU/Linux

charlton@tileserver:~/tilestream$ history

    1  ifconfig

    2  sudo reboot

    3  sudo apt-add-repository ppa:chris-lea/node.js

    4  sudo apt-get update

    5  sudo apt-get install nodejs nodejs-dev npm

    6  node -v

    7  sudo apt-get install nodejs

    8  sudo apt-get install git-core

    9  sudo apt-get update

   10  git clone https://github.com/mapbox/tilestream.git

   11  cd tilestream/

   12  npm install

   13  ./index.js 

   14  vim package.json 

   15  ifconfig

   16  rm -fR node_modules/ && npm install && ./index.js --host 10.0.0.34

   17  history


This can be shortened to :

sudo apt-add-repository ppa:chris-lea/node.js

sudo apt-get update

sudo apt-get -y install nodejs git-core

git clone https://github.com/mapbox/tilestream.git && cd tilestream

vim package.json # edit the connect line

npm install && ./index.js --host 10.0.0.34

Hope this saves some hell!!

GPX to postgres bash script

To import a gpx file into postgres do this:

# ogr2ogr -append -f PostgreSQL "PG:dbname=ctbto user=postgres" 2016114.gpx

cool


And a bash script would look like this (compliments of Mapbox from http://www.mapbox.com/guides/postgis-manual/):

files=(

        2016114.gpx

        2016115.gpx

)

for file in ${files[@]}; do

        ogr2ogr -append -f PostgreSQL "PG:dbname=ctbto user=postgres" $file

done


or

for file in /dir/*.gpx
do
     ogr2ogr -append -f PostgreSQL "PG:dbname=ctbto user=postgres" $file
done


Adding a column while importing:

ogrinfo -f PostgreSQL "PG:dbname=your_db user=xxxx password=yyyy" -sql "UPDATE tracks SET track_id = 1234 WHERE ogc_fid = (SELECT max( ogc_fid ) FROM tracks)"

(from: http://gis.stackexchange.com/questions/101825/add-gpx-files-to-postgresql-database-and-add-one-column-with-a-given-value)

Coverting Radio Mobile coverage charts for use with Tilemill and QGIS

Radio mobile can produce radio transmission coverage charts based on a set of particulars related to a transmitter and a receiver. Such a thing is good to theoretically predict signal coverage in various terrain. Since the program can use SRTM data for terrain backgrounds it is reasonably useful when visualized on a GIS. The files produced when creating a coverage plot can be in various formats, .png, .tif, .kml etc but no geo-referencing data is embedded in the file so it cannot be dropped into QGIS or Tilemill for instance over a geo-referenced map background. External files containing the georeferencing data are produced but in my case I needed the data embedded in a single file.

I use the following GDAL commands to translate, warp and embed metadata in the image file so that it can be dropped into QGIS or Tilemill. I used a .png as an output from Radio Mobile which has a native alpha transparency channel so that the white parts of the picture become transparent. You can play with the various color schemes to get a good visual result. For Tilemill I used WGS84.

The radio Mobile setup was:

Picture Properties

Radio Plot Properties

Using the results of a plot named Picture16.png which is 1680 pixels by 1024 and Picture16.geo that was created while saving the picture file, use the the following command sets geographic positions for each corner pixel starting at top left (-gcp 0 0) moving clockwise, and writes to a file called temp.tif:

# gdal_translate -of GTiff -a_ullr 17.07981 49.19989 21.52019 47.40011 -a_srs EPSG:4326 -expand rgba Picture16.png output.tif

I hope someone finds this useful as I messed around for some time getting it to work, I'm sure there are other ways to get it done and don't hesitate to point out any better ways.