I just pushed quickdiff public, which I’ve been working on the past few days in relation to work on notepag.es.

It’s a small jQuery plugin that uses some assumptions about live preview DOM changes to provide cheap partial updates from a live DOM to a newly generated, and can cope with transformations on the live DOM. This was particularly needed for notepag.es, which uses MathJax. Due to the changes MathJax makes to the output, a simple DOM comparison cannot be made. quickdiff solves this issue by allowing filters to treat certain DOM nodes differently for comparison.

quickdiff works by doing a forward traversal of the two DOMs in parallel, comparing nodes as it goes. If it finds two nodes are different, it records the path through the DOM to the difference. It then performs a reverse traversal over the two DOMs and finds the first difference. If the paths are the same, it has found a single changed node in the DOM. If the paths differ, then either multiple elements have changed, or nodes have been inserted or deleted. Using the shared path of the differences gives the root node of the change, and then the next steps in the paths can be used to return a segment of nodes which contain the change.

This is as far as it goes in trying to find the actual change, to reduce computation time, and because of the assumption that the two DOMs are changing in isolated chunks each time. The chance for a live preview to have changes at both the top and bottom of the output are fairly small, excepting for operations such as search/replace.


If you spot an error and would like to submit a correction, you can view the source for this post on GitHub.

I spent a few evenings this week putting together an online notepad (paste sites are quite useful, for instance notepad.cc) that supported markdown and latex rendering, using the brilliant MathJax project. The site does 3 transformation passes, first turning markdown into html, and then highlighting code with Highlight.js, and rendering math with MathJax.

There are two editting modes, widescreen and narrowscreen. Widescreen gives a live preview on the left and editor on the right, while narrowscreen has a central editor/preview with a toggle between them. With a bit of maintenance code, scrolling/selection is preserved switching between these, so it works rather well even on small screens.

Pages can also be optionally passworded on creation, but cannot be passworded later. They’re still viewable, but need a password to save changes to.

The backend is a Node.js instance backed by a MongoDB database. Using Grasshopper and Mongoose, the server is a single file, and only 100 lines long, which also defines all model and index definitions and error handling. Nginx fronts this server and serves the static files, proxying page requests to node.

Current issue I’m working on is XSS hardiness so the site links can be trusted. Issue is that modern browsers seem to reject obvious attacks such as javascript in source attributes, so testing what is actually vulnerable isn’t so straightforward. I’ve tried a few obvious vectors from http://ha.ckers.org/xss.html, none of which seem to work. I don’t know whether I care to include IE6 as possible targets, given how many attacks work against it.

The DNS changes are still propagating, but it should be available at http://notepag.es sometime today or tomorrow.


If you spot an error and would like to submit a correction, you can view the source for this post on GitHub.

After getting deployment working on my own server, I thought I’d document the steps I took to getting a generally setup webserver on Linode. This series of steps will install git and nginx on a new server (specifically I will be using Ubuntu 10.4 LTS on Linode). Gitosis will also be setup for external access, and deployment script setup so that pushes to a repository will also update a working copy. My local machine is OSX.

First login and personal user

First step with a new server installation is to get connected and upgrade to latest packages for what’s installed already.

ssh root@yourdomain.com
apt-get update
apt-get upgrade

We also need to create a new user, and I also copy across the bash settings from root, as they enable a colour terminal and other goodies. Of course if you have your own shell configuration you want to use, you’ll know how to set that up. We also need to add our personal user to the sudoers list with visudo.

adduser <username>
cp /root/.bashrc /home/<username>/.bashrc
visudo

Below root, add a similar line like so: ALL=(ALL) ALL

SSH

We can reconnect to the server as our own user account now, or use su. We need to create an .ssh directory in the home directory.

mkdir ~/.ssh

On your own machine generate a public/private key pair if you don’t have one, or want a seperate one.

ssh-keygen -C <user-identifier>

Then copy this key onto your server into the .ssh directory. Now we don’t need to provide a password when using ssh, scp, or git. If you’ve used a passphrase with your key, you will be prompted for it. On OSX you can use keychain functionality to store it, or ssh-agent on OSX or *nix.

scp ~/.ssh/id_rsa.pub <username>@yourdomain.com:.ssh/authorized_keys2

Securing SSH

At the moment our server is accepting all password attempts for ssh, and root logins. It’s common sense to lock this down a bit to reduce the security risks. The config can be editted like so:

sudo nano /etc/ssh/sshd_config

I usually disable both RootLogin and PasswordAuthentication to be quite safe. Disabling PasswordAuthentication means you must have ssh keys already shared in order to login to the server. Alternatively to reduce risk, you can change the ssh port to thwart bots that attack public ssh on 22.

PermitRootLogin no
PasswordAuthentication no

Once configured, restart sshd with the following command.

sudo /etc/init.d/ssh restart

Nginx

Nginx is a lightweight webserver that can easily serve static content and forward requests onto other local services for dynamic content.

sudo apt-get install nginx
sudo /etc/init.d/nginx start

The configuration file can be edited inside etc. The default contains some examples of different configurations, and you can declare multiple virtual hosts in the same file.

sudo nano /etc/nginx/sites-available/default

I will be setting my default sites directory to /home/<username>/www. Once your configuration is done, you can tell nginx to reload the config with the following command.

sudo nginx -s reload

It’s a good idea after changing configuration to make sure it’s all setup, although nginx will complain at you if the configuration file is not syntactically valid.

mkdir ~/www
echo "Hello from Nginx." > ~/www/index.html

Git

Next step is to install git-core and gitosis so we can host repositories. The following script installs the dependencies, sets up a user for git, and initialises gitosis in the new users home directory, with your own ssh key used as the first authorised user (using path from earlier, change if necessary).

sudo apt-get install git-core gitosis

sudo adduser \
  --system \
  --shell /bin/sh \
  --gecos 'git version control' \
  --group \
  --disabled-password \
  --home /home/git \
  git

sudo -H -u git gitosis-init < ~/.ssh/authorized_keys2

On your local machine you can then pull down gitosis-admin, and configure remotely using git.

git clone git@example.com:gitosis-admin.git

This will give a directory with a configuration file, and a keys directory for adding new public keys as authorised for access.

Deployment Test Repo

For now we’ll create a test_deploy repository. To do this we’ll need to add a group to our gitosis configuration as below. Your user identifier is used in the gitosis-admin group declaration, but can also be found by looking at the end of your public key file.

[group deploy-test]
writable = deploy_test
members = <user-identifier>

Then we can push the new configuration to the server.

git commit -am "Allow creation of deploy_test repo."
git push

And then we can setup a local repo and push its contents to our new remote repo.

mkdir deploy_test
cd deploy_test
git init
echo "Hello from deploy_test" > index.html
git add .
git commit -am "First Commit"
git remote add origin git@example.com:deploy_test.git
git push
git pull origin master

Auto deployment (single user)

If having your deployments owned by the git user is ok for your needs, then you can pull your application into a directory owned by git.

sudo su git -c 'git clone /home/git/repositories/deploy_test.git /home/git/www'

You can also change your nginx configuration so your site is pointing at this new directory. For deployment, skip to post-update hook.

Auto deployment (multi-user)

If we want to deploy our code under different users than the git user, then we have a little bit of gymnastics to perform. The choice is either to have git and your other users in the same group, and make sure all files are in the appropriate group and permissions set. The other option is detailed below, which is to create an executable to perform a git pull, and use the SUID bit to perform it as our user during the post-update hook.

We can’t do this in a script as the linux kernel ignores the SUID bit on scripts for security reasons. First we make sure to have gcc and libc6-dev installed so we can compile the executable.

sudo apt-get install gcc libc6-dev
nano ~/gitpull.c

The code for our executable then looks like this:

#include <unistd.h>

int main(int argc, char *argv[]) {
  if (argc == 1) return 0;
  chdir(argv[1]);
  system("git pull");
  return 0;
}

All this does is move to a provided directory and perform a git pull. Now we need to lock this binary to be only executable, and set the SUID option.

gcc gitpull.c -o gitpull
chmod 111
chmod u+s gitpull

We also need to pull our repo into www.

rm -rf ~/www
git clone /home/git/repositories/deploy_test.git ~/www

post-update hook

Now we have our target git copy, we just need to create the post-update hook for git to execute.

cd /home/git/repositories/deploy_test.git/hooks
sudo cp post-update.sample post-update
sudo nano post-update

If you have created the executable for performing the pull, you can use the script below (pointing appropriately to the gitpull binary and the repo).

unset GIT_DIR
/home/<username>/gitpull "/home/<username>/www"
set GIT_DIR = "."
exec git-update-server-info

Otherwise the following script can be used.

unset GIT_DIR
cd /home/<username>/www
git pull
set GIT_DIR = "."
exec git-update-server-info

And now we can deploy to our website with a simple git push!

chris-mbp:deploy_test chris$ echo "Change :o" > index.htm
chris-mbp:deploy_test chris$ git commit -am "Deploy test"
[master 956589b] Deploy test
 1 files changed, 1 insertions(+), 1 deletions(-)
chris-mbp:deploy_test chris$ git push
Counting objects: 5, done.
Writing objects: 100% (3/3), 247 bytes, done.
Total 3 (delta 0), reused 0 (delta 0)
remote: From /home/git/repositories/deploy_test
remote:    613b710..956589b  master     -> origin/master
remote: Updating 613b710..956589b
remote: Fast-forward
remote:  index.htm |    2 +-
remote:  1 files changed, 1 insertions(+), 1 deletions(-)
To git@chris-spencer.co.uk:deploy_test.git
   1e1a850..956589b  master -> master

I hope this has been helpful!


If you spot an error and would like to submit a correction, you can view the source for this post on GitHub.