Julian Yap

    Digital Ocean - First impressions

    I signed up for Digital Ocean the other day since I was planning on using it for some online services. I also felt the service had reached a point of growth and maturity where it felt like they were "sticking around".

    Here are some first impressions.

    You can choose to create a virtual machine (a "droplet") in 1 of 4 data centers (a "region"). Only the "New York 2" data center supports Private Networking so for example if you create a VM in "San Francisco 1", the VM will only have a public IP interface. If you are thinking of deploying a permanent system which will use N+1 servers then deploying to the "New York 2" data center is your only option so you don't need to expose services on a public IP just to talk to another one of your servers. When creating a VM, "New York 2" is the default data center selected.

    Ubuntu 12.04 x32 is the default distribution selected when creating a VM. You should change this to a 64-bit distribution so that you have less hassles in case you want to resize your VM. I had initially installed an Ubuntu 13.04 x64 VM but I had issues starting up the IPTables firewall and noticed that all of the documentation refers to 12.04 which would then be considered their best supported and maintained Ubuntu distribution version.

    You should add a SSH key first in the web console so that you can select this during the install. The installer only installs with a 'root' user and emails you the created password. Having your SSH key pre-installed means you can automate additional install process items such creating another user account and disabling remote 'root' access. Digital Ocean also has a nice REST API which spits back JSON.

    On the default Ubuntu 12.04 x64 install there was no firewall installed by default. You should definitely install UFW and implement a security policy for SSH access at a bare minimum.

    I ran some quick tests and benchmarks and Digital Ocean lives up to the hype. Performance is fast and consistent.

    The main issue I have with VM online providers is inconsistent performance, be it disk, CPU or network IO. Inconsistent performance means you need to implement less than ideal workarounds such as overprovisioning your servers or performing workarounds like implementing software RAID 10.

    Imagine: The flexibility of virtual machines with the predictability of bare metal.

    Another thing I see myself doing is spinning up a VM just for quick testing. Creating a VM literally takes a minute.

    Digital Ocean looks to have a great differentiator with SSD disks, pricing and overall impressive virtual machine performance. At this early stage I can definitely recommend checking them out.

    Here is a quick ApacheBench test of a Go application serving up a static template.

    # sysctl -w net.netfilter.nf_conntrack_max=131072
    # ab -n 20000 -c 20 http://myserver/new/
    This is ApacheBench, Version 2.3 <$Revision: 655654 $>
    Document Path:          /new/
    Document Length:        4022 bytes
    Concurrency Level:      20
    Time taken for tests:   29.186 seconds
    Complete requests:      20000
    Failed requests:        0
    Write errors:           0
    Total transferred:      82740000 bytes
    HTML transferred:       80440000 bytes
    Requests per second:    685.27 [#/sec] (mean)
    Time per request:       29.186 [ms] (mean)
    Time per request:       1.459 [ms] (mean, across all concurrent requests)
    Transfer rate:          2768.51 [Kbytes/sec] received
    Connection Times (ms)
                  min  mean[+/-sd] median   max
    Connect:        0    0   0.1      0       5
    Processing:     2   29   3.0     29      53
    Waiting:        0   29   3.0     29      53
    Total:          2   29   3.0     29      53
    Percentage of the requests served within a certain time (ms)
      50%     29 
      66%     30 
      75%     31 
      80%     31 
      90%     32 
      95%     33 
      98%     34 
      99%     35 
     100%     53 (longest request)

    Here is a test of downloading a 100MB file from various locations from a VM created at the "San Francisco 1" data center.

    # curl -O http://speedtest.fremont.linode.com/100MB-fremont.bin
      % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                     Dload  Upload   Total   Spent    Left  Speed
    100  100M  100  100M    0     0  44.7M      0  0:00:02  0:00:02 --:--:-- 51.2M
    # curl -O http://speedtest.tokyo.linode.com/100MB-tokyo.bin
      % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                     Dload  Upload   Total   Spent    Left  Speed
    100  100M  100  100M    0     0  12.5M      0  0:00:07  0:00:07 --:--:-- 16.4M
    # curl -O http://speedtest.london.linode.com/100MB-london.bin
      % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                     Dload  Upload   Total   Spent    Left  Speed
    100  100M  100  100M    0     0  8459k      0  0:00:12  0:00:12 --:--:-- 10.0M
    # curl -O http://speedtest.dallas.linode.com/100MB-dallas.bin
      % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                     Dload  Upload   Total   Spent    Left  Speed
    100  100M  100  100M    0     0  21.9M      0  0:00:04  0:00:04 --:--:-- 22.8M


    Using Anonymous Structs to pass data to templates in GoLang

    One thing that is quite common with Python web frameworks like Django, Flask or Bottle is passing multiple objects to a view.

    Something like this in Django is common where a context dictionary with multiple objects is passed back to the template for rendering:

    from django.shortcuts import render
    def my_view(request):
            context = {'poll': p, 'error_message': "You didn't select a choice.",}
            return render(request, 'polls/detail.html', context)

    While reading through some examples in GoLang is seemed strange that all the tutorials and examples only passed a single object back to the template.

    Here is an example:

    package main
    import (
    type Paste struct {
        Expiration string
        Content    []byte
        UUID       string
    func pasteHandler(w http.ResponseWriter, r *http.Request) {
        paste_id := strings.TrimPrefix(r.URL.Path, "/paste")
        paste := &Paste{UUID: paste_id}
        data := paste
        t, _ := template.ParseFiles("templates/paste.html")
        t.Execute(w, data)

    Your template code would then look something like this:

    Expiration: {{ .Expiration }}
    UUID: {{ .UUID }}

    What if we want to pass multiple objects to the template?

    A really clean solution I found was to use an anonymous struct.

    In the following example, we are now modifying the pasteHandler function in the above example to pass extra boolean flags to the template.

    func pasteHandler(w http.ResponseWriter, r *http.Request) {
        paste_id := strings.TrimPrefix(r.URL.Path, "/paste")
        paste := &Paste{UUID: paste_id}
        keep_alive := false
        burn_after_reading := false
        data := struct {
            Paste *Paste
            KeepAlive bool
            BurnAfterReading bool
        } {
        t, _ := template.ParseFiles("templates/paste.html")
        t.Execute(w, data)

    We then need to modify out template code to access the objects like this:

    Expiration: {{ .Paste.Expiration }}
    UUID: {{ .Paste.UUID}}
    {{ if .BurnAfterReading }}
    BurnAfterReading: True
    {{ else }}
    BurnAfterReading: False
    {{ end }}

    TUESDAY, 26 MARCH 2013

    Setting up a Ruby rbenv environment on OS X

    rbenv is the best way to set up a Ruby environment. It’s pretty essential since you can test your stack against different versions of Ruby and gems such as Rails.

    … It’s pretty much like virtualenv for Python.

    I don’t always use Ruby but when I do I use rbenv (said in the Dos Equis man voice).

    Install rbenv and ruby-build via brew

    brew update
    brew install rbenv
    brew install ruby-build

    Add rbenv init to your shell to enable shims and autocompletion.

    echo 'eval "$(rbenv init -)"' >> ~/.bash_profile

    Select the version of Ruby to install. First list all the available versions

    rbenv install -l

    For example, let’s install the latest version in the 1.9.3 branch

    export VERSION=1.9.3-p392

    Build and install Ruby without documentation (saves time)

    CONFIGURE_OPTS="--disable-install-doc" rbenv install $VERSION

    Set the global Ruby version

    rbenv global $VERSION

    Disable ri and rdoc documentation for Gems. Add to ~/.gemrc

    install: --no-rdoc --no-ri
    update: --no-rdoc --no-ri


    cat << EOT > ~/.gemrc
    install: --no-rdoc --no-ri
    update: --no-rdoc --no-ri

    Bonus points. Install Rails

    gem install rails

    MONDAY, 11 JUNE 2012

    Mac Pro users and their sense of entitlement

    I bet this is the last Mac Pro. If you wanted to kill a product line, an “update” like today’s would be a good way to clear out parts and keep selling to a few desperate buyers for a bit longer without any real investment.

    This is pathetic.

    No. What’s pathetic is the sense of entitlement that certain Mac users have.

    Relax, the Mac Pro will get a proper update next year. It’s as if the 2012 debut of an iPad with retina display, the debut of a MacBook Pro with retina display and new form factor for the future of that line and an upcoming iPhone 5 wasn’t good enough for some people.

    According to Tim Cook:

    Our pro customers are really important to us…don’t worry as we’re working on something really great for later next year.

    Apple hasn’t forgotten about ‘pros’. They fully understand their roots in the desktop publishing market and other professional industries. It’s just that Apple has changed as a company and is now the biggest technology company in the world with a different product line from 10 years ago. With that new found status comes priorities, increased shareholder interest and increased competition. Perhaps the picture of where Apple is now with the general computing market is where Apple always wanted to be when they first began in the late 70’s? I think so.

    Nonetheless, Apple will always give you the power to create.

    FRIDAY, 30 MARCH 2012

    Senator Al Franken on antitrust laws and privacy

    United States Senator Al Franken spoke to the American Bar Association last night on the topic of antitrust laws and privacy in the US. Page 18 to 27 discuss Google and Facebook.

    … accumulating data about you isn’t just a strange hobby for these corporations. It’s their whole business model. And you are not their client. You are their product.


Follow Me

Google+ RSS