Alson Kemp

CoffeeScript is a great example of how to de-noise a language like Javascript. (Of course, I know people that consider braces to be a good thing, but lots of us consider them noise and prefer significant whitespace, so I’m speaking to those folks.) What would Go code look like with some of CoffeeScript’s denoising?

TL;DR : the answer is that de-noised Go would not look much different than normal Go…

As an experiment, I picked some rules from CoffeeScript and re-wrote the Mandelbrot example from The Computer Benchmarks Game. Note: this is someone else’s original Go code, so I can’t vouch for the quality of the Go code….

Here’s the original Go code:

/* targeting a q6600 system, one cpu worker per core */
const pool = 4

const ZERO float64 = 0
const LIMIT = 2.0
const ITER = 50   // Benchmark parameter
const SIZE = 16000

var rows []byte
var bytesPerRow int

// This func is responsible for rendering a row of pixels,
// and when complete writing it out to the file.

func renderRow(w, h, bytes int, workChan chan int,iter int, finishChan chan bool) {

   var Zr, Zi, Tr, Ti, Cr float64
   var x,i int

   for y := range workChan {

      offset := bytesPerRow * y
      Ci := (2*float64(y)/float64(h) - 1.0)

      for x = 0; x < w; x++ {
         Zr, Zi, Tr, Ti = ZERO, ZERO, ZERO, ZERO
         Cr = (2*float64(x)/float64(w) - 1.5)

         for i = 0; i < iter && Tr+Ti <= LIMIT*LIMIT; i++ {
            Zi = 2*Zr*Zi + Ci
            Zr = Tr - Ti + Cr
            Tr = Zr * Zr
            Ti = Zi * Zi
         }

         // Store the value in the array of ints
         if Tr+Ti <= LIMIT*LIMIT {
            rows[offset+x/8] |= (byte(1) << uint(7-(x%8)))
         }
      }
   }
   /* tell master I'm finished */
   finishChan <- true

My quick de-noising rules are:

• Eliminate var since it can be inferred.
• Use ‘:’ instead of const (a la Ruby’s symbols).
• Eliminate func in favor of ‘-> and variables for functions.
• Replace braces {} with significant whitespace
• Replace C-style comments with shell comments “#”
• Try to leave other spacing along to not fudge on line count
• Replace simple loops with an “in” and range form

The de-noised Go code:

# targeting a q6600 system, one cpu worker per core
:pool = 4

:ZERO float64 = 0  # These are constants
:LIMIT = 2.0
:ITER = 50   # Benchmark parameter
:SIZE = 16000

rows []byte
bytesPerRow int

# This func is responsible for rendering a row of pixels,
# and when complete writing it out to the file.

renderRow = (w, h, bytes int, workChan chan int,iter int, finishChan chan bool) ->

   Zr, Zi, Tr, Ti, Cr float64
   x,i int

   for y := range workChan
      offset := bytesPerRow * y
      Ci := (2*float64(y)/float64(h) - 1.0)

      for x in [0..w]
         Zr, Zi, Tr, Ti = ZERO, ZERO, ZERO, ZERO
         Cr = (2*float64(x)/float64(w) - 1.5)

         i = 0
         while i++ < iter && Tr+Ti <= LIMIT*LIMIT
            Zi = 2*Zr*Zi + Ci
            Zr = Tr - Ti + Cr
            Tr = Zr * Zr
            Ti = Zi * Zi

         # Store the value in the array of ints
         if Tr+Ti <= LIMIT*LIMIT
            rows[offset+x/8] |= (byte(1) << uint(7-(x%8)))
   # tell master I'm finished
   finishChan <- true

That seems to be a pretty small win in return for a syntax adjustment that does not produce significantly enhanced readability. Some bits are nice: I prefer the significant whitespace, but the braces just aren’t that obtrusive in Go; I do prefer the shell comment style, but it’s not a deal breaker; the simplified loop is nice, but not incredible; eliding “var” is okay, but harms readability given the need to declare the types of some variables; I do prefer the colon for constants. Whereas Coffeescript can dramatically shorten and de-noise a Javascript file, it looks as though Go is already pretty terse.

Obviously, I didn’t deal with all of Go in this experiment, so I’ll look over more of it soon, but Go appears to be quite terse already given its design…

First, a note: this is a tiny synthetic bench.  It’s not intended to answer the question: is GCCGo a good compiler.  It is intended to answer the question: as someone investigating Go, should I also investigate GCCGo?

While reading some announcements about the impending release of Go 1.1, I noticed that GCC was implementing a Go frontend.  Interesting.  So the benefits of the Go language coupled with the GCC toolchain?  Sounds good.  The benefits of the Go language combing with GCC’s decades of x86 optimization?  Sounds great.

So I grabbed GCCGo and built it.  Instructions here: http://golang.org/doc/install/gccgo

Important bits:

• Definitely follow the instructions to build GCC in a separate directory from the source.
• My configuration was:

/tmp/gccgo/configure --disable-multilib --enable-languages=c,c++,go

I used the Mandelbrot script from The Benchmarks Game at mandlebrot.go.  Compiled using go and gccgo, respectively:

go build mandel.go
gccgo -v -lpthread -B /tmp/gccgo-build/gcc/ -B /tmp/gccgo-build/lto-plugin/ \
  -B /tmp/gccgo-build/x86_64-unknown-linux-gnu/libgo/ \
  -I /tmp/gccgo-build/x86_64-unknown-linux-gnu/libgo/ \
  -m64 -fgo-relative-import-path=_/home/me/apps/go/bin \
  -o ./mandel.gccgo ./mandel.go -O3

Since I didn’t install GCCGo and after flailing at compiler options for getting “go build” to find includes, libraries, etc, I gave up on the simple “go -compiler” syntax for gccgo. So the above gccgo command is the sausage-making version.

So the two files:

4,532,110 mandel.gccgo  - Compiled in 0.3s
1,877,120 mandel.golang - Compiled in 0.5s

As a HackerNewser noted, stripping the executables could be good. Stripped:

1,605,472 mandel.gccgo
1,308,840 mandel.golang

Note: the stripped GCCGo executables don’t actually work, so take the “stripped” value with a grain of salt for the moment. Bug here.

GCCGo produced an *unstripped* executable 2.5x as large as Go produced. Stripped, the executables were similar, but the GCCGo executable didn’t work. So far the Go compiler is winning.

Performance [on a tiny, synthetic, CPU bound, floating point math dominated program]:

time ./mandel.golang 16000 > /dev/null 

real  0m10.610s
user  0m41.091s
sys  0m0.068s

time ./mandel.gccgo 16000 > /dev/null 

real  0m9.719s
user  0m37.758s
sys  0m0.064s

So GCCGo produces executables that are about 10% faster than does Go, but the executable is nearly 3x the size.  I think I’ll stick with the Go compiler for now, especially since the tooling built into/around Go is very solid.

Additional notes from HN discussion:

• GCC was 4.8.0.  Go was 1.1rc1.  Both AMD64.

For those of you interested in Turbinado, here’s a quick status update:

• I separated the code for the turbinado.org website from the code for the framework.  The framework is here and the website code is here.
• I’m going to finish up implementing HAML templating for Turbinado in the next few days.
• After HAML templates are in, I’ll provide a tutorial on implementing a mini-CMS/wiki in Turbinado (the code is already in the website.  The standard-Rails-ish “look, Mom!  No code!” type of tutorial.  Just enough to convince you to download it, but not enough to get you to be significantly productive. 
• Adam Stark is providing some greatly needed polish here as he attempts to get this beastie to build.  Turbinado really needs to be easier to build…
• Diego Echeverri is doing some work to get Turbinado to work with GHC 6.10 here. I had a difficult time getting my HSP-ish View templates working with 6.10, so I hope Diego can do it. I’d greatly prefer to be working with 6.10, but I couldn’t get there…

Writing a little web framework turns out to be a lot of work (it’s all the little stuff (documentation!!) that really gets ya).  I’ve greatly appreciated the ability to build on the work of others (especially Niklas Broberg, Don Stewart, Bjorn Bringert and John Goerzen) and am grateful that others are providing help to this fledgling project.

GitHub is a pretty sweet system. One of the best aspects of web 2.0 has been the focus on simple, straightforward web app usability/utility.  GitHub is a great example of some of web 2.0′s best attributes: a really interesting model coupled with straightforward usability and just a touch of AJAX.  I never would have tried Git without GitHub.

Git is pretty interesting, too. Turns out that git is effective at encouraging social involvement in coding. The Turbinado project has been forked a few times… ‘Forked’?! Wait! That’s what happens when projects are in deep trouble, right?!

Maybe so, but not necessarily in git. Since git is a decentralized revision control system (like our beloved darcs), ‘forking’ is roughly equivalent to ‘checking out’. A ‘fork’ is a good thing, so Rails’ fork count (398) is heroic.

[Post updated to reflect comments. ...too much late night typing...] Lazy evaluation is a very novel aspect of Haskell. Turns out that it’s not that difficult to think about.

A very common example of lazy-ish evaluation is ‘&&’ operators used in lots of languages (using C#):

if ( (obj != null) && (obj.someMethod() == somethingElse) ) {
  // do something
}

Read the rest of this entry »

HAML‘s lovely. As I’ve been working with Turbinado, I’ve been having some issues with HSP. HSP is an insanely impressive piece of software, but its error messages can be a bit unclear. So I started playing around a bit with HAML. Got me wondering “How easy would be to write a HAML parser in Haskell?”

So I tried. Here’s a first-pass, to-be-updated [and somewhat incomplete] HAML parser for Haskell. Not all of the features are implemented, but it’s a start.  It generates HTML bits suitable for compilation by GHC.

Input

f = content
 
page = #content
         .left.column
         %h2 Welcome to our site!
         %p = print_information
         %p
           = print_inline
         .right.column
           = render
         [abba = ding, ding = abba] dinger

Output

f =
    (stringToHtml "content")
page =
       ((tag "div"![strAttr "id" "content"])
         ((tag "div"![strAttr "class" "left column"])
         ((tag "h2")
             (stringToHtml "Welcome to our site!")
         )
         +++
         ((tag "p")
            (stringToHtml print_information)
         )
         +++
         ((tag "p")
           (stringToHtml print_inline)
         )
         +++
         ((tag "div"![strAttr "class" "right column"])
           (stringToHtml render)
         )
         +++
         ((tag "div"![strAttr "abba" "ding", strAttr "ding" "abba"])
                                    (stringToHtml "dinger")
         )
         )
       )

le Code

module Main where
 
import Text.ParserCombinators.Parsec
import Text.ParserCombinators.Parsec.Language
import Text.ParserCombinators.Parsec.Pos
import qualified Text.ParserCombinators.Parsec.Token as T
import Data.Char
import Data.List
import Data.Maybe
import System.IO.Unsafe
 
main = do s <- getContents
          case (parse mainParser "stdin" s) of
            Left  err -> putStrLn "Error: " >> print err
            Right hs  -> putStrLn hs
 
-- Try to parse HAML, otherwise re-output raw lines
 
mainParser = do whiteSpace
                ls <- many1 (hamlCode <|> tilEOL)
                return $ unlines ls
--
-- * HAML lexer
--
hamlLexer = T.makeTokenParser emptyDef
whiteSpace= T.whiteSpace hamlLexer
lexeme    = T.lexeme hamlLexer
symbol    = T.symbol hamlLexer
natural   = T.natural hamlLexer
parens    = T.parens hamlLexer
semi      = T.semi hamlLexer
squares   = T.squares hamlLexer
stringLiteral= T.stringLiteral hamlLexer
identifier= T.identifier hamlLexer
reserved  = T.reserved hamlLexer
reservedOp= T.reservedOp hamlLexer
commaSep1 = T.commaSep1 hamlLexer
--
-- * Main HAML parsers
--
 
-- hamlCode is just many identifiers (e.g. 'func a b c' followed by '=' followed by a hamlBlock
-- func a b c = %somehaml
hamlCode = try ( do is <- many1 identifier
                    symbol "="
                    currentPos <- getPosition
                    x <- manyTill1
                          (lexeme $ hamlBlock)
                          (notSameIndent currentPos)
                    return $ (concat $ intersperse " " is) ++
                             " = \n" ++
                             (concat $ (intersperse (indent currentPos ++ "+++\n")  $ filter (not . null) $ x))
                  )
 
-- A Block may start with some whitespace, then has a valid bit of data
hamlBlock   = do currentPos <- getPosition
                 bs <- manyTill1
                      (pTag <|> pText)
                      (notSameIndent currentPos)
                 return $ intercalate (indent currentPos ++ "+++\n") bs
 
pTag    = do    currentPos <- getPosition
                try
                    (do t  <- lexeme tagParser
                        ts <- (isInline currentPos >> char '/' >> return []) <|>
                              (hamlBlock)
                        return $ intercalate "\n" $ filter (not . null) $
                          [ (indent currentPos) ++ "((" ++ (if (null ts) then "i" else "") ++ t  ++ ")"
                          , if null ts then [] else ts
                          , (indent currentPos) ++ ")\n"]
                    )
 
pText = lexeme stringParser
 
notSameIndent p = (eof >> return []) <|>
                  (do innerPos <- getPosition
                      case (sourceColumn p) == (sourceColumn innerPos) of
                                True  -> pzero
                                False -> return []
                  )
 
--
-- * Various little parsers
--
 
tagParser :: CharParser () String
tagParser = do     t <- optionMaybe tagParser'
                   i <- optionMaybe idParser
                   c <- optionMaybe (many1 classParser)
                   a <- optionMaybe attributesParser
                   if (isJust t || isJust i || isJust c || isJust a)
                     then
                       do return $ "tag \"" ++ (fromMaybe "div" t) ++ "\"" ++
                           (if not (isJust i || isJust c || isJust a) then "" else
                              concat $
                               [ "!["
                               , intercalate ", " $ filter (not . null)
                                   [ (maybe "" (\i' -> "strAttr \"id\" \"" ++ i' ++ "\"") i)
                                   , (maybe "" (\c' -> "strAttr \"class\" \"" ++ (intercalate " " c') ++ "\"") c)
                                   , (maybe "" (\kv -> intercalate ", " $ map (\(k,v) -> "strAttr \"" ++ k ++ "\" \"" ++ v ++ "\"") kv) a)
                                   ]
                               , "]"]
                           )
                     else pzero
 
tagParser' :: CharParser () String
tagParser' =  do char '%'
                 many1 termChar
 
idParser :: CharParser () String
idParser = do char '#'
              many1 termChar
 
classParser :: CharParser () String
classParser = do char '.'
                 many1 termChar
 
attributesParser :: CharParser () [(String, String)]
attributesParser = squares (commaSep1 attributeParser)
 
attributeParser :: CharParser () (String, String)
attributeParser = do k <- identifier
                     symbol "="
                     cs <- many1 identifier
                     return (k, intercalate " " cs)
 
stringParser :: CharParser () String
stringParser = do   currentPos <- getPosition
                    modifier <- optionMaybe (char '=' <|> char '-')
                    whiteSpace
                    c <- alphaNum
                    cs<- tilEOL
                    case modifier of
                      Just '-' -> return $ (indent currentPos) ++ "-" ++ c:cs
                      Just '=' -> return $ (indent currentPos) ++ "(stringToHtml " ++ c:cs ++ ")"
                      Nothing  -> return $ (indent currentPos) ++ "(stringToHtml \"" ++ c:cs ++ "\")"
 
--
-- * Utility functions
--
 
isInline     p = do p2 <- getPosition
                    case (sourceLine p  ) == (sourceLine p2) of
                      True -> return []
                      False -> pzero
isSameIndent p1 p2 = (sourceColumn p1) == (sourceColumn p2)
 
tilEOL = manyTill1 (noneOf "\n") eol
eol = newline <|> (eof >> return '\n')
 
termChar = satisfy (\c -> (isAlphaNum c) || (c `elem` termPunctuation) )
termPunctuation = "-_"
indent p = take (sourceColumn (p) - 1) (repeat ' ')
 
manyTill1 p e =  do ms <- manyTill p e
                    case (null ms) of
                      True  -> pzero
                      False -> return ms

Golly, but I wish that I’d cleaned up the code, but there it is in all of its raw, un-thought-through glory…

More progress on Turbinado. Look! A whole “+0.1″!

Progress continues on Turbinado.  Turbinado can be found at:
 http://www.turbinado.org

The source can be found at:
 http://github.com/alsonkemp/turbinado/tree/master
   (see the /App directory for the code for www.turbinado.org)

New in V0.2:
  * A better "Environment" type (rather than using Dynamics) (-> a 50%
speed boost?  That's unpossible!);
  * A functional early version of an ORM for PostgreSQL (note: still
needs to handle updates; hdbc-postgresql has a bug with sensing
nullable columns);
  * A prettier website;
  * Licensing -> BSD.

Expect V0.3 in the next week or so with:
  * More view helpers;
  * An ORM which handles INSERT/UPDATE...;
  * A little CMS built using the ORM.

Future release:
  * Separate the website out from the framework.  For now, they're
evolving together, so live together. 

Posted to the Haskell mailing list:

I'd like to announce Turbinado, a very young and raw MVC web framework
for Haskell.  While the framework doesn't exactly copy Ruby on Rails,
it certainly rhymes...  It's very early days for Turbinado, but the
framework is moving along nicely.  There are still issues to be ironed
out and architectural details to be decided, so help/contribution would be
very much appreciated.

Turbinado can be found at:
<a rel="nofollow" href="http://www.turbinado.org/" target="_top">http://www.turbinado.org</a>

The source can be found at:
<a rel="nofollow" href="http://github.com/alsonkemp/turbinado/tree/master" target="_top">http://github.com/alsonkemp/turbinado/tree/master</a>
(see the /App directory for the code for www.turbinado.org)

Turbinado:
* Provides a fast web server (based on HSP; see
<a rel="nofollow" href="http://turbinado.org/Home/Performance%29;" target="_top">http://turbinado.org/Home/Performance);</a>
* Provides a straightforward organization for your website (courtesy
of Rails);
* Uses simple HTML-like templating (courtesy of HSX);
* Is easily extensible (courtesy of an Environment built out of _Map
String Dynamic_, not the most type-safe of beasties; Help!);
* Configurable routing (see Config/Routes.hs).

Turbinado is currently lacking:
* Documentation...
* An easy install...
* A database ORM based on HDBC (visibly incomplete and ugly in
Turbinado/Database/ORM);
* Many more HTML helpers;
* Controllers for partials (lightweight "controls" ala ASP.NET);
* Strong error reporting and handling;
* Lots of functionality and plugins;
* ... the favorite feature that you want to develop for Turbinado ...

You’ve probably heard of Six Sigma, the quality management philosophy/practice started at Motorola. Great way to identify problem areas of product design, development and production. Sets a goal of having 99.9997% efficiency in the product life cycle.

We have a fairly traditional SDLC process for our web site development and I’ve been thinking about what the “right” process is for our development efforts. While poking around, I’ve seen a few blog posts about applying six sigma to web marketing or software development.

Got me thinking: should 6 sigma be applied to web software development? Maybe 4 sigma would be better?

Really, do I want NetFlix to provide me with 99.9997% quality in their website? Well, they better handle credit cards with 99.9997% accuracy, but I would prefer that they provide very good quality on their website and focus on rolling out new and improved features. I wouldn’t be too irked if their recommendation engine got better and, in return, I saw an occasional visual flaw or 404. Considering that most web flaws are quickly spotted and easily fixed, it makes some sense to “release early, release often”: don’t overinvest in quality when fixing the issue costs little in terms of time, money or reputation, especially when the benefit of additional features to the user is great.

What about Microsoft Windows? Unlike a web application, I have to install this bit of software and, unlike a web application, it’s not easily updated.

So I’ll vote for Four Sigma quality levels on non-critical areas of consumer websites. 99.379% quality in features is pretty good and I won’t lose sleep over the 0.621% of features that have some issues… But we’ll commit to fix them ASAP.

Great post about the benefits of using AJAX to shrink a conversion funnel here.

Recently, we were working on a workflow within our site and and spent a bunch of time taking a 6-page funnel down to a pretty AJAXed 3-page funnel. Got it all tested, put it in production!, and …no effect on fallout in the funnel… But, wait, everyone knows that “making a workflow short and snappy will decrease fallout”, right?

The explanation we came up with when thinking about the lack of improvement was that we’d neglected to consider our user’s motivations & incentives when thinking about our funnel.

The funnel we were working on was a Submit A Review type funnel. We’re unusual in that we collect a lot of data from members in that funnel. We get big, comprehensive reviews from members. It takes 2-3 minutes to fill out our review form. Which is awesome for our other members because, when they’re trying to figure out with whom to spend $75,000 adding a room to their house, they get serious data on the possible contractors.

That said, it’s kinda hard on the member that’s submitting the data and the member knows that’s it’s going to be hard, so they only enter the funnel if they’re serious about completing it.  The hard work we did on simplifying the funnel made things easier on the highly-motivated members who were going to submit reports anyway. Great for the member; not so great for us (ROI ~= 0%).

Why are these members so highly motivated? Our guess is because their incentive to post a review is altruism. They had a great experience with a service provider or a horrible experience with a service provider and they want to let the other members know.  Posting a review gets them a bit of personal satisfaction.

Naturally, the number of members who are driven by altruism is pretty low. So if we want to get review submissions up, we need to add other incentives. The incentive hierarchy probably looks something like: altruism … reminder/call-to-action … community/reputation … chance-to-win-a-prize … $$$.  As we use other incentives to drive review submissions, the shorter, snapper funnel will help keep fallout low. So our work wasn’t for nought… Just maybe not for much right now…