Alson Kemp

For those of you interested in Turbinado, here’s a quick status update:

• I separated the code for the turbinado.org website from the code for the framework.  The framework is here and the website code is here.
• I’m going to finish up implementing HAML templating for Turbinado in the next few days.
• After HAML templates are in, I’ll provide a tutorial on implementing a mini-CMS/wiki in Turbinado (the code is already in the website.  The standard-Rails-ish “look, Mom!  No code!” type of tutorial.  Just enough to convince you to download it, but not enough to get you to be significantly productive. 
• Adam Stark is providing some greatly needed polish here as he attempts to get this beastie to build.  Turbinado really needs to be easier to build…
• Diego Echeverri is doing some work to get Turbinado to work with GHC 6.10 here. I had a difficult time getting my HSP-ish View templates working with 6.10, so I hope Diego can do it. I’d greatly prefer to be working with 6.10, but I couldn’t get there…

Writing a little web framework turns out to be a lot of work (it’s all the little stuff (documentation!!) that really gets ya).  I’ve greatly appreciated the ability to build on the work of others (especially Niklas Broberg, Don Stewart, Bjorn Bringert and John Goerzen) and am grateful that others are providing help to this fledgling project.

GitHub is a pretty sweet system. One of the best aspects of web 2.0 has been the focus on simple, straightforward web app usability/utility.  GitHub is a great example of some of web 2.0′s best attributes: a really interesting model coupled with straightforward usability and just a touch of AJAX.  I never would have tried Git without GitHub.

Git is pretty interesting, too. Turns out that git is effective at encouraging social involvement in coding. The Turbinado project has been forked a few times… ‘Forked’?! Wait! That’s what happens when projects are in deep trouble, right?!

Maybe so, but not necessarily in git. Since git is a decentralized revision control system (like our beloved darcs), ‘forking’ is roughly equivalent to ‘checking out’. A ‘fork’ is a good thing, so Rails’ fork count (398) is heroic.

[Post updated to reflect comments. ...too much late night typing...] Lazy evaluation is a very novel aspect of Haskell. Turns out that it’s not that difficult to think about.

A very common example of lazy-ish evaluation is ‘&&’ operators used in lots of languages (using C#):

if ( (obj != null) && (obj.someMethod() == somethingElse) ) {
  // do something
}

Read the rest of this entry »

HAML‘s lovely. As I’ve been working with Turbinado, I’ve been having some issues with HSP. HSP is an insanely impressive piece of software, but its error messages can be a bit unclear. So I started playing around a bit with HAML. Got me wondering “How easy would be to write a HAML parser in Haskell?”

So I tried. Here’s a first-pass, to-be-updated [and somewhat incomplete] HAML parser for Haskell. Not all of the features are implemented, but it’s a start.  It generates HTML bits suitable for compilation by GHC.

Input

f = content
 
page = #content
         .left.column
         %h2 Welcome to our site!
         %p = print_information
         %p
           = print_inline
         .right.column
           = render
         [abba = ding, ding = abba] dinger

Output

f =
    (stringToHtml "content")
page =
       ((tag "div"![strAttr "id" "content"])
         ((tag "div"![strAttr "class" "left column"])
         ((tag "h2")
             (stringToHtml "Welcome to our site!")
         )
         +++
         ((tag "p")
            (stringToHtml print_information)
         )
         +++
         ((tag "p")
           (stringToHtml print_inline)
         )
         +++
         ((tag "div"![strAttr "class" "right column"])
           (stringToHtml render)
         )
         +++
         ((tag "div"![strAttr "abba" "ding", strAttr "ding" "abba"])
                                    (stringToHtml "dinger")
         )
         )
       )

le Code

module Main where
 
import Text.ParserCombinators.Parsec
import Text.ParserCombinators.Parsec.Language
import Text.ParserCombinators.Parsec.Pos
import qualified Text.ParserCombinators.Parsec.Token as T
import Data.Char
import Data.List
import Data.Maybe
import System.IO.Unsafe
 
main = do s <- getContents
          case (parse mainParser "stdin" s) of
            Left  err -> putStrLn "Error: " >> print err
            Right hs  -> putStrLn hs
 
-- Try to parse HAML, otherwise re-output raw lines
 
mainParser = do whiteSpace
                ls <- many1 (hamlCode <|> tilEOL)
                return $ unlines ls
--
-- * HAML lexer
--
hamlLexer = T.makeTokenParser emptyDef
whiteSpace= T.whiteSpace hamlLexer
lexeme    = T.lexeme hamlLexer
symbol    = T.symbol hamlLexer
natural   = T.natural hamlLexer
parens    = T.parens hamlLexer
semi      = T.semi hamlLexer
squares   = T.squares hamlLexer
stringLiteral= T.stringLiteral hamlLexer
identifier= T.identifier hamlLexer
reserved  = T.reserved hamlLexer
reservedOp= T.reservedOp hamlLexer
commaSep1 = T.commaSep1 hamlLexer
--
-- * Main HAML parsers
--
 
-- hamlCode is just many identifiers (e.g. 'func a b c' followed by '=' followed by a hamlBlock
-- func a b c = %somehaml
hamlCode = try ( do is <- many1 identifier
                    symbol "="
                    currentPos <- getPosition
                    x <- manyTill1
                          (lexeme $ hamlBlock)
                          (notSameIndent currentPos)
                    return $ (concat $ intersperse " " is) ++
                             " = \n" ++
                             (concat $ (intersperse (indent currentPos ++ "+++\n")  $ filter (not . null) $ x))
                  )
 
-- A Block may start with some whitespace, then has a valid bit of data
hamlBlock   = do currentPos <- getPosition
                 bs <- manyTill1
                      (pTag <|> pText)
                      (notSameIndent currentPos)
                 return $ intercalate (indent currentPos ++ "+++\n") bs
 
pTag    = do    currentPos <- getPosition
                try
                    (do t  <- lexeme tagParser
                        ts <- (isInline currentPos >> char '/' >> return []) <|>
                              (hamlBlock)
                        return $ intercalate "\n" $ filter (not . null) $
                          [ (indent currentPos) ++ "((" ++ (if (null ts) then "i" else "") ++ t  ++ ")"
                          , if null ts then [] else ts
                          , (indent currentPos) ++ ")\n"]
                    )
 
pText = lexeme stringParser
 
notSameIndent p = (eof >> return []) <|>
                  (do innerPos <- getPosition
                      case (sourceColumn p) == (sourceColumn innerPos) of
                                True  -> pzero
                                False -> return []
                  )
 
--
-- * Various little parsers
--
 
tagParser :: CharParser () String
tagParser = do     t <- optionMaybe tagParser'
                   i <- optionMaybe idParser
                   c <- optionMaybe (many1 classParser)
                   a <- optionMaybe attributesParser
                   if (isJust t || isJust i || isJust c || isJust a)
                     then
                       do return $ "tag \"" ++ (fromMaybe "div" t) ++ "\"" ++
                           (if not (isJust i || isJust c || isJust a) then "" else
                              concat $
                               [ "!["
                               , intercalate ", " $ filter (not . null)
                                   [ (maybe "" (\i' -> "strAttr \"id\" \"" ++ i' ++ "\"") i)
                                   , (maybe "" (\c' -> "strAttr \"class\" \"" ++ (intercalate " " c') ++ "\"") c)
                                   , (maybe "" (\kv -> intercalate ", " $ map (\(k,v) -> "strAttr \"" ++ k ++ "\" \"" ++ v ++ "\"") kv) a)
                                   ]
                               , "]"]
                           )
                     else pzero
 
tagParser' :: CharParser () String
tagParser' =  do char '%'
                 many1 termChar
 
idParser :: CharParser () String
idParser = do char '#'
              many1 termChar
 
classParser :: CharParser () String
classParser = do char '.'
                 many1 termChar
 
attributesParser :: CharParser () [(String, String)]
attributesParser = squares (commaSep1 attributeParser)
 
attributeParser :: CharParser () (String, String)
attributeParser = do k <- identifier
                     symbol "="
                     cs <- many1 identifier
                     return (k, intercalate " " cs)
 
stringParser :: CharParser () String
stringParser = do   currentPos <- getPosition
                    modifier <- optionMaybe (char '=' <|> char '-')
                    whiteSpace
                    c <- alphaNum
                    cs<- tilEOL
                    case modifier of
                      Just '-' -> return $ (indent currentPos) ++ "-" ++ c:cs
                      Just '=' -> return $ (indent currentPos) ++ "(stringToHtml " ++ c:cs ++ ")"
                      Nothing  -> return $ (indent currentPos) ++ "(stringToHtml \"" ++ c:cs ++ "\")"
 
--
-- * Utility functions
--
 
isInline     p = do p2 <- getPosition
                    case (sourceLine p  ) == (sourceLine p2) of
                      True -> return []
                      False -> pzero
isSameIndent p1 p2 = (sourceColumn p1) == (sourceColumn p2)
 
tilEOL = manyTill1 (noneOf "\n") eol
eol = newline <|> (eof >> return '\n')
 
termChar = satisfy (\c -> (isAlphaNum c) || (c `elem` termPunctuation) )
termPunctuation = "-_"
indent p = take (sourceColumn (p) - 1) (repeat ' ')
 
manyTill1 p e =  do ms <- manyTill p e
                    case (null ms) of
                      True  -> pzero
                      False -> return ms

Golly, but I wish that I’d cleaned up the code, but there it is in all of its raw, un-thought-through glory…

More progress on Turbinado. Look! A whole “+0.1″!

Progress continues on Turbinado.  Turbinado can be found at:

http://www.turbinado.org


The source can be found at:

http://github.com/alsonkemp/turbinado/tree/master

   (see the /App directory for the code for www.turbinado.org)

New in V0.2:
  * A better "Environment" type (rather than using Dynamics) (-> a 50%
speed boost?  That's unpossible!);
  * A functional early version of an ORM for PostgreSQL (note: still
needs to handle updates; hdbc-postgresql has a bug with sensing
nullable columns);
  * A prettier website;
  * Licensing -> BSD.

Expect V0.3 in the next week or so with:
  * More view helpers;
  * An ORM which handles INSERT/UPDATE...;
  * A little CMS built using the ORM.

Future release:
  * Separate the website out from the framework.  For now, they're
evolving together, so live together. 

Posted to the Haskell mailing list:

I'd like to announce Turbinado, a very young and raw MVC web framework
for Haskell.  While the framework doesn't exactly copy Ruby on Rails,
it certainly rhymes...  It's very early days for Turbinado, but the
framework is moving along nicely.  There are still issues to be ironed
out and architectural details to be decided, so help/contribution would be
very much appreciated.


Turbinado can be found at:
http://www.turbinado.org

The source can be found at:
http://github.com/alsonkemp/turbinado/tree/master
(see the /App directory for the code for www.turbinado.org)

Turbinado:
* Provides a fast web server (based on HSP; see
http://turbinado.org/Home/Performance);
* Provides a straightforward organization for your website (courtesy
of Rails);
* Uses simple HTML-like templating (courtesy of HSX);
* Is easily extensible (courtesy of an Environment built out of Map
String Dynamic, not the most type-safe of beasties; Help!);
* Configurable routing (see Config/Routes.hs).

Turbinado is currently lacking:
* Documentation...
* An easy install...
* A database ORM based on HDBC (visibly incomplete and ugly in
Turbinado/Database/ORM);
* Many more HTML helpers;
* Controllers for partials (lightweight "controls" ala ASP.NET);
* Strong error reporting and handling;
* Lots of functionality and plugins;
* ... the favorite feature that you want to develop for Turbinado ...

You’ve probably heard of Six Sigma, the quality management philosophy/practice started at Motorola. Great way to identify problem areas of product design, development and production. Sets a goal of having 99.9997% efficiency in the product life cycle.

We have a fairly traditional SDLC process for our web site development and I’ve been thinking about what the “right” process is for our development efforts. While poking around, I’ve seen a few blog posts about applying six sigma to web marketing or software development.

Got me thinking: should 6 sigma be applied to web software development? Maybe 4 sigma would be better?

Really, do I want NetFlix to provide me with 99.9997% quality in their website? Well, they better handle credit cards with 99.9997% accuracy, but I would prefer that they provide very good quality on their website and focus on rolling out new and improved features. I wouldn’t be too irked if their recommendation engine got better and, in return, I saw an occasional visual flaw or 404. Considering that most web flaws are quickly spotted and easily fixed, it makes some sense to “release early, release often”: don’t overinvest in quality when fixing the issue costs little in terms of time, money or reputation, especially when the benefit of additional features to the user is great.

What about Microsoft Windows? Unlike a web application, I have to install this bit of software and, unlike a web application, it’s not easily updated.

So I’ll vote for Four Sigma quality levels on non-critical areas of consumer websites. 99.379% quality in features is pretty good and I won’t lose sleep over the 0.621% of features that have some issues… But we’ll commit to fix them ASAP.

Great post about the benefits of using AJAX to shrink a conversion funnel here.

Recently, we were working on a workflow within our site and and spent a bunch of time taking a 6-page funnel down to a pretty AJAXed 3-page funnel. Got it all tested, put it in production!, and …no effect on fallout in the funnel… But, wait, everyone knows that “making a workflow short and snappy will decrease fallout”, right?

The explanation we came up with when thinking about the lack of improvement was that we’d neglected to consider our user’s motivations & incentives when thinking about our funnel.

The funnel we were working on was a Submit A Review type funnel. We’re unusual in that we collect a lot of data from members in that funnel. We get big, comprehensive reviews from members. It takes 2-3 minutes to fill out our review form. Which is awesome for our other members because, when they’re trying to figure out with whom to spend $75,000 adding a room to their house, they get serious data on the possible contractors.

That said, it’s kinda hard on the member that’s submitting the data and the member knows that’s it’s going to be hard, so they only enter the funnel if they’re serious about completing it.  The hard work we did on simplifying the funnel made things easier on the highly-motivated members who were going to submit reports anyway. Great for the member; not so great for us (ROI ~= 0%).

Why are these members so highly motivated? Our guess is because their incentive to post a review is altruism. They had a great experience with a service provider or a horrible experience with a service provider and they want to let the other members know.  Posting a review gets them a bit of personal satisfaction.

Naturally, the number of members who are driven by altruism is pretty low. So if we want to get review submissions up, we need to add other incentives. The incentive hierarchy probably looks something like: altruism … reminder/call-to-action … community/reputation … chance-to-win-a-prize … $$$.  As we use other incentives to drive review submissions, the shorter, snapper funnel will help keep fallout low. So our work wasn’t for nought… Just maybe not for much right now…