Hackfoofery

Alson Kemp

ANNOUNCE: Turbinado V0.6

Written by alson

March 12th, 2009 at 10:20 pm

Posted in Haskell,Turbinado

Bookmark and Share

It’s been long enough since Turbinado V0.4 that I figured I’d skip V0.5 and go straight to announcing Turbinado V0.6.  Lots of new excellent features:

  • By popular demand, support for CGI serving. Apparently some web hosts don’t support HTTP proxying, so some folks requested CGI support.
  • Statically compiled Layouts, Views, Controllers.
  • Support for “.format” in routes. If a request path is “/User/List.xml”, then the following View will be called: /App/Views/User/ListXml.hs.
  • Lower case paths.
  • Support for cookies (see here for examples).
  • Encrypted cookie sessions (see here to see how to use them).
  • Much easier installs using cabal-install.
  • Support for GHC 6.10.  GHC 6.8 is no longer supported.

Turbinado V0.7 will be all about:

  • Documentation. (seriously.)
  • User authentication.
  • Tutorials.

Installation

Installation is pretty painless if you use cabal-install, so make sure that you have cabal-install installed first.  See here: http://hackage.haskell.org/trac/hackage/wiki/CabalInstall.

To install Turbinado:

git clone git@github.com:alsonkemp/turbinado-website.git
cd turbinado-website
cabal install
[This might fail, saying that "trhsx" can't be found.  However, "trhsx" was built during the install and is probably at ~/.cabal/bin/trhsx, so copy "trhsx" to your path and re-run "cabal install".]

CGI Configuration

[Note: you don't want to use CGI without statically compiling in some Controllers, Layouts and Views.  See below.]

Usually, Turbinado is called with “-p” to specify the port the process should listen on (e.g. “turbinado -p 8080″).  When called with the “-c” flag, Turbinado will handle CGI requests.  However, because of the process setup and tear down times, responding to CGI requests takes about 250ms, which is considerably slower than responding to HTTP requests (about 1ms).

Again following Rails, Turbinado includes a CGI script called “dispatch.cgi” in the /static directory.

Apache Configuration

In order to use Turbinado’s CGI functionality with Apache, you’ll need to something like the following in order to tell Apache to allow CGI scripts in your Turbinado /static directory and to send all requests (e.g. “^(.*)$”) to the “dispatch.cgi” script.

  DocumentRoot /home/alson/turbinado-website/static
  <Directory "/home/alson/turbinado-website/static">
     Options +FollowSymLinks +ExecCGI +Includes
     AddHandler cgi-script .cgi
     AllowOverride None
     Order allow,deny
     Allow from all
  </Directory>
  RewriteRule ^(.*)$ %{DOCUMENT_ROOT}/dispatch.cgi [QSA,L]

Static Compilation Of Resources

Turbinado is designed to dynamically compile and link in the various resources (Controllers, Layouts, Views) needed to serve a request. However, it can take up to 15 seconds to complete that process the first time a particular page is requested (subsequent requests are very fast). With CGI, the server only ever sees the first request, so Turbinado would never be able to serve a CGI request faster than 10-20 seconds.

To fix this, you can now compile into the server particular resources. See Config/Routes.hs here. Turbinado stores a function along with the file path and function name in a tuple, so you just give Turbinado that information in the Routes.hs file and it’ll load those functions into the CodeStore at startup:

staticLayouts =
    [ ("App/Layouts/Default.hs", "markup", App.Layouts.Default.markup)
    ]

Support for “formats”

Rails has great support for file formats. Turbinado is trying to follow that lead. The system will try to figure out the MIME type based on the extension. According to the standard routes (Config/Routes.hs), the following path=>View mappings will occur:

/abba/ding => /App/Views/Abba/Ding.hs
/abba/ding.xml => /App/Views/Abba/DingXml.hs
/bloof/snort/1.csv => /App/Views/Bloof/SnortCsv.hs

The same Controller handles all formats; only the View will change. Also, usage of a format causes a blank Layout to be used (on the assumption that you don’t want a Layout used with a CSV, XML, etc output).

Lower case paths

Turbinado now defaults to using lower case paths (configured in Config/App.hs), so the following paths get mapped to Controllers and Views as follows:

/abba/ding_fling => /App/Controllers/Abba.hs : dingFling
                 => /App/Views/Abba/DingFling.hs

Cookies

Cookies are now supported. Examples here.

setCookie $ mkCookie "counter" (show 0)
v <- getCookieValue "counter"
deleteCookie "counter"

Cookie Sessions

Session data is encrypted and stuffed into a cookie. The encryption key is set in Config/App.hs. Session usage examples are here.

setSessionValue "counter" "0"
v <- setSessionValue "counter"
deleteSessionKey "counter"
abandonSession

with 14 comments

PS3 + Linux Media Serving (MythTV? GMediaServer? MediaTomb?)

Written by alson

March 9th, 2009 at 3:19 pm

Posted in Geekery

Tagged with

Bookmark and Share

I recently got a PS3 (which is a lovely piece of hardware and software) and, given our collections of MP3s, WMAs and AVIs (a bunch of ripped children’s DVDs so that my son is free to physically shred up the actual DVDs), have been trying to figure out how to serve media to it over the network.

Serving from Windows is ludicrously simple using Windows Media Player 11 or TVersity, but I have a small, super-old-school Linux file server used for backups-and-such and I wanted to use it to serve media.  Turns out to be very simple to do so.

MythTV is the big guy in this space, but it had a number of issues for me:

  • Not particularly straightforward to set up (considering the very simple use case I had for it).
  • Tons of functionality that I didn’t need, including a heavy front-end app (though there is a lighter weight web app I could have installed).
  • I couldn’t figure out how to get it to serve WMAs and, since I’ve ripped a bunch of CDs to WMA, this was a killer.

None of this is to say that MythTV is not a great piece of software; it was just way more than I needed.

GMediaServer is a GNU uPNP media server and it looked pretty good, but I wasn’t sure that it would serve video.  Documentation is a also bit lacking.

Enter MediaTomb.  Simple, lightweight, basic media serving.  “apt-get install mediatomb” and I was pretty much there.  A slight, very well documented modification to the configuration file and I popped open a web browser, browsed to the built-in web interface and told MediaTomb to server my /share/media directory.  Walked over to my PS3 (on which the MediaTomb server was already listed) and started browsing my media files.

The only issue I had was when I updated the media files on the server.  Sometimes MediaTomb wouldn’t see the modifications and would send to the PS3 out-of-date data, probably because I was using inotify rather than just time-based refresh.  I switched to the time-based refresh and deleted the MediaTomb SQLite database and all was right with the world.

Useful links:

  • http://www.mediatomb.cc
  • http://ubuntuforums.org/showthread.php?t=650020
  • http://www.freesoftwaremagazine.com/columns/upnp_mediatomb_ps3_and_me

without comments

Kudos to Bodo

Written by alson

February 10th, 2009 at 2:28 pm

Posted in Haskell,Turbinado

Bookmark and Share

Bodo Nauman (http://www.naumann.cc/) got Turbinado to build with GHC 6.10 (after forcing me to fix a bunch of recently added bugs…).  Details are here.  There’s currently an issue with the crypto Hackage package which doesn’t build with GHC 6.10**.  Fortunately, the darcs version of crypto builds fine.

Bodo beat me by a couple of days to an updated “How to install Turbinado” post.  Since v0.5 is freshly minted (ahem… and debugged) (and includes cookies, cookie sessions (lifted from Michael Snoyman’s hweb), a Rails-like respondTo function, …), I’ll post an update in a day or two.

Adolfo Builes also has started to replicate a Rails tutorial in Turbinado, so I hope I can point you to his results soon.


** The crypto build fails when building some test applications.  What’s weird is that “cabal install” installs the test applications into the bin directory.  Why would I want test applications in /usr/local/bin?  I don’t see a Cabal flag for “just build the libraries”…

with 3 comments

Turbinado is “not nearly as innovative”

Written by alson

February 3rd, 2009 at 2:28 pm

Posted in Haskell,Turbinado

Bookmark and Share

From Happstack:

“[If HApps doesn't do anything...] the project will eventually be superceded by other Haskell frameworks like Turbinado which are not nearly as innovative”

Woohoo!  People are talking about Turbinado!

err… Wait! Turbinado is being dissed!

Turbinado vs. HApps

Each project is valid and valuable, so I’m hesitant to get to far into a X vs. Y discussion, but here are some aspects that I thought about when evaluating HApps and developing Turbinado:

  • Turbinado is built on top of Nicklas Broberg‘s HSP/HSPR, which I consider to be an innovative-port of ASP to Haskell…
  • HApps is a much bigger project with many more features; Turbinado is small and needs your help [shameless plug: help here].
  • HApps seems more like a library of really useful functions which is very flexible; Turbinado is more like a web app framework and provides a web server along with defined mechanisms for adding Controller, Views and ComponentsConvention over Configuration = different styles, not better styles.
  • HApps seems less than simple (see here); Turbinado is all about simple (see here)
  • HApps has an aversion to relational databases; Turbinado observes that lots of people use relational databases and supports them.

Turbinado vs. HApps, again

To me, it seems as though the choice between HApps and Turbinado depends more on orientation than on functionality.  If you are interested in: opposing RDMBSs, but are into something like Sinatra, then HApps is probably the best choice; if you’re interested in writing web apps in the style of Ruby On Rails, then Turbinado (though young and pretty) may be the best choice, especially given the simplicity of building Turbinado (newly cabal installable!  to be described in a forthcoming post with tutorials, install details, singing, dancing, high kicks, etc).

Here’s To Success

Most successful languages have more than one web framework, so I hope for success for both HApps and for Turbinado. They’re very different frameworks and serve very different needs. The Haskell community would be well served if both frameworks survived and thrived.  As Merb and Rails have demonstrated, cross-pollination is excellent; the stronger HApps and Turbinado are, the more they can benefit each other.

with 26 comments

Cabal-install is awesome

Written by alson

January 31st, 2009 at 9:15 pm

Posted in Haskell

Bookmark and Share

See also: 2009-The Year Of Hackage and A Plea For Cabal-install


I’m finally getting Turbinado updated for GHC 6.10 and I owe a huge debt to the cabal-install team (my preferred currency for debt payment is beer).  Turbinado is a bitch to install by hand because it depends on specific version of various packages.  One minor rev off and everything explodes.

So I’ve been bouncing between 3 machines working on Turbinado and trying to get the build cleaned up.  Once cabal-install is installed, it’s a simple “cabal install” to get all dependencies downloaded and built.  Much better than setting up a local packages directory, downloading 10 packages, trying to get the install order figured out and trying to remember which of the 10 packages is installed or not.

It’s fantastic that GHC is finally getting a package and build manager.

If you haven’t played with cabal-install, you should:

darcs get --partial http://darcs.haskell.org/cabal-install/
cd cabal-install
sh bootstrap.sh
ln -s ~/.cabal/bin/cabal ~/bin/cabal  ### or somewhere else useful
cabal update

My only want for cabal-install is for it to install executables somewhere more useful than ~/.cabal.  But that’s a minor gripe…

with 13 comments

ANNOUNCE: Turbinado V0.4

Written by alson

January 18th, 2009 at 2:07 pm

Posted in Haskell,Turbinado

Bookmark and Share

Turbinado continues to evolve:

Turbinado (http://www.turbinado.org) is an easy 
to use Model-View-Controller-ish web framework for Haskell.

The source for the framework can be found at:
http://github.com/alsonkemp/turbinado

The source for the website turbinado.org can be found at:
http://github.com/alsonkemp/turbinado-website
(see the /App directory for the code for www.turbinado.org)

Release 0.4 contains:
* A dramatically improved ORM (or Type-Relation Mapper) 
   which handles foreign keys.  Still PostgreSQL only at this point.
* All dependencies in tmp/dependencies to ease building the application.
* In code documentation (not complete, but starting).
* Documentation (see http://turbinado.org/Architecture).

Release 0.5 will focus on:
* Ease of installation!
* Moving to GHC 6.10 (whenever Debian shifts).  Diego Eche 
   provided a port from plugins to ghc-api.
* Additional functionality (e.g. sessions, authentication, etc).
* Tutorials.

without comments

Sad about Import Cycles …

Written by alson

January 3rd, 2009 at 8:39 pm

Posted in Haskell

Bookmark and Share

Turbinado has the beginnings of a database ORM built in.  I’m hoping to make it super simple to interact with a relational DB, but I’m bumping up again import cycles. Ticket about import cycles here.

The Problem

import qualified App.Models.PageModel as Page
import qualified App.Models.CommentModel as Comment
 
someFn = do p  <- Page.find "tutorial"  -- find using a primary key of "tutorial"
            cs <- Page.findAllChildComment p
            let c = head cs
            p2 <- Comment.getParentPage c

Yeah, could be prettier (hmm… maybe Rails’ pluralization is good after all…), but that’s a pretty nice way to access related Pages and Comments in a SQL database. What doesn’t work, though are the imports. Since Page.findAllChildComment returns Comments, PageModel needs to import CommentModel. But, since CommentModel also uses the Page type, CommentModel needs to import PageModel. And thus the cycle begins…

Fix #1

The custom seems to be to push the data types out to a Types.hs file.

Assuming that Types.hs is imported and exported by all Models, then the following doesn’t work:

import qualified App.Models.PageModel as PageModel
import qualified App.Models.CommentModel as CommentModel
 
someFn = do let p = Page {title = "blah", content = "ack", _id = "page1"}
            PageModel.insert p False

Is the Page type the one from PageModel or from CommentModel? Both models export the Page type.

Update: As Twan points out in the Comments, the above would work if I were happy to do either of:

  let p = PageModel.Page {title = "blah", content = "ack", _id = "page1"}
  let p = CommentModel.Page {title = "blah", content = "ack", _id = "page1"}

Perhaps it’s unreasonable of me, but I want to be able to refer to the type without being forced to refer to the module. Besides, this post is really a gripe about import cycles, so I can’t have Fix #1 work and still gripe…

Fix #2

Push the data types out to a Types.hs file, but don’t export Types from the Models. Now you have to explicitly import Types whenever you’re using a Model:

import App.Models.Types   -- I'm not pretty...
import qualified App.Models.PageModel as PageModel
import qualified App.Models.CommentModel as CommentModel
 
someFn = do let p = Page {title = "blah", content = "ack", _id = "page1"}
            PageModel.insert p False

Now Page is clearly taken from App.Models.Types. But my delicate aesthetic sensibilities are offended.

But wait! We’re still broken! Since PageModel and CommentModel may use functions from each other, import cycles can still be produced. For example, Page.findAllChildComment uses a function from CommentModel:

instance HasChildComment (Page) where
    findAllChildComment p = CommentModel.findAllWhere "page_id = ?" [HDBC.toSql $ _id p]

Now if CommentModel needs a function from PageModel, then an import cycle is formed. So for the ORM to work relations in the DB can only form an acyclic graph. That’s bad.

Fix #3

Even though PageModel depends on CommentModel, don’t import it. Just duplicate the CommentModel code into PageModel. Ugly:

import App.Models.Types
 
instance HasChildComment (Page) where
    findAllChildComment p = do
        conn <- getEnvironment >>= (return . fromJust . getDatabase )
        res <- liftIO $ HDBC.handleSqlError $ HDBC.quickQuery' conn ("SELECT _id , commenter , page_id , post FROM comment WHERE (page_id = ?) ")  [_id p]
        return $ map (\r -> Comment (HDBC.fromSql (r !! 0)) (HDBC.fromSql (r !! 1)) (HDBC.fromSql (r !! 2)) (HDBC.fromSql (r !! 3))) res

I’m Sad

Haskell is such a lovely language, but my workaround for my module import cycles is pretty hideous. Am I missing something?

with 15 comments

2009: The Year Of Hackage

Written by alson

January 1st, 2009 at 10:24 pm

Posted in Haskell

Bookmark and Share

Haskell (even with the GHC extensions) seems pretty stable, but the greater GHC platform seems pretty unstable. I’ve already written about how we’re [sorta] missing a vital part of the tool chain (it was broken when I tried to build it using 6.10 the other day), but in this post I’ll focus on Haskell’s libraries (especially user generated ones).

After fairly heavy usage of Haskell over the past few months, I exit 2008 thinking that, while Haskell is a wonderful language, the library situation with GHC prevents GHC from being more heavily used. Unless you’re writing completely custom software, it can be very difficult to try a piece of Haskell software. One of the reasons Rails is so successful is that it Just Installs and the tutorials are Just That Easy. OTOH, when I start to install a new package in GHC, I never quite know what will happen.

Here are some problems I’ve had with Haskell’s libraries over the past few months/years along with possible fixes for those issues:

  • Hackage != Haskell Library Repository: Hackage doesn’t necessarily contain the latest and greatest versions of a library. For example, harp on Hackage is V0.2, but harp on code.haskell.org is V0.4.
    • This could most easily be fixed by automating the pulling of code from repos. Since harp is located at code.haskell.org/hsp/harp, Hackage could automatically pull code and build packages from there.
    • The Hackage team has put together a great system. It seems fair that they have T.O.S. which keeps Hackage usable and clean. Part of the TOS would be that a package maintainer: keep the package up to date in Hackage; respond to e-mails; notify the Hackage team if the maintainer is abandoning the package.
  • 6.10 ‘base’ Library Reorg: the base library reorg looks good, but it changes around a lot of the … er … basic libraries of the system.
    • Not sure of a fix here. Seems like a nice reorg and if base weren’t reorged, I’d probably be bitching about the need to reorg base… Here’s to hoping we don’t go through another base reorg any time soon.
  • Bit-rotted Libraries: Libraries exist in Hackage which won’t build and which will probably never build. For example, HaskellDB is an awe-inspiring library… that hasn’t built since 6.6 and just isn’t going to build any time soon. As a newbie looking for a database library, HaskellDB would be the obvious first place to look and trying to use it would produce serious frustration (it certainly did for me). It’s nice to have a historical reference to HaskellDB in Hackage, but the package shouldn’t be listed as currently usable. Maybe it’d be sufficient to default to filtering by the latest GHC version.
    • Given Don’s post, it might not be too difficult to script weekly library builds and auto-email library maintainers about their library being broken. After a few missed e-mails, the library could be shifted to another maintainer or, at worst, deprecated.
    • Gripe about git as much as you like, but GitHub has done a great job of building an environment for ‘social coding’. Using something like GitHub would make it easy to: allow those-who-want-to-work-on-a-library to do so; move a package to a new library maintainer; provide a consistent location from which Hackage could pull code. FWIW, Rails has gone this direction…

Summary

Haskell is an awesome language. I’m a big fan (as evinced by Turbinado). Working with Haskell has forced me to think about coding in a new way and has seriously improved my coding ability. However, Haskell has also proved enormously frustrating as I try to put together a project which depends on many libraries.

A great way to increase the visibility and usage of Haskell is to make it easy to develop and build complex software in Haskell. Naturally, that requires a robust set of user libraries and that seems to require Hackage. So let’s make 2009 the Year of Hackage.

Updates

Over on reddit, Don suggests we make “2009: the year distros start supporting the platform”. enauv replies that “[GHC is] too radical and changes too quickly for platforms to truly support it.” I tend to agree with enauv, so much so that this post was originally titled “2009: GHC 6.10 LTS?” because GHC (the overall platform) does tend to change too quickly for distros to support it effectively. Maybe GHC needs to migrate GHC to Long Term Support and move bleeding edge stuff to GHC 7.0?


This post is written by a practicioner and not by an academic. I understand that Haskell is an ‘academic’ language, but Haskell has gotten popular enough that it’s no longer ‘academic’ and that’s the greatest compliment a language could receive.


How am I going to solve the library dependency problem in Turbinado? I’m going to bundle all libraries on which Turbinado depends into Turbinado. Not a solution I like, but it’s one that works…

with 6 comments

A Plea For “cabal install”

Written by alson

December 26th, 2008 at 4:43 pm

Posted in Haskell,Turbinado

Tagged with , , ,

Bookmark and Share

Updated per Ganesh’s comments.


Over here, Adolfo commented:

“Hi,I tried to do follow the example, but I couldn’t even install the packages, hsx and hs-plugins were impossible. I tried with cabal and manually, and neither of those worked . any suggestion, known issue with this packages?”

I’ve been busy adding features to Turbinado and haven’t circled back around to making sure that it’s easy to build, so I can claim a lot of the blame for the build problems.  Turns out to be really important that publicly released packages are easily buildable…

Thinking back, I have really struggled to build Turbinado… and I wrote Turbinado!  Turbinado depends on some particular bugfix-ish library releases (e.g. GSomething 0.6.1). With GHC 6.10, a bunch of libraries have broken or have changed so much that they badly break Turbinado. (I need to specify better the versions in turbinado.cabal.)

At times, I’ve considered bundling the dependencies into Turbinado so that building Turbinado would be easy, but that’s always felt like a cop-out. So I’m pleading for “cabal install”. Given Turbinado’s dependence on particular versions of libraries, I would love to able to do:

cabal install turbinado
  OR  (from /home/alson/turbinado)
cabal build

Cabal Install

Most casual users of Ruby, Python, Perl, Java, etc, know that those languages have automagic build/dependencies system (respectively, gem, eggs, CPAN, maven). The tools may be of varying quality, but many tutorials include something like “First, use GEM to install the package: gem install rails” and demonstrate just how simple it is to get a useful piece of software installed.

This is not the case in Haskell. I’d guess that no more than 5% of Haskellers know about the cabal command line tool and “cabal install”. On the other hand, I’d guess that 95% of novice Rubyers know about “gem install”. These automated build/dependency system are now critical to the success of languages. As a beginner in Ruby, I always knew that I could easily try out various libraries by using GEM to install bits of software. I’m now fairly experienced with Haskell and, partly because of that experience, I don’t believe that I can easily try out various Haskell libraries.

Niklas Broberg’s HSP is a great example of the challenge of building Haskell programs. HSP is very nicely separated into modular libraries which: makes it easy to apply pieces of HSP’s functionality to a program; makes it hard for a human (at least for me) to build any one part of HSP because each part depends on so many other parts. A build/dependency tool would make HSP much easier to build into existing programs.

The Plea

I love using Haskell and Haskell will only get better if more people are able to use it. IMO, a pre-condition to the growth of the language is a solid, easy to use build/dependency system. Cabal is that system for GHC and the cabal command line tool is a key part of that system.

Unfortunately, the cabal command line tool isn’t bundled with GHC, but … Please get it, build it, use it, report any bugs, compliment the Cabal team, etc. It’ll be a great help to the Haskell community.

darcs get http://darcs.haskell.org/cabal-install
cd cabal-install
sh bootstrap.sh

Update: Haskell Platform

Ganesh points out the Haskell Platform Proposal, so it looks as though there is a plan to incorporate the cabal command line tool. See the following:

http://www.haskell.org/haskellwiki/Haskell_Platform

http://www.haskell.org/haskellwiki/Haskell_Platform/FAQ


P.S. Anyone know if the cabal command line tool is going to make it into GHC?


Links to cabal install information:

http://hackage.haskell.org/trac/hackage/wiki/CabalInstall

http://hackage.haskell.org/cgi-bin/hackage-scripts/package/cabal-install

http://ghcmutterings.wordpress.com/2008/11/10/bootstrapping-cabal-install/

with 13 comments

Cyptol on Slashdot

Written by alson

December 26th, 2008 at 4:02 pm

Posted in Haskell

Bookmark and Share

First Haskell-ish post I’ve seen on Slashdot in a while:

Cryptol, Language of Cryptography, Now Available To the Public

Maybe I’m old school, but getting through the moderation on Slashdot seems like a big deal… Congrats to the Galois team!

without comments