Cascalog + Hadoop Counters, Finally!

I've just merged a Cascalog pull request of mine that gives Cascalog operations access to the statistics that Cascading generates at the end of each job. I've also added global inc! and inc-by! functions that let you increment custom Hadoop counters from within your functions and operations without having to »

Wyoming Rando Roundup, Pt 2

This is my race report from the 2015 Jackson Hole skimo race, the second of two ski mountaineering races I completed this weekend. (Here's my report from yesterday's race at Grand Targhee.) Jackson was going to be bigger and badder than Targhee, no question. I knew that I'd be hucking »

5 Books for Endurance Addicts

This post is about some of the books related to endurance training and sports that have inspired me over the last couple of years. Training for endurance events changes you. At the beginning of a season, on my first long run, ten minutes in I feel anxiety creep in. How »

Cascalog 2.0 In Depth

Cascalog 2.0 has been out for over a year now, and outside of a post to the mailing list and a talk at Clojure/Conj 2013 (slides here), I've never written up the startingly long list of new features brought by that release. So shameful. This post fixes that. »

SkiMo Season is Here!

(Jason Antin cranking up A Basin in our second Skimo race of the year. Photo by Camera Photography.) My running season ended in September with the Wasatch Front 100, and I resigned myself to a winter of weight lifting and sporadic winter trail running. It takes a LOT of motivation »

API Authentication with Liberator and Friend

I've just finished rewriting a number of PaddleGuru's internal APIs using two great open-source libraries; Liberator and Friend. Liberator is a library for writing RESTful resources in Clojure. Friend is an authorization and authentication library written by the prolific Chas Emerick, Dominator, Esquire. You've certainly seen his stuff around if »

Upcoming Talks in 2013

This is the year I teach myself to become a better public speaker. I've spent the past year coding up a number of powerful Scala and Clojure projects, all the while avoiding the important and difficult work of teaching and writing about the import and use of those projects. Well, »

Cascalog Testing 2.0

A few months ago I announced Midje-Cascalog, my layer of Midje testing macros over the Cascalog MapReduce DSL. These allow you to write tests for your Cascalog jobs in a style that mimics Cascalog's own query execution syntax. In this post I discuss midje-cascalog's 0.4.0 release, which brings »

Introducing Cascalog-Contrib

I've had the pleasure of working with Cascalog for about ten months now, and have seen the community produce some fantastic work. A number of businesses are using Cascalog in production; I use Cascalog at Twitter every day to write MapReduce queries for the new Twitter Web Analytics product. One »

Testing Cascalog with Midje

I've been working on a Cascalog testing suite these past few weeks, an extension to Brian Marick's Midje, that eases much of the pain of testing MapReduce workflows. I think a lot of the dull work we see in the Hadoop community is a direct result of fear. Without proper »

No more posts