Jan 22, 2008 / software ~ distributed
Distributed PIG

I have an unhealthy fascination with large-scale parellelism and distributed programming, despite the fact I've never had to work on anything truly involved with it.

Anyway, the PIG project looks like one to play with when the time comes. Described as:

"Pig is a platform for analyzing large data sets. It consists of a high-level language for expressing data analysis programs along with the infrastructure for evaluating these programs. The salient property of Pig programs is that their structure is amenable to substantial parallelization, which in turns enables them to handle very large data sets."

You may also like...