A recent area of interest in machine learning involves drawing inferences from a large number of agents, each with some partial information. Often, these tasks must be accomplished in a distributed setting and in the face of scarce resources (time, bandwidth, and power). This talk describes some of our work on two problems in this area that use ideas from kernel methods and graphical models: (i) aggregating probability forecasts; and (ii) an approach for collaborative regression.