New sources of large-scale visual data raise both opportunities and challenges for computer vision. For example, each of the nearly trillion photos on Facebook is an observation of what the world looked like at a particular point in time and space, and what a particular photographer was paying attention to. Meanwhile, low-cost wearable cameras (like GoPro) are entering the mainstream, allowing people to record and share their lives from a first-person, "egocentric" perspective. How can vision help people organize these (and other) vast but noisy datasets? What could mining these rich datasets reveal about ourselves and about the world in general? In this talk, I'll describe recent work investigating these questions, focusing on two lines of work on egocentric imagery as examples. The first is for consumer applications, where our goal is to develop automated classifiers to help categorize lifelogging images across several dimensions. The second is an interdisciplinary project using computer vision with wearable cameras to study parent-child interactions in order to better understand child learning. Despite the different goals, these applications share common themes of robustly recognizing image content in noisy, highly dynamic, unstructured imagery.