Virtual worlds are becoming ever more popular and important for the information society allowing to meet “face-to-face” and at the same time be distributed across different places. This offers numerous possibilities of revolutionizing the way learning is realized over long distances and at a given location. But current uses of environments like Second Life make it very clear that there is a lack of interaction and learning concepts that are tailored to these kinds of collaborative environments resulting more or less in the replication of “always the same” this time in a virtual world. An example is a typical lecture that is now available as an in-world podcast. This chapter examines current state-of-the-art approaches of learning in and with virtual worlds in relation to the features of such environments and then proposes a research agenda tailored at making the learning experience truly interactive, collaborative, multi modal and situation- and context-aware.