Effective exploration is a vital component in the success of robotic applications in ocean and space exploration, environmental monitoring, and search and rescue tasks. This talk presents a novel formulation of exploration that permits optimality criteria and performance guarantees for robotic exploration tasks. We define the problem of exploration as a coverage problem on continuous (infinite-dimensional) spaces based on ergodic theory and derive control methods that satisfy various notions of optimality and robustness such as asymptotic coverage, set-invariance, time-optimality, and reachability in exploration tasks. Last, we demonstrate successful execution of the approach on a range robotic systems and present an outlook on novel directions.