Since their discovery in 1932 by Chadwick, neutrons have been a critical probe in physics and sciences in general. It was Fermi who first realized that neutrons which are traveling slow enough would be totally reflected by a material surface. These slow moving neutrons can be trapped in material bottles and are called ultracold neutrons (UCN). Experiments performed with UCN take advantage of their slow velocities and long trapping time. Currently, UCN hold the world leading sensitivity of the neutron lifetime, a parameter critical to our understanding of the Weak force and Big Bang Nucleosynthesis. UCN are competitive in measuring decay correlations of the neutron and constraining tensor interactions. They are also used to search for Dark Matter and other exotic particles and interactions. Furthermore, UCN are responsible for the world limit on the sensitivity of the permanent neutron electric dipole moment (nEDM), a T and P symmetry violating observable. By the CPT theorem EDMs also violate CP. The Sakharov conditions require a new CP violating interaction for the observed dominance of matter over anti-matter. Thus, a new source of CP violation is expected, and is perhaps mediated by particles beyond the Standard Model. The present limit, 3×10⁻²⁶ e cm, already has a reach for new CP violating physics generically at the TeV, and up to the PeV scale in some specific supersymmetric models. Future experiments plan to increase the sensitivity by up to two orders of magnitude. The goal sensitivities of these experiments are challenging targets, and require fascinating technological achievements for their success.