We derive an explicit map between the singlet sector of the free and critical O(N) and U(N) vector models in any spacetime dimension above two and to all orders in 1/N, and a bulk higher spin theory in anti-de Sitter space in one higher dimension. For the boundary theory, we use the bilocal formalism of Jevicki et al to restrict to the singlet sector of the vector model. The bulk theory is defined from the boundary theory via our mapping and is a consistent quantum higher spin theory with a well defined action. Our mapping relates bilocal operators in the boundary theory to higher spin fields in the bulk, while single trace local operators in the boundary theory are related to boundary values of higher spin fields.