It didn't take long for Prince Harry to get back on the polo field after his trip to the U.K., but things didn't go so smoothly ... cuz he ended up on the ground. The Duke of Sussex competed in a match at Santa Barbara Polo and Racquet Club over…