If you would like to stop Favoriting someone, go to the Tools menu by tapping the wrench icon in Live.  Go to the "Favorites" section where you will be able to swipe to reveal the remove option (iOS) or press and hold to reveal the remove option (Android).