You can delete an app you don't use or a song you don't like from your iDevice, but it will still be hanging around in the iCloud, wasting space, hoping you'll re-download it. Fortunately there is a way to permanently delete content through iTunes, even if it lacks the satisfaction that comes with defenestration.
Open iTunes and find that annoying Yoga for the Face app that never worked or that book you don't want anyone to know you read. Right click on it and choose Delete. Follow up by deleting the app from your iDevice and then syncing your iDevice.