Ь  The “soft sign” (мягкий знак - "myagkii znak") has no sound value, is used to modify the pronunciation of the preceding consonant by making it soft (palatalized). A consonant becomes soft when it is pronounced with the middle of the tongue raised towards the roof of the mouth - in the word “конь” (stallion, male horse), compared to “кон” (round, as in the next round of a game).

Ъ  The “hard sign” (твёрдый знак - "tvyordyi znak") has no sound value. It occurs only between a consonant and a vowel as in the word “въезд” (entry). It is used to show that the consonant should not be palatalized and that the consequent vowel is preceded by the the “y” sound as in yes.

A long time ago, the “soft sign” (ь) and the “hard sign” (ъ) were both short vowels, pronounced something like the /i/ in “pit” and the /u/ in “put.

The trouble was that in time ь and ъ lost their vocalic quality, i.e., they stopped being pronounced as vowels and no longer formed syllables. The ь and ъ continued to be written in all Russian texts, but now they were used as “phonetic markers” - the ь indicated that the preceding consonant was “soft” (palatalized), while the ъ indicated that the preceding consonant was “hard” (non-palatalized).

More information can be found here.