The story comes under the heading of Apocryphal Carols and is derived from the apocryphal Gospel of Pseudo-Matthew, which combines many earlier apocryphal Nativity traditions; however, in Pseudo-Matthew, the event takes place during the flight into Egypt, and the fruit tree is a palm tree (presumably a date palm) and not a cherry tree. In the apocryphal Gospel, Jesus has already been born and so Joseph’s truculence is unrelated to any dismay over Mary’s pregnancy, but has to do with an inability to reach the fruits of the palm and a concern over the family’s lack of water. [Note from Francis James Child – English and Scottish Popular Ballads]. The story also appears in the Coventry Mystery Plays of the 16th Century.
The story has long retained a fascination for generations of singers and collected versions have appeared in many places in the English-speaking world.
Note that the tune sung my Mrs Roberts is very similar to the chorus tune of versions of “The Virgin Unspotted”.