Milburn played Matthew Bannerman in Families and Liam in Green Wing. He has also been in Me Without You, The Bill, Backup, Tess of the D'Urbervilles, David Copperfield (as James Steerforth), Sweet Medicine, Byron, Born and Bred, The Forsyte Saga: To Let and Bodies. Milburn also joined the cast of Mistresses in 2009. In 2011, he played the role of Edgar Linton in the film adaptation of Emily Brontë's Wuthering Heights. In 2013, he lent his voice to the characters of Bartholomew Roberts and John in the video game Assassin's Creed IV: Black Flag.
Born in Dorset, Milburn was educated at the Dragon School in Oxford, and then Eton College. He is married to the Channel 4 news reporter Katie Razzall, daughter of Liberal Democrat peer Tim Razzall, Baron Razzall. They have a daughter, Matilda. The Milburns were honeymooning in Sri Lanka when the 2004 Boxing Day tsunami struck; Katie went on to file news reports on the events.