In 1974, Baker took over the role of the Doctor from Jon Pertwee to become the Fourth Doctor in the BBC TV series. He was recommended to producer Barry Letts by the BBC's Head of Serials, Bill Slater, who had directed Baker in a Play of the Month production of Shaw's play The Millionairess. Impressed by Baker upon meeting him, Letts then became convinced he was right for the part after seeing his performance in The Golden Voyage of Sinbad. Baker was working on a construction site at the time, as acting jobs were scarce. Initially he was dubbed "Boiler Suit Tom" by the media because he had been supplied for a press conference with some old studio set clothes to replace his modest garments.
Baker quickly made the part his own, viewing figures for his first few years returning to a level not seen since the height of 'Dalekmania' a decade earlier. His eccentric style of dress and quirky personality (particularly his trademark long scarf and fondness for jelly babies), as well as his distinctive voice, made him an immediately recognisable figure and he quickly caught the viewing public's imagination. Baker played the Doctor for seven consecutive series, making him the longest-serving actor in the part. Baker himself suggested many aspects of his Doctor's personality, but the distinctive scarf was created by accident. James Acheson, the costume designer assigned to his first story, had provided far more wool than was necessary to the knitter, Begonia Pope, intending for her to choose a suitable colour. However, due to miscommunication Pope knitted all the wool she was given. It was Baker who suggested that he wear the ridiculously long scarf, which he did once it had been shortened a bit to make it more manageable.
The Doctor played by Baker is often regarded as the most popular of the Doctors. In polls conducted by Doctor Who Magazine, Baker has lost the "Best Doctor" category only three times: once to Sylvester McCoy in 1990, and twice to David Tennant in 2006 and 2009. Many of the stories from his era are considered to be classics of the series, including The Ark in Space, Genesis of the Daleks, The Brain of Morbius, The Deadly Assassin and The Robots of Death. However, the violent tone of the stories produced by Letts' successor, Philip Hinchcliffe, saw the series come under heavy criticism from morality campaigner Mary Whitehouse. Concerns over violence during this early period led to a lightening of the tone and an "erratic decline" in both the popularity and quality of the series. Baker has described Hinchcliffe as "amazing" and identified that as his favourite period of his time on the series. He described Hinchcliffe's successor, Graham Williams, as "absolutely devoted" but lacking Hinchcliffe's flair. He has acknowledged that his final producer on the series, John Nathan-Turner, made changes he did not agree with and they "did not see eye-to-eye really about very much". He said they became good friends afterwards and forgot their disagreements. Baker suggested that he may have stayed in the role for one series too many.
Baker continues to be associated with the Doctor, appearing on documentaries such as The Story of Doctor Who and Doctor Who Confidential and giving interviews about his time on the programme. He reappeared as the Doctor for the 1993 charity special Dimensions in Time and audio for the PC game Doctor Who: Destiny of the Doctors. In 1996 he appraised his time on the show as the highlight of his life. He is often interviewed as part of documentaries on the extras of Doctor Who DVD releases from his era as the Doctor and has recorded DVD commentaries for many of the stories. In a 2004 interview regarding the series' revival, Baker suggested that he be cast as the Master. In a 2010 interview, Baker said that he had not watched Tennant's performance as the Doctor but thought his Hamlet was excellent.
While Peter Davison, Colin Baker, Sylvester McCoy, and Paul McGann have all reprised their roles for audio adventures produced since the 1990s by Big Finish (and sometimes the BBC), Baker declined to voice the Doctor until 2009, saying that he had not seen a script he liked. In July 2009, the BBC announced that Baker would return to the role for a series of five audio dramas, co-starring Richard Franklin as Captain Mike Yates, which would begin release in September. The five audios comprise a single linked story under the banner title Hornets' Nest, written by author Paul Magrs. He returned with a sequel to Hornets' Nest called Demon Quest. Baker has also filmed inserts for a video release of the unfinished Shada in 1992, presented the video release The Tom Baker Years (1992), the latter a look back at his time on the series watching short clips from his episodes and also provided narration for several BBC audio releases of old Doctor Who stories.
In March 2011, it was announced that Baker would be returning as the Fourth Doctor initially for two series of plays for Big Finish Productions, starring alongside former companions Leela (Louise Jameson) and Romana I (Mary Tamm). The first series of six audios were released starting from January 2012. Big Finish had also arranged for Baker to record a series of stories reuniting him with Elisabeth Sladen's character Sarah Jane Smith (for which special permission was obtained from the producers of The Sarah Jane Adventures TV series), but Sladen died in April 2011 before any stories could be recorded. Baker recently recorded several Big Finish audio stories with Matthew Waterhouse, who played Adric, and Lalla Ward, who played Romana II (though Ward recorded her sections separately).
Baker has been involved in the reading of old Target novelisations in the BBC Audio range of talking books, "Doctor Who (Classic Novels)". Doctor Who and the Giant Robot was the first release in the range read by Baker, released on 5 November 2007, followed by Baker reading Doctor Who and the Brain of Morbius (released 4 February 2008), Doctor Who and the Creature from the Pit (released on 7 April 2008) and Doctor Who and the Pyramids of Mars (released 14 August 2008). In October 2009, Baker was interviewed for BBC Radio 4's Last Word to pay tribute to deceased former Doctor Who producer Barry Letts. He described Letts, who originally cast him in the role, as "the big link in changing my entire life".
On 20 November 2013, Baker revealed that he would appear in the 50th anniversary special, The Day of the Doctor, stating, "I am in the special. I'm not supposed to tell you that, but I tell you that very willingly and specifically; the BBC told me not to tell anybody but I'm telling you straight away." The episode saw Baker in the role of a mysterious curator in the National Gallery who openly discusses his resemblance to the Fourth Doctor with the Eleventh Doctor.
In November 2017, Baker made a return to the role of the Doctor, completing an episode originally begun in 1979 but abandoned due to strike action. The story – Shada, written by Douglas Adams – was filmed in Cambridge. Animation was added to complete the original story. He also filmed one new scene for inclusion in the final episode.