The building is separated from the Victoria Embankment by a greenspace that in Tudor times was occupied by Whitehall Palace.

The building was proposed in 1909 and designed in 1915 by E. Vincent Harris.  However, its construction was delayed by two World Wars and the depression of the 1930s.  Construction began in 1939 but it wasn't until 1951 that the Board of Trade moved into the northern section of the building.  In 1958/59, the Air Ministry occupied the Richmond Terrace end of the building

The design called for a building that was 205 feet wide at the Richmond Terrace end and 300 feet wide on Horse Guards Avenue.  It had three internal courtyards that were originally open to the sky but have now been fitted with glazed roofs. 

In 1964, when the three armed services were merged and the Ministry of Defence was created, there was a need for a large building for its home.  It was at this point that the Board of Trade moved out and it became the MoD Main Building.

The MoD say that in the mid 1990s they concluded that," ... the building had suffered cumulative decline, was expensive to maintain and had severe Health & Safety shortcomings. Having never had a major refurbishment, the environment was no longer consistent with a modern business headquarters. ...  Following a three-year decant period, the MoD moved 3,150 staff back into the redeveloped building. This phased re-occupation was completed in September 2004, bringing the programme to a conclusion on budget and ahead of schedule. ... The redeveloped workspace provides desks for all people and also includes shared areas with breakout spaces, quiet rooms and informal meeting areas. New facilities such as the café, restaurant, gym, information centre, conference suite and business suite have also been introduced.  This new, more open plan environment has increased the capacity of the building by some 700 desks, and has allowed for the disposal of five other London sites. The new environment is better able to support interaction, communication and collaboration in a more team-based, less hierarchical workspace."

The Horse Guards Avenue entrance features sculptures by Sir Charles Wheeler entitled 'Earth' and 'Water'.  Apparently the plan was to add two more called 'Air' and 'Fire' at the other end but that didn't happen.

There are a number of statues in the garden on the Victoria Embankment side of the building including this one dedicated to the Fleet Air Arm by the sculptor James Butler.  It was erected as a tribute to the more than 6,000 individuals who lost their lives in the Royal Navy Air Service since World War One.

The statue below, of General Gordon, who was killed at the siege of Khartoum in 1885, used to sit in Trafalgar Square but was moved to the Victoria Embankment gardens in 1953.  It is attributed to Sir Hamo Thornycroft and was created in 1887.