Tillerson went to the embassy on Monday to greet US Ambassador to Britain Robert Wood Johnson. A planned meeting with embassy staffers was cancelled due to a government shutdown back home.
The top US diplomat will also meet with UK Prime Minister Theresa May and Foreign Secretary Boris Johnson later in the day.
The new embassy became a point of strain in US-UK relations earlier this month when Trump said he had cancelled plans to visit Britain and formally open the diplomatic facility in February.
Trump had complained that the embassy was too costly and was in a less desirable location than the old embassy.
In a tweet earlier this month, Trump criticized the administration of former US President Barack Obama for selling the previous embassy compound and building a new one on the south side of the River Thames.
The decision to move the embassy, however, was made by the administration of former President George W. Bush in 2008 after it determined the old facility in Grosvenor Square had uncorrectable security issues.
The $1 billion construction was funded by the sale of other US properties in London.
Trump has not yet visited Britain since taking office a year ago over fears of mass protests, and has been involved in rows with the government over issues including trade and his re-tweeting of an anti-Muslim video posted by a British far-right group.
London Mayor Sadiq Khan also said he "would not be welcome" in the city, suggesting there could be widespread protests.
May offered Trump a state visit to Britain one year ago, when she became the first foreign leader to visit the White House after his inauguration.
Last week, British Labor Party leader Jeremy Corbyn rejected the so-called “special relationship” between the US and the UK, saying that Washington is not the most important ally of London.
The special relationship is an unofficial term for the close political, diplomatic, economic, military and cultural relations between the United States and the United Kingdom. It was first used in a 1946 speech by former UK prime minister Winston Churchill.