Columbia refers to places and institutions in North America while Colombia is the proper name for the country in South America. Colombia, the gateway to the South America. The continent's 5th largest country. The name was a reference to the New World and the explorer who had reached the northern coast of what is now Venezuela during a later voyage.
Colombia comes from the name of the Spanish explorer Cristopher Columbus. Columbus was his English name. But since he was of Italian origin his name, in Italian, was Cristoforo Colombo. And his name in Spanish was Cristobal Colon.
The country of Colombia achieved its independence from Spain in 1819. The country became the United States of Colombia in 1863 and the Republic of Colombia in 1886.
Spain is Espana in Spanish, Italy is Italia, Brazil is Brasil, France is Francia, Leghorn is Livonia - anyway you get the idea. Columbus is just the anglicized version substituting a 'u' for an 'o'.
And the North American continent is awash with places and things named after Columbus.
A river running through Canada and Washington State USA
A sportswear brand
A state in Canada's - British Columbia
Washington D.C. - District of Colombia - the capital of the USA
The capital of South Carolina a state in the USA
Columbia University in Chicago
A film company - Colombia Pictures
A record company - Colombia Records
A NASA space shuttle
So that's the difference between the two spellings? Colombia is the correct spelling of the country. And if you are writing about the country - spell check is not your friend here. Be sure to always check your spelling because -