A loud noise was heard inside the building in the 100 block of Marconi Boulevard around 7:30 p.m. Tuesday.
Police asked people inside the building to evacuate by the stairs.
The Franklin County Jails, located on South Front Street and Jackson Pike, were placed on lockdown at about 9:30 p.m.
Police and fire crews were going building by building checking for the source.
Homeland security officials arrived on scene shortly after 8:30 p.m.
Columbus fire officials said there were no reports of injuries.
Columbus police dispatchers said the noise was heard on the second floor where a deli is located. Dispatchers said it did not start a fire.
An officer inside the building told 10TV that another officer smelled gunpowder at the time of the reported explosion.
There were also reports of suspicious people in the area at the time of the reported explosion, police said.
K9 units also were on scene, and officials were searching nearby parking garages for explosives.
Some streets surrounding police headquarters is closed.