There has also been a suicide bomb attack close to the port of the capital, Mogadishu, which injured at least seven.
Militant group al-Shabab said it was behind both incidents.
The UN-backed government has been battling against al-Shabab for control of the country for more than a decade.
The militants, who are allied to al-Qaeda, said that in Baidoa they were targeting tax collectors, who were holding a meeting at the restaurant, and soldiers.
They added that two soldiers died in the attack but officials said the victims were all civilians.
Police told the BBC that in Mogadishu officers opened fire on a vehicle after it failed to stop at a checkpoint.
The suicide bomber driving the car is said to have tried to hit a police post in front of the port but the security forces shot him and the vehicle exploded.
Two police officers and five passers-by were wounded.
“Metal debris fell all over us inside the port and we heard gunfire,” a port worker told Reuters.