Overall, the city is poor, but seems rather safe. I suggest taking taxis around at night and it definitely helps to speak French, however I never felt threatened and people were always willing to help.
There are many historical sites in town and the French restaurants were quite good and very affordable.
Some interesting places to visit if you want to see the old Malagasy, British and French history and also captive native animals at the Zoo.
City is crowded with 3.4 million desperate people living on $2 per day, so hang on to everything you treasure. A fine example of what North Korean influence can do to what was one of Africa's greatest cities 70 years ago.