I went there about 10 years ago and loved it.
It had the typical central European roccocco architecture but was dilapidated still from years of communist rule. Much of that would be rectified now.
My advice is have look at flickr there are a few budapest groups and this will give you some ideas. But my prime piece of advice is walk the streets. The Castle District, Fishmans Bastion, the bridges across the Danube (espcially the chain bridge) , the Citadel, the Danube itself all provide great shooting - and you see many of these in typical shots of budapest. I also made a point of going to many out of the way places. I loved the run down beauty of the old Jewish quarter for example. Check out Gozdu Udvar - a series of joined courtyards inside several large apartment buildings. Very atmospheric - although now being restored as apartments I believe. Walk up some of the streets that run from the banks of the Danube to the stairs leading to the Castle District and Fishermans Bastion and you will get some great shots -see some of mine, linked below . Also check out some squares like Battyanyi Ter on the Buda side. Walk out of the Vienna Gate of the Castle District and along some of the streets back towards the Danube or the other way towards Moscow Ter (itself a rather ugly tram interchange). Back then the streets were lined with buildings still showing great damage from WW2 or possibly 1956. Very interesting part of the world. The people are generally frinedly (though the language is incomprehensible) but back then there were some scams for the unwary - taxi fares and some street crime. Not sure now.
Here are some black and white shots I took back then (using a Leica M4P and a Nikon 801s and scanned from prints) which I posted at photo net a few years ago. They are a bit washed out but give an idea of what you might see.
http://photo.net/leica-rangefinders-forum/00JyZE