Safety when photographing Washington DC during the wee hours

So I know that many people go to DC to take pictures as well as visit museums.

Unlike others, I plan to be up early and catch the sunrise for the National Mall and be around that area early (5-6AM) and later in the evening (9-10PM or later) for the night. I know that terrorism has created a lot of things like instilling fear into people.

Are there any safety problems (e.g. getting robbed or security guards telling me I can't be taking pictures during those times) that I should be aware of? I ask this since I got asked by a few people when I was part of staff taking pictures during a fairgrounds for their advertisement campaign. I haven't been asked in San Francisco so I assume it would be the same but, last I remembered, isn't Washington DC is a special district? I'm not sure if not only the usual street smarts but also rules apply or maybe they don't and people with large backpacks are deemed dangerous?

Update1: I plan to be there during the winter if that makes any difference in the next few weeks or so. I will have a large backpack and also tripod if that helps. I will be taking pictures both during the weekday and weekend and be in and out of the area.