Ferry system from St Petersburg to Tallin and Helsinki

The St Peter Line appears to be the only ferry operator serving those routes. They run two sailings per week to Helsinki (15 hours) and one sailing per week to Tallinn (18 hours). They also operate a service from Helsinki to Tallinn and serve Stockholm (Full schedule could be found here).

The rest of your question is really a matter of opinion. Google suggests the driving time from St Petersburg to Tallinn is between 7 and 8 hours. You'd have to judge whether you think an 18hr ferry crossing is a better option than an 8hr drive with a Russian border crossing. I haven't looked at options for flying, nor have I attempted to cost this.

As to which destination: do your research, make a choice. Unless you have a specific objective question it's probably off-topic here.

@Airsick is completely right - St. Peter Line is the only operating ferry company for now in Saint-Petersburg. The reason that the route to Tallinn is 18 hours long is very simple - they do not go right to it, they have a stop in Helsinki. So you still need to go to Helsinki to use a ferry (and St Peter Line has not the best of them, personally I prefer the Tallink Silja, but other options are Viking Line and Eckerö Line).

Note that you can get from Saint-Petersburg to either Helsinki or Tallinn not only by ferry, but by train too - there are two train routes to Helsinki (Allegro - 3.5 hours in high-speed train to go, and Lev Tolstoy - about 9 hours to go), and one to Tallinn directly (Baltic Express - 8 hours to go). Also there are a bus routes from Saint-Petersburg to Tallinn and Helsinki, but it's not comfortable at all, and I do not recommend it, but it's cheap :)

As the last point of your trip is Warsaw, I suggest you to visit Tallinn and spend more time there, however, this is a primary-opinion based advice :) You may also choose to see both cities, starting from Helsinki, with Allegro speed you'll save the time, and a ferry route from there to Tallinn is about 2-3 hours, so you basically can gather them all!

As was mentioned before, ferries take 15 and 18 hours to Helsinki and Tallinn respectively.

For this reason, it makes more sense to take a train. I am Russian myself, from St. Petersburg. I took a trip through Scandinavia last year, starting in Denmark, then Norway, Sweden, and finishing in Helsinki. Then I took Helsinki St. Petersburg train (Allegro) and it was the best decision I could make. I was home in 3.5 hours. Plus, the train station is in the city center, so it works perfectly.

It takes 7 hours to get to Tallinn from St. Petersburg by train though, but it's still not 18!