While this is a very subjective question I tend to lean on the side of no. While I do believe that a setup cost should be tallied for all orders, I think there are variables that either entice or negate the need or desire for setup fees.
These variables can include the client, order size, order difficulty, client relationship, the project, and a host of other factors. Please keep in mind setup fees are charged to help offset the costs of setting up and breaking down each print job. These costs are going to accrue and have an effect upon your bottom line regardless of whether you pass them on to your customer or not, so know the numbers and see how they will affect you before making a final decision.
Weigh your relationship, project the amount of future work you may receive, see how you feel about things, and make a financial determination.
Also, if you should decide to travel down that path, it could be challenging to try and change course later.