I would not worry about harm to the baby from the fever, then.
A fever is simply the body's way to cook out a bad bacteria or virus that you have picked up. Enduring it might be a bit unpleasant, but it has that useful purpose.
Don't worry about harm to the baby unless it hits 103 or 104, then you might take something, and if you do decide to take something call the doctor first to be sure it is OK to take during pregnancy.