There are several possible reasons why your data may have bad pointing.

The first thing to check is that the coordinates that were given for the target were correct: on occasions, mistakes were made when typing the coordinates for targets in the original proposal for submission, or the far-IR coordinates for the target were quite uncertain when entered and may be significantly off from the true position of the source. If this is the case the QC information for the observation will usually have a note stating that it is so.

For Solar System Objects it is possible that the ephemeris that was used for planning the observations had a significant error. For fast-moving targets and also for objects with poorly-known orbit at the time of observation (this applies in particular to some Trans-Neptunian Objects) this effect may be important and, in extreme cases, can reach some tens of arcseconds. When observations of Solar System Objects were re-processed with the latest HIPE version it was done with the latest available ephemeris, which was the best available and may be significantly different to the one used to obtain the observations.

A second possibility is that this is a day when the Startracker switched from STR-1 to the back-up STR-2. There was an offset between the two, giving a systematic pointing error. This happened for all or part of ODs 161, 585, 586, 732 and 733, plus various occasions during the Commissioning and PV phases of the mission; the offset may affect all or part of an OD.

A third possibility is that there was a Spacecraft Velocity Vector re-set that affected part of the OD. A list of ODs affected by SVV re-sets and Startracker switch-overs and some practical advice for these cases is given in http://herschel.esac.esa.int/twiki/bin/view/Public/StrSwitchOversAndSvvResets. Software to correct for the effects of SVV re-sets and STR switch-overs was implemented in HIPE 13. You can resolve these problems by re-processing with HIPE 13, or by retrieving the latest products, processed with HIPE 15, from the Herschel Science Archive (HSA).

Another reason for poor pointing, although normally with somewhat smaller offsets than the effects noted above, is that it may also be caused by a pointing to a Solar Aspect Angle greater than 110 degrees. Heating of the base of the Service Module at such large SAAs caused a progressive pointing offset until the spacecraft had cooled again. On occasion it was necessary to hold the spacecraft at SAA>110 degrees for 2 ODs for instrument maintenance. The QC summary will indicate if this has happened. Normally only pointing-robust observations (maps) were scheduled at or after prolonged excursions to large SAAs. There is a technical note on the thermo-elastic effects caused by pointings at large SAAs available at: https://www.cosmos.esa.int/documents/12133/1028864/Astrometry+correction+for+observations+affected+by+a+thermoelastic+drift; this document also explains how such effects can be corrected by using a special script in HIPE.

Bear in mind that pointing errors are statistical and that in early mission phases a random 3-sigma error in the pointing can mean an off-set of around 8 arcsec without anything actually being explicitly wrong with the observation, or with the pointing.

If you have PACS Spectroscopy observations, there is a script for treating pointing off-sets that corrects the flux for the offset. The Herschel Science Centre has produced a useful short videotutorial that shows you how to use the Pointing Offset Correction Script. You can find it here:

https://www.youtube.com/watch?v=f861D8CoYqc