Knowledgebase : Data Processing

We had a major server issue in early 2022, which brought down all HIPE services. Herschel services are now supported on a best-efforts basis by a small number of Herschel veterans, who are working full-time on other missions and respond to issues with Herschel infrastructure as best as they can, albeit with very limited resources. Various volunteers investigated the issues and proposed a solution that has, after not inconsiderable efforts, been implemented by ESA’s Science and Operations IT Support Team. The implementation has been tested, and the system made more robust against future failures. We have also taken advantage to update some of the links in the HIPE documentation pages so that they too are more robust.

HIPE should now be fully functional and can be installed correctly. Nevertheless, if you find any issues, please let us know. This fix has been tested within the limits of our limited capabilities, but the best test of all is to have users working with Herschel data.


Extended sources provide the user with a particular set of problems that require special techniques.

We assume that you have watched the Herschel Science Centre short videotutorial called Starting with PACS Spectroscopy:

https://www.youtube.com/watch?v=rcY487Z9Kyg 

You should also watch the Herschel Science Centre short videotutorial called PACS Cubes in Detail:

https://www.youtube.com/watch?v=CxZF9-9D8UM

Or, at least, are familiar with the contents of these short videotutorials and their associated documentation. There is a Herschel Science Centre short videotutorial that will guide you through the process of processing and analyzing extended sources. You can find the short videotutorial here:

https://www.youtube.com/watch?v=j270VI0Ah1g

A pre-requisite for processing extended sources is to be familiar with the different spectroscopy products: cubes and tables that are produced when processing PACS spectroscopy.
Here is how to specify a temporary directory different from the default /tmp one. Put the following line

java.vm.options = -Djava.io.tmpdir=/mytmp

in your /.hcss/user.props, /installed.properties or in your /.hcss/appname.props, where (in the latter) appname is a application specific file (e.g. hipe.props).

Or you can start hipe with

hipe -properties java.io.tmpdir=/mytmp

Answer supplied by Álvar García:

HIPE user installers do not work on Mac Operating System from Mac OS X 10.8 (Mountain Lion) onwards due to additional security checks first implemented in this operating system version. 

There are two workarounds: 1. In a terminal window, use the chmod u+x command to change the permissions of the file HIPE_9_1_0_INSTALLER.app/Contents/MacOS/HIPE_9_1_0_INSTALLER.

Change the file name according to the HIPE version you are installing. Unless there is a specific reason not to, you should use the latest HIPE version (15.0). 2. In Mac OS X, from 'System Preferences --> Security & Privacy --> General --> Allow applications downloads from' select 'ANYWHERE'.

If you have installation problems other than the one reported here, please consider installing instead Virtual HIPE, which runs a Debian Linux installation of HIPE on a VirtualBox Virtual Machine. This should run on any OS/Platform combination, present, or future. You can find full download instructions at: https://www.cosmos.esa.int/documents/12133/1028864/Virtual+HIPE+Quick+Start+Guide

The Herschel Science Centre has also produced a useful short videotutorial that shows you how to install and configure Virtual HIPE. You can find the videotutorial here:

https://www.youtube.com/watch?v=9VqNT1QkpVQ


There are two, well-known sources of potential problems:

First, check that you are using the secure "https" protocol and not "http". If you have the latter, the link will fail.

Second, it is possible that the script is searching to the wrong directory because, unfortunately there were some changes to the URL where the on-line kernels were kept that were not within the control of the Herschel Science Centre. In order to fix the directory error, you have to edit the PhotometryConvoloveResolutionKernel.py script and change this line (line #424):

kernelRoot = '
http://www.astro.princeton.edu/~ganiano/Kernels/Ker_2012_May/Kernels_fits_Files/Hi_Resolution/
'

With this line:

kernelRoot = '
https://www.astro.princeton.edu/~ganiano/Kernels/Ker_2012/Kernels_fits_Files/Hi_Resolution/
'

This should resolve both issues.

There are two, well-known sources of potential problems:

First, check that you are using the secure "https" protocol and not "http". If you have the latter, the link will fail.

Second, it is possible that the script is searching to the wrong directory because, unfortunately there were some changes to the URL where the on-line kernels were kept that were not within the control of the Herschel Science Centre. In order to fix the directory error, you have to edit the SpectroscopyConvolveResolutionKernel.py script and change this line:

kernelRoot = '
http://www.astro.princeton.edu/~ganiano/Kernels/Ker_2012_May/Kernels_fits_Files/Hi_Resolution/
'

With this line:

kernelRoot = '
https://www.astro.princeton.edu/~ganiano/Kernels/Ker_2012/Kernels_fits_Files/Hi_Resolution/
'

This should resolve both issues.


Even though most Herschel data is now processed to the highest degree that is possible with the automatic processing pipeline and the results cannot be improved significantly, there are still datasets for which human intervention, particularly in post-processing, can improve the result. Many videotutorials are directed at users of Herschel data that benefit from the additional analysis and post-processing that HIPE can offer.

You can find a full list of videotutorials, sorted by instrument, at this URL:

https://www.cosmos.esa.int/web/herschel/legacy-training-materials

If you wish to learn about HIPE and how to install it on your computer, you should check this videotutorial on how to  install HIPE and how to import and to inspect data products:

https://www.youtube.com/watch?v=H9vof8iyqSc