Knowledgebase

To establish the SAMP connection between HSA and HIPE follow the steps given in the attached video:

1. Open the HSA Interface
2. Open the SAMP Hub
3. Open HIPE
4. Send data from HSA to HIPE

Note that HIPE and any SAMP connection opened by HIPE should be closed before starting the SAMP connection with the HSA interface. 


At the time of writing (late July, 2018), we are unaware of any OS/Platform combinations on which HIPE will not install or run, although we are aware that it is possible that it will start to fail on some future OS/Platorm combinations, or because of the progressive failure of servers.

The immense majority of HIPE issues are due to configuration problems. These should be notified to the Herschel Science Centre Helpdesk in the normal way and we will advise how to resolve them.

However, we have prepared a virtualized version of HIPE that will run on any current or future OS/Platform combination. This is based on a Linux installation that is known to work well.. You can find out how to install, configure and run Virtual HIPE in the following Herschel Science Centre short videotutorial:

https://www.youtube.com/watch?v=9VqNT1QkpVQ

Fringes, caused by standing waves, may be present in HIFI spectra. If present, and at a level that the user feels affects data quality, these have to be removed interactively in post-processing as there is no simple recipe for doing it automatically. There is a task in HIPE, fitHifiFringe that allows sine waves to be fitted to the spectra interactively by the user. There is a Herschel Science Centre short videotutorial that shows you how to carry out this process. You can find the short videotutorial here:

https://www.youtube.com/watch?v=Po4iGKf-0MA

The short videotutorial shows you the user interface for this task and talks you through the different parameters that must be set and what they mean, before a practical demonstration of running this process in HIPE using a HIFI Point Mode observation. You will be shown where to find the task, what parameters to set and the recommended values to set for them and the results that are obtained.

To find out about the steps to follow in reducing SPIRE spectroscopy and, in particular, how to carry background subtraction of spectra, there is a Herschel Science Centre short videotutorial that shows you the steps involved and also describes the data products that are available for which experts have carried out background reduction already. You can find the videotutorial here:

 https://www.youtube.com/watch?v=O_pqN4VDu9M

This videotutorial describes a useful interactive script for background reduction and shows a practical example of how to carry out background subtraction and the decisions involved in carrying out background subtraction interactively, identifying and eliminating interactively the outliers in background subtraction. It also gives you details of the available expert-reduced background subtracted spectra available as a Highly Processed Data Product.

As for SPIRE FTS spectroscopy, semi-extended sources provide the user with a very special set of problems, as they are sources of a size intermediate between point sources (i.e. unresolved) and extended sources (clearly resolved), for which neither point source reduction techniques, nor extended source techniques work well.

We assume that you have watched the Herschel Science Centre short videotutorial called Starting with PACS Spectroscopy:

https://www.youtube.com/watch?v=rcY487Z9Kyg 

Or, at least, are familiar with the contents of this short videotutorial and its associated documentation. There is a Herschel Science Centre short videotutorial that will guide you through the process of processing and analyzing semi-extended sources. You can find the short videotutorial here:

https://www.youtube.com/watch?v=PNSuxM105UY

A pre-requisite for processing semi-extended sources is to know:

1. The surface brightness distribution.

and

2. The off-set of the source from the centre of the central spaxel.

If you do not have this information, you can still attempt to use these techniques (the short videotutorial will advise you on this), but may have larger errors in your calibration. 

You can find a package of useful support materials for this short videotutorial in the Herschel Explanatory Legacy library (HELL) in the Level 2, Data reduction, PACS spectroscopy section.

This short videotutorial you will be shown how to use the useful script that was developed to deal with semi-extended sources that extracts the spectrum of the central spaxel of a re-binned cube and applies a point source correction to it. It then compares the surface brightness of a point source with the known surface brightness distribution and creates an extended-to-point correction too that is used to generate the semi-extended source correction. It will show you how to apply the models and which data products to use for different observing modes. Finally, you will be shown some examples of PACS data processed with the script.

We recommend, as background information, that you consult the following Herschel Science Centre short videotutorials too:

Point Source Spectroscopy with PACS, https://www.youtube.com/watch?v=nYWTccEQz_M

PACS Cubes in Detail, https://www.youtube.com/watch?v=CxZF9-9D8UM

Or the following guide:

Dealing with extended sources observed with PACS spectroscopy

Extended sources provide the user with a particular set of problems that require special techniques.

We assume that you have watched the Herschel Science Centre short videotutorial called Starting with PACS Spectroscopy:

https://www.youtube.com/watch?v=rcY487Z9Kyg 

You should also watch the Herschel Science Centre short videotutorial called PACS Cubes in Detail:

https://www.youtube.com/watch?v=CxZF9-9D8UM

Or, at least, are familiar with the contents of these short videotutorials and their associated documentation. There is a Herschel Science Centre short videotutorial that will guide you through the process of processing and analyzing extended sources. You can find the short videotutorial here:

https://www.youtube.com/watch?v=j270VI0Ah1g

A pre-requisite for processing extended sources is to be familiar with the different spectroscopy products: cubes and tables that are produced when processing PACS spectroscopy.

HIFI Level 2.5 (i.e. science ready) data are calibrated in an antenna temperature scale that is very specific to HIFI, making direct comparison of data difficult with other observatories. HIFI antenna temperature can be converted to a more physical scale, a process that is described in detail in the HIFI Handbook (see Section 5.8.6) and the HIFI Data Reduction Guide (see Section 18.2). There is a HIPE task called convertK2Jy that can do this conversion in post-processing.

The Herschel Science Centre has produced a short videotutorial that walks you through this process, You can find this short videotutorial here

https://www.youtube.com/watch?v=KK-hbAJbWIw

The short videotutorial will show the the GUI (Graphic User Interface) for the convertK2Jy task and will explain the different parameters that must be set and the recommended values for them. It will then show two examples of HIPE sessions with HIFI data: first for a HIFI Pont Mode observation and then, for a HIFI Mapping Mode observation.

The demonstration will show you how to modify the script to carry out the conversion and the results that are obtained froom it.



We assume here that you have watched the Herschel Science Centre short videotutorial called Starting with PACS Spectroscopy:

https://www.youtube.com/watch?v=rcY487Z9Kyg 

And the Herschel Science Centre short videotutorial Point Source Spectroscopy with PACS:

https://www.youtube.com/watch?v=nYWTccEQz_M

Or, at least, are familiar with the contents of both short videotutorials and their associated documentation. If you have/are, you may find that using the PACS Pointing Off-set Correction Script will improve the quality of you results. There is a Herschel Science Centre short videotutorial that will show you how to use it and teach you the background information that you need to make effective use of it. You can find the short videotutorial here:

https://www.youtube.com/watch?v=f861D8CoYqc

The Pointing Off-set Correction Script will improve the spectra of point sources that suffer from pointing off-sets, or pointing jitter, provided that the off-set is no greater than 10 arcseconds. The script measures the average pointing off-set and its variation with time (the jitter). The script works best for:

1. Sources with a flux of about 10Jy or more.

2. Sources that are off-set by no more than 4 arcseconds.

For these objects you will get a much improved continuum shape and will be able to detect broad features much more reliably.

In contrast, the script will not work for:
  • A source located at the edge of the central 3x3 spaxel box.
  • Un-chopped spectroscopy.
And it will be unreliable for faint sources.

You can find the Pointing Off-set Correction Script in the HIPE Scripts menu and must be run from Level 0: i.e. with raw data. It is fully described in the PACS Data Reduction Guide. It will produce two spectra from the extractCentralSpectrum task - the so-called c9 and the c129 spectra - of which, the one with the higher flux is the one to use. You will see several examples of spectra of sources with off-sets from 2-6 arcseconds that have been processed with and without the Pointing Off-set Correction Script to illustrate the differences that can be found.


HIFI spectra can be of two kinds: ON source spectra, in which the keyword "isLine=True) and OFF source, reference position spectra, for which isLine=FALSE. The reference position though may, itself, be contaminated by background emission, compromising the reduced on-source spectrum. The best way to eliminate or, at least, to assess this effect is to examine the OFF source spectra. The Herschel Science Centre has produced a short videotutorial that explains how to examine the OFF spectra. You can find this videotutorial here:

https://www.youtube.com/watch?v=Tpte_3LTL-s

OFF spectra can be different things for different observing modes. The short videotutorial will show you what the OFF spectrum represents for each mode and will take you through a practical example of how to examine and plot the OFF spectrum in the observation context. It will also show you the effects of line contamination in an OFF spectrum when it is used for background subtraction.

Both point source photometry - from source fitting - and aperture photometry can be extracted from SPIRE maps.

If you prefer to obtain aperture photometry from SPIRE maps, you should check the appropriate Herschel Science Centre short videotutorial for this method instead (https://www.youtube.com/watch?v=fjzf-VN_X0s).

Point source photometry can be obtained from SPIRE maps either for a single source, or by using a source detection routine, such as Sussextractor to give you a list of detected sources in a field as input to extract photometry for multiple sources.

There Herschel Science Centre has produced a short videotutorial on point source photometry, which reviews the methods of source extraction for single and multiple sources in a field, how to use the source extraction as input for photometry and which are the best photometric methods to use with your data.

You can find this short videotutorial here:

https://www.youtube.com/watch?v=nbN6OWtWIR0

The Herschel Explanatory Legacy Library - sometimes known by its acronym, HELL - contains over 1350 documents and resources from all phases of the Herschel mission from original concept to Archive Phase and is being constantly expanded and updated. The Documentation Catalogue contains complete catalogue information on all available documents and resources.

The catalogue itself is available in three formats:

Excel spreadsheet: http://herschel.esac.esa.int/twiki/pub/HSC/HerschelLegacyDocumentation/All_documentation.xlsx

Comma Separated Values (CSV): https://www.cosmos.esa.int/documents/12133/1028864/All+documentation+in+HELL+CSV+format

Portable Document Format (PDF): https://www.cosmos.esa.int/documents/12133/1028864/All+documentation+in+HELL+PDF+format

The "What's New" page - https://www.cosmos.esa.int/documents/12133/1028864/Recent+Updates+to+the+HELL+Pages - will tell you what new documents and features have been added.

There is a short videotutorial that offers a guide to the documentation catalogue and its contents and explains some simple tricks for making searches:

https://www.youtube.com/watch?v=fSDXVorlHb0

For users who need more detail and information about the Herschel Documentation Catalogue, there is a second videotutorial that explores it in a little more depth:

https://www.youtube.com/watch?v=8mNlhGgeTEY
There are several ways of accessing the Herschel short videotutorials.

A series of legacy videotutorials have been produced, with useful knowledge for current and future users of Herschel data and services. They cover:
  • Herschel data and the Herschel Science Archive.
  • Herschel legacy software.
  • Herschel legacy documentation and legacy documentation catalogues.
  • An overview of Herschel services for legacy users.
  • Herschel data and how to examine it.
  • How to search for specific types of Herschel data in the Herschel Science Archive (e.g. unpublished data, data that has been specially processed by an instrument expert, catalogues of sources, or of spectra, etc.)
  • Data analysis techniques specific to each Herschel instrument (e.g. tricks for carrying out photometry, for analyzing extended sources, for baseline fitting of spectra, for analyzing the spectra of sources that are not perfectly centred on the detector, etc.)
You can find a list of all the short legacy videotutorials, most of which are from 5-10 minutes long, on the Herschel Web Pages at:


The last legacy short videotutorials will be added early in 2019 and, after that, the contents of this list of short videotutorials will be frozen.

There is also a dedicated Herschel YouTube Channel called "Herschel Academy", with many interesting videotutorials on different aspects of data analysis and on the mission in general that were generated through the mission. You can find Herschel Academy at:


Do bear in mind that some of the older videos have, inevitably, become slightly out of date as the software to analyse some observing modes and types of observations was still evolving, even after the end of the mission, although these older videotutorials are still useful as background material. Always check to see if there is a newer video available on the topic that interests you to ensure that you have the most up-to-date information.


Fitting a baseline to HIFI spectra is an essential part of post-processing of HIFI data. There is a HIPE task called fitBaseline that allows a baseline to be fitted to your HIFI data interactively, subtracting, or dividing by a polynomial baseline. A Herschel Science Centre short videotutorial is available, which shows you the process of fitting a baseline to your data in post-processing, the parameters to set and how to set them. You can find the short videotutorial here;

https://www.youtube.com/watch?v=fP2ZNsSqvv8

The short videotutorial will show you how to run the task and will show you what the parameters that must be set are and what they mean. You will be guided through the process of fitting the baseline in HIPE with a HIFI Point Mode observation, how to mask the lines in the spectrum and how to iterate the process to test different fits until you obtain the one that most satisfies you.

The Herschel Science Centre has produced a useful short videotutorial that shows you how to use a useful script in HIPE for SPIRE line fiiting (NB: it does not cover the fully interactive mode of line-fitting). You can find the short videotutorial here:

https://www.youtube.com/watch?v=OYmJzolPF00

The short videotutorial covers the Instrumental Line Shape (ILS) of the SPIRE spectrometer, what preliminary processing of spectra is required, the inputs and outputs of the line-fitting script, and how the script works. Examples are shown of fitting lines to real SPIRE Fourier Transform Spectrometer data, how to inspect the residuals of the fit and to add missing lines in the fitted spectrum. Finally, it will introduce the Spectral Feature Finder Catalogue and how to use it as input for the line fitting script, running a spectrum fit with Spectral feature catalogue.

We have prepared a short videotutorial on Inspecting Data Products in HIPE. This will explain to you the different types of data products that you can find and their content and applicability. You can find it here:

https://www.youtube.com/watch?v=Y7LJK0mE_3I


The HIFI Observation Context consists of six categories of products that are found in the Herschel Science Archive (data at various levels of processing, browse products. calibration products, auxiliary products, quality products and trend analysis products). There is a Herschel Science Centre short videotutorial that explains the products and how to access them. You can find the short videotutorial here:

https://www.youtube.com/watch?v=GlRiqfzvFvk

The short videotutorial will introduce you briefly to the various types of observation context before showing a demonstration of how to examine the Observation Context in HIPE using a HIFI Mapping observation.

The Herschel Science Centre has produced a short videotutorial guide for users of SPIRE spectrometry data to show how to use the observation context and where to find it.

https://www.youtube.com/watch?v=D66Yxz8waVI&t=413s

This short videotutorial gives an introduction to the Observational Context and how to look at it in the Herschel Science Archive. It describes the two types of products at level-2, for Sparse Mode and Mapping Mode. It looks at the Sparse Mode High+Low resolution mode and the detectors and continues with the Mapping Mode, describing the cube structure and the Spectrum2d table. It then looks at the tar Archive file, how to inspect the Observational Context in HIPE and how to use HIPE with data cubes.


If you are an absolutely beginner with Herschel data and want to learn how to do photometry on PACS observations there are two places to start before working with data.

First, you should look at the Quick Start Guide for Photometry (CSG), otherwise known as the PACS Data Reduction Launchpad: Photometry. This will teach you some of the basics of PACS data analysis. It will also guide you towards more detailed documentation that will explore photometry in more depth.

Second, you should consult the Herschel Science Centre short videotutorial on Starting with PACS Photometry. You can find it here:

https://www.youtube.com/watch?v=yLHLA5Jjtiw

The short videotutorial will talk you through the PACS photometry data products that are available in the Herschel Science Archive (HSA) and which products are best for the analysis of what type of object. You will be shown the structure of a PACS photometry observation when downloaded from the HSA and how the filename structure of a product tells you what type of product it is. You will be shown also what Highly Processed Data Products (HPDPs) - also known as "Expert Reduced Data Products" - are available and how they may be used.

There are two excellent starting points for learning how to use PACS spectroscopy data. We recommend strongly that you start by reading the Quick Start Guide to Spectroscopy (also known as the PACS Data Reduction Launchpad: Spectroscopy), which summarises the PACS spectroscopy observing modes, data products, calibration uncertainties and science-readiness of the different PACS spectroscopy products.

There is also a Herschel Science Centre short videotutorial called Starting with PACS Spectroscopy, which will walk you through the basics of PACS Spectroscopy. You can find the short videotutorial here:

https://www.youtube.com/watch?v=rcY487Z9Kyg

This short videotutorial will explain the three basic observing modes for PACS spectroscopy and their uses, how to determine from the Herschel Science Archive (HSA), or from the FITs header in which observing mode an observation was made, detailing the main keywords that users should be familiar with. For each product, you will find out which data products you need and what products are available at different levels for the different observing modes. You will learn what the PACS cubes are and their contents for different modes, PACS data tables and so-called slices for each mode, which are the different presentations of the cubes and tables. You will learn what a PACS spectroscopy observation looks like in the HSA and what post-processing (if any) is required for sources of different sizes. Finally, you will see which Highly Processed Data Products (Expert Reduced Data Products) have been prepared by the experts at the Herschel Science Centre for particularly difficult cases.


If you are an absolutely beginner with Herschel data and want to learn how to do photometry on PACS observations, there are two places to start before working with data.

First, you should look at the Quick Start Guide for Photometry (CSG), otherwise known as the PACS Data Reduction Launchpad: Photometry. This will teach you some of the basics of PACS data analysis. It will also guide you towards more detailed documentation that will explore photometry in more depth.

Second, you should consult the Herschel Science Centre short videotutorial on Starting with PACS Photometry. You can find it here:

https://www.youtube.com/watch?v=yLHLA5Jjtiw

These two resources will teach you the basics of PACS data, its structure, what data is suitable for which purpose and where to begin. To follow this up and learn how to take aperture photometry with PACS data, there is a second Herschel Science Centre short videotutorial that has been prepared that will talk you through the choices and decisions that you must make when carrying out aperture photometry with PACS. You can find it here:

https://www.youtube.com/watch?v=R5S8VnfbNEI

This short videotutorial will show you where to obtain the best product for your object of interest, how to select the best aperture size to use, what the PACS photometer beam profile is, how to calculate the aperture correction (i.e. the lost flux that falls outside the aperture), colour corrections and how to estimate errors and the background. It will also show you where to find and how to personalize a useful script within HIPE to carry out aperture photometry with PACS data.

The Herschel Science Centre has produced a short videotutorial that gives a simple, step-by-step guide for anyone who would like to perform aperture photometry on a SPIRE image. You can find it in the following short videotutorial:

https://www.youtube.com/watch?v=fjzf-VN_X0s

This short videotutorial will show you how to perform aperture photometry, what tools to use and what corrections to perform on the data to get the best final result. The procedure is different depending on whether you have a point source or an extended source calibrated map. In the former case you need to convert from Jy/beam to Jy/pixel, measure the photometry and then apply corrections for the beam, the aperture and the colour. In the latter case, you take the photometry and apply only the colour correction for extended sources. You will be guided through the flowchart for obtaining photometry and will be shown a practical example of data reduction showing the commands required and shows you the different tasks that are available.

 
Reduction and analysis of SPIRE FTS spectra treats two, generic cases: point sources (sources smaller than 3-4 arcseconds that are effectively a point-source for SPIRE) and extended sources (anything larger than 40 arcseconds). However, there are many sources between these two extremes that can neither be treated as point sources, nor as extended. The immediate consequence for observers is that neither the point source reduction, nor the extended source reduction give a good result: users will see this in the form of a large discontinuity between the two SPIRE spectra ranges.

There is a Herschel Science Centre short videotutorial that shows you how to categorise sources by diameter and how to identify cases in which neither the point source calibration, nor the extended source calibration give a satisfactory result and how to treat these cases. You can find it here:

https://www.youtube.com/watch?v=mxZpHgHVtP4

The short videotutorial will walk you through the basics of SPIRE FTS spectra data reduction, including the need to carry out certain, basic pre-reduction such as to subtract the background first before applying the Semi-Extended Calibration Tool (SECT) to the spectra. It will show you how to set the parameters and, through a practical example, how to make an iterative fit to the effective source diameter. Finally, it will show you the checks that you can carry out to ensure a self-consistent result.

If your observation is a spectral map, the products are 3-D spectral cubes and not 1-D spectra. This makes saving data more difficult, as no tools were implemented in HIPE to save a 3-D structure as a text file.

So, unlike for single point spectra, you cannot you cannot use the "Save to text file" option on a spectral cube. So, first you must extract a 1-D spectrum from a pre-selected cube pixel (aka spaxel).

The Herschel Science Centre has created a short videotutorial that explains how to do this. You can find it on the Herschel Academy YouTube channel at:

Saving a 3-D SPIRE FTS spectral cube as a 1D spectrum


The HSC has created a short videotutorial (6 minutes) on how to save an FTS spectrum to a csv file. It's available on the Herschel Academy YouTube channel on the following link:
 
Saving SPIRE FTS spectra as 1D CSV or ASCII files 

 
The HIPE script from the video is copied below. To run it, you need to be logged in to the HSA.

### example HIPE script
obsid =
1342268315
obs = getObservation(obsid,useHsa=True,instrument='SPIRE')
#
point_spectrum = obs.level2.getProduct("HR_spectrum_point")
ext_spectrum = obs.level2.getProduct("HR_spectrum_ext")
#
asciiTableWriter(table=point_spectrum["0000"]["SSWD4"], file='/Users/ivaltcha/test_spec_sswd4.csv')
asciiTableWriter(table=point_spectrum["0000"]["SLWC3"], file='/Users/ivaltcha/test_spec_slwc3.csv')
#### end script

For a simple ASCII file you can also view your spectrum in the Editor window in the HSA. Right click on the spectrum that you wish to save and select "Send to" --> "Text file" in the pop-up menu. These instructions are also shown in the short videotutorial referenced above.

The Herschel Science Archive (HSA) allows multiple categories of searches to be carried out using different criteria. In particular:
  • Search by target name.
  • Search by coordinates (in a radius around a particular position).
  • Search by catalogue (for example, in the Herschel PACS, or Herschel SPIRE Point Source Catalogues, or for data published as a Highly Processed Data Product, or... etc.)
  • Search for unpublished observations.
The Herschel Science Centre has produced a short videotutorial that guides you through the different methods of search in the HSA. It guides you through the use of the HSA Search Bar and how to use the three, individual tabs for specific searches: the search tab, the catalogues tab and the publications tab, as well as some of the more advanced features available when carrying out searches to tailor your results.

You can find the short videotutorial here:

https://www.youtube.com/watch?v=sHWQqp2zGR4&feature=youtu.be



There is a useful script in HIPE that helps the observer to carry out the reduction of spectra of extended sources observed with the SPIRE Fourier Transform Spectrometer. The following Herschel Science Centre short videotutorial explains how to use this script to reduce SPIRE FTS data:

https://www.youtube.com/watch?v=xo0fc1XggIU

The short videotutorial shows you how to visualise the source surroundings to see how much extended emission there is in the field of view, where to find the useful script and how to personalize it for a particular observation, how to overlay the FTS footprint plot on an image of your source, how to assess the background or foreground emission that may contaminate your spectra and how to assess whether or not there are off-axis detectors that you can use to eliminate this contamination. And, finally, for users who prefer to carry out this process outside HIPE, it explains how to do this in an external script.

Assuming that you have watched the Herschel Science Centre short videotutorial called Starting with PACS Spectroscopy:

https://www.youtube.com/watch?v=rcY487Z9Kyg 

And the Herschel Science Centre short videotutorial Point Source Spectroscopy with PACS:

https://www.youtube.com/watch?v=nYWTccEQz_M

Or, at least, are familiar with the contents of both short videotutorials and their associated documentation, you will find that the Herschel Science Centre short videotutorial on Point source spectroscopy with PACS: point sources in the central 3x3 spaxels, which you can find here:

https://www.youtube.com/watch?v=v1W5pT9DYUk

will show you how to proceed if your point source is in the central 3x3 spaxels, but not in the central spaxel itself. This case is valid if you have a point source, with a pointed observation, in which the target is centred within one of the eight spaxels surrounding the central spaxel, or for a mapping observation, in which, for one of the raster positions, the point source is centred within one of the eight spaxels surrounding the central spaxel. You will learn to identify cases for which the process may not work well (contamination, poorly centred point sources, ...) You will learn how to identify the spaxel coordinates of a source and which HIPE scripts and tasks to apply in the different possible cases. It will also show you a "last resort" case that you can use if your source is not even centred in the central 3x3 spaxels. You will also be guided towards the other processing and post-processing that may be helpful, such as the Pointing Offset Correction pipeline, or the Extended-to-point Correction.


Assuming that you have watched the Herschel Science Centre short videotutorial called Starting with PACS Spectroscopy:

https://www.youtube.com/watch?v=rcY487Z9Kyg 

And the Herschel Science Centre short videotutorial Point Source Spectroscopy with PACS:

https://www.youtube.com/watch?v=nYWTccEQz_M

Or, at least, are familiar with the contents of both short videotutorials and their associated documentation, you will find that the Herschel Science Centre short videotutorial on Point source spectroscopy with PACS: point sources in the central spaxel, which you can find here:

https://www.youtube.com/watch?v=_cVncMct3NE

will show you how to proceed if your point source is in the central spaxel itself. The short videotutorial will show you how to use the Product Decision Tree to decide which products and tools to use. This case is valid if you have a point source, with a pointed observation, in which the target is centred within the central spaxel, or for a mapping observation, in which, for one of the raster positions, the point source is centred within one of the eight spaxels surrounding the central spaxel. You will learn to identify cases for which the process may not work well (contamination, poorly centred point sources, ...) You will learn how to identify the spaxel coordinates of a source and which HIPE scripts and tasks to apply in the different possible cases. 

The short videotutorial will show you how to use the extractCentralSpectrum task, which is designed for these cases, how it operates and the corrections that it applies. It will show you how to decide which extraction to apply to the data (c1 - extracts from just the central spaxel, c9 - extracts from the central 3x3 (i.e. 9) spaxels , or c129 - which extracts both and scales the central spaxel to the central 3x3 spaxel box), given the various possible combinations of source flux and centering, although as a general guide, whichever extraction gives the highest flux is the best one to use.

You will see examples of extractions of data showing you what the different extractions show and how to select the most adequate one for you, as well as simple Quality Control checks to apply on the results. It will also show you what other documentation and scripts may be useful to you, as well as giving you indications on how you can do these analysis outside HIPE too.

Before going further, there are two excellent starting points for learning how to use PACS spectroscopy data. We recommend strongly that, if you are unfamiliar with PACS spectroscopy, you start by reading the Quick Start Guide to Spectroscopy (also known as the PACS Data Reduction Launchpad: Spectroscopy), which summarises the PACS spectroscopy observing modes, data products, calibration uncertainties and science-readiness of the different PACS spectroscopy products.

There is also a Herschel Science Centre short videotutorial called Starting with PACS Spectroscopy, which will walk you through the basics of PACS Spectroscopy. You can find the short videotutorial here:

https://www.youtube.com/watch?v=rcY487Z9Kyg

If you are already familiar with these items and are ready to advance to the next stage, there are two further documents that you should read (or, at least, have to hand) that give essential background: The PACS Products Explained and the Product Decision Tree, knowledge of both is implicit in other documents and training materials. These and other interesting documents are in the PACS section of the Herschel Explanatory Legacy Library (HELL).

There is a Herschel Science Centre short videotutorial that will guide you through the basics of the process of working with PACS spectra, giving you the knowledge essential to work with them. You can find it here:

https://www.youtube.com/watch?v=nYWTccEQz_M

The short videotutorial will explain what is meant by point source spectroscopy and the caveats and corrections that need to be applied, showing the footprint of the PACS Integral Field Unit on a typical point source, showing the re-binned and interpolated cubes that are an essential intermediate step in PACS spectroscopy reduction. It will show you the main keywords to look for to know the observing mode and how to determine where the point source is in the field of view for different observing modes, which is essential information for knowing which case to apply in data reduction and which product you need to use. It will then show you how to decide which HIPE task or script to use for different cases, depending on where the target is located in the field of view.

According to the source centering case that you find, you should then continue with one of the two specialist short videotutorials that deal with the different special cases:

Point sources in the central 3x3 spaxels.

Point sources in the central spaxel.

 

Name searches in the HSA only work by cross-referencing with either the SIMBAD database of (mainly) stellar objects or the NED database of extragalactic objects that both, logically, exclude Solar System Objects. These must be identified using their NAIF ID, selecting the “NAIF ID” tab, as there is no name-searcher available to the HSA for Solar System Objects.

For comets and asteroids, the NAIF ID is a unique, 7-digit identifier. The NAIF ID for comets has the format 100****, while for numbered asteroids, it is 2******, with the digits following the leading “2” being the asteroid number. A few objects observed by Herschel were un-numbered asteroids with lower-quality orbits, which have the format 3******. You may also input a text file with a list of NAIF IDs to check.

The HSA uses wild-cards automatically. If you enter 100 and press Search, you will obtain a listing of all comet observations.

For planets and their satellites, the NAIF ID is 3-digits starting with the planet number in order of distance from the Sun (Mars = 4, Jupiter = 5, etc). The planet itself is 499, 599, 699, … while the satellites are the planet number followed by the satellite number so, Titan, or Saturn VI, is 606, while Callisto, Jupiter IV, is 504.

To return the NAIF ID for any named Solar System Object, enter the following line in a browser:

https://ssd.jpl.nasa.gov/x/smb_spk.cgi?OPTION=Look+up&OBJECT=ceres

Just replace the name "ceres" with the solar system object that you wish to identify (name - if named [e.g. Haumea], number [e.g. 2000], or provisional designation [e.g. 1950 DA]).

The Herschel Science Centre has produced a useful short videotutorial, How to use the Herschel Science Archive User Interface, to show new users how to access and use the Herschel Science Archive and about the data products that are available within it. You can find this short videotutorial here:

http://youtu.be/IgGNpUgubkw


Helpdesk tickets are linked to your registered e-mail address. This means that the replies will always go to that address until you change it. You can change your e-mail as explained in the following page, under "Edit your user profile":

When you change your e-mail address, all new tickets that you open will be linked to your new e-mail. Old tickets though will continue to be linked to the address with which the ticket was opened, as a ticket cannot be "owned" by two e-mail addresses and thus, by two separate people in the registration system (this is a basic security feature that ensures that your correspondence within Helpdesk can only be seen by you and by HSC personnel). So, if you open a ticket, then change your registered address and try to respond to the ticket again by clicking on the link in the e-mail reply, the system will get confused and refuse you access.

You can still see your tickets and respond to them by entering directly in the user interface with your username and password and, can, if necessary re-submit a ticket using your new credentials to link it to your new e-mail address and continue corresponding.


We have prepared a short videotutorial to explain to users how to use the Herschel Science Archive User Interface. You can find it here:

http://youtu.be/IgGNpUgubkw


HSpot will now run only on older computers, for example, those running Windows 7 and earlier and very old versions of Mac. In a nutshell, the JRE (Java Runtime Environment) fails to install on many more recent OS/Platform combinations.

However, there is a virtualized version of HSpot that has been prepared that works with almost any OS/Platform combination (save for a few, more exotic Linux Platforms) and that allows proposal retrieval, examination of AORs, etc to be carried out as if with classic HSpot.

In the following Herschel short videotutorial you can find out how to install, configure and run Virtual HSpot on your computer:

https://www.youtube.com/watch?v=910Gj7_IAmg

 

Classic HSpot will no longer install on recent OS/Platform combinations (for Windows, later than Windows 7; for Mac, only on platforms previous to Mountain Lion; etc.) Technically, the JRE (Java Runtime Environment) fails to install correctly. To allow HSpot to work on different current and future OS/Platform combinations, a virtualised version of HSpot has been prepared, running a Debian Linux version of HSpot on VirtualBox. Full instructions for installation of Virtual HSpot, the HSpot 8.0 release,  can be found here:

https://www.cosmos.esa.int/documents/12133/1028864/Virtual+HSpot+Quick+Start+Guide+HSpot+8.0

You can find a Herschel Science Centre short videotutorial explaining how to install and run Virtual HSpot, here:

https://www.youtube.com/watch?v=910Gj7_IAmg

Herschel Science Centre has released also a useful short videotutorial explaining the uses and limitations of HSpot after post-Operations have finished. You can find the videotutorial here:

https://www.youtube.com/watch?v=MsUZAfmGXcU




 

Herschel Helpdesk is not configured to accept e-mail. The address hschelp@sciops.esa.int is a no-reply address that only manages Helpdesk traffic. To send a question you must enter https://support.cosmos.esa.int/herschel/ and send the question from your account, logging in with your credentials as a registered Herschel user.


Assuming that you have watched the Herschel Science Centre short videotutorial called Starting with PACS Spectroscopy:

https://www.youtube.com/watch?v=rcY487Z9Kyg 

And the Herschel Science Centre short videotutorial Point Source Spectroscopy with PACS:

https://www.youtube.com/watch?v=nYWTccEQz_M

Or, at least, are familiar with the contents of both short videotutorials and their associated documentation, you will find that the Herschel Science Centre short videotutorial on PACS Cubes in Detail here:

https://www.youtube.com/watch?v=CxZF9-9D8UM

The PACS spectrometer produces five types of data cube and one table as products, although not all types of data have all types of cube. You will learn about each type of cube (re-binned cubes, mosaic cubes - including interpolated, projected, drizzled equidistant mosaic cubes), how and why it is produced and what data it contains, as well as the re-binned cube tables. You will learn also what useful scripts are available to transform cubes to a product of your own personal specifications (e.g. one that has spaxels of a different size).

Answer supplied by David Teyssier:

 

The calibration uncertainties vary from one instrument to another. A summary of those is provided in each of the instrument overview pages, namely:

HIFI: https://www.cosmos.esa.int/web/herschel/hifi-overview

PACS: https://www.cosmos.esa.int/web/herschel/pacs-overview

SPIRE: https://www.cosmos.esa.int/web/herschel/spire-overview

The calibration uncertainties are provided in the Performance and Calibration section, typically with a distinction between absolute and relative (repeatability) accuracy. For HIFI a dedicated calibration uncertainty budget table exists per observation, and can be retrieved as an HPDP - see https://www.cosmos.esa.int/web/herschel/highly-processed-data-products.

The Herschel Science Centre has produced a number of short videotutorials about the analysis of data with the Herschel instruments. These contain information on the uncertainties in photometry and on the flux calibration of spectroscopy and how to treat the different effects that may influence it. You can find the videotutorials, which we recommend strongly viewing, here:

https://www.cosmos.esa.int/web/herschel/legacy-training-materials


The Herschel Science Centre has produced two simple, step-by-step guides for anyone who would like to perform photometry on a SPIRE image. These give details of how to perform photometry and the corrections that must be performed on the raw photometry.

The first short videotutorial explains how to perform aperture photometry on SPIRE maps and gives an overview of the corrections that are required, If you are familiar with SPIRE photometry, you can pass straight to the second video, although we recommend that users watch both:

https://www.youtube.com/watch?v=fjzf-VN_X0s

The second short videotutorial explains in more detail the steps in applying corrections to the raw photometry, in particular, the sequence of the corrections to apply and how to apply the colour correction to SPIRE photometry are explained. This videotutorial shows practical examples of applying the corrections to real data from SPIRE:

https://www.youtube.com/watch?v=i98LQWEwd8I


Ancillary Data Products (ADPs) are data products that correspond to an instrument or too the observatory rather than to a single science observation. These include a wide variety of engineering and diagnostic data. The Herschel Science Centre has produced a short videotutorial with an overview of the many HIFI ADPs that are available. You can find the short videotutorial here:

https://www.youtube.com/watch?v=_PTe-SkVVrc

The HIFI ADPs include the PSF for the instrument, the prime calibrator models (Mars), plus a large quantity of engineering, trend analysis and quality data. Examples of the ADPs and the data that they contain are shown in the short videotutorial, such as the HIFI 2D beam model, the azimuthally averaged profile and the Encircled Energy Fraction and the gas cell spectra from pre-launch.


Highly processed data products are sets of data that have been reduced by experts working for the Herschel Science Centre because they have issues that makes them extremely difficult to reduce optimally by automatic pipeline, or even by a non-expert in interactive mode. The Herschel Science Centre has produced a short videotutorial that gives a guided tour of the different HIFI Highly Processed Data Products (HPDPs) that are available. You can find the short videotutorial here:

https://www.youtube.com/watch?v=AV9HwZYQt_I

The short videotutorial will introduce you to the types of HIFI HPDPs that are available, before showing you the scope and content of each HPDP dataset, with a description and examples of  the products that it contains: spectral scans, spectral maps, off spectra (reference position spectra), the HIFI lines catalogue, and flux uncertainty budget tables.

The Herschel Science Archive (HSA) provides a large amount of information in response to a search for observations of a particular target. In particular:
  • How many observations there are in the HSA for this target, their status and quality.
  • What publications have been produced from each observation.
  • What User Provided Data Products are available for each observation.
  • What Highly Processed Data Products are available for each observation and how these compare with the standard pipeline products.
  • How to filter searches to obtain more, or less information.
The Herschel Science Centre has produced a short videotutorial that guides you through the results of searchs in the HSA. It guides you through the use of the HSA tabs and how to use the  individual tabs to obtain specific information about individual observations.

You can find the short videotutorial here:

https://www.youtube.com/watch?v=sHWQqp2zGR4&feature=youtu.be

 

The formal end of Herschel Science Centre support to Herschel activities is 31st December 2017. From then, all support is on a best-efforts basis, although it is expected that there will be a small level of dedicated effort for user support until at least the end of 2018 and, probably, 2019.



The Herschel Explanatory Legacy Library (HELL) can be found at the url:

https://www.cosmos.esa.int/web/herschel/legacy-documentation

It is arranged on three levels hanging from an Explanatory Supplement: one for the Observatory and one for each of HIFI, PACS and SPIRE. Level 1 documentation is the most basic and important, while Level 2 is more detailed and technical. Level 3 documentation is the most technical and, in many cases, is historical and mainly for instrument experts.

There is a catalogue of documentation at the url:

https://www.cosmos.esa.int/documents/12133/1028864/All+documentation+in+HELL+PDF+format (PDF)

or

http://herschel.esac.esa.int/twiki/pub/HSC/HerschelLegacyDocumentation/All_documentation.xlsx (Excel worksheet)

You can find a short video tutorial about HELL here:

https://www.youtube.com/watch?v=M3NXs6sRVr0

And a more detailed explanation of the catalogue and its contents in this short video tutorial:

https://www.youtube.com/watch?v=8mNlhGgeTEY


A series of short, legacy tutorials have been produced, covering all aspects of data reduction, data analysis, the Herschel Science Archive (HSA) and Herschel Legacy documentation. You can find a listing of all available videotutorials here:

https://www.cosmos.esa.int/web/herschel/legacy-training-materials

These videotutorials are available as a playlist on the Herschel Academy YouTube Channel at:

https://www.youtube.com/user/hipeacademy

There are many training videos available on the Herschel Academy side.

Go to the URL

https://www.youtube.com/playlist?list=PLu1dCeIcs9P6Mb8VLDT54Xqq1RBMifDOU

For the Herschel legacy short videotutorial playlist.

Answered supplied by David Teyssier:

 

The HIFI spatial response is described in the following document:

http://herschel.esac.esa.int/twiki/pub/Public/HifiDocsEditableTable/HIFI_Beam_AncillaryDataProducts_ReleaseNotes.pdf

On top of that, 2D beam profiles and 1D Encircled Energy Fraction for each HIFI band are available as Ancillary Data Products, and can be retrieved at:

http://archives.esac.esa.int/hsa/legacy/ADP/PSF/HIFI/

 


There are two, well-known sources of potential problems:

First, check that you are using the secure "https" protocol and not "http". If you have the latter, the link will fail.

Second, it is possible that the script is searching to the wrong directory because, unfortunately there were some changes to the URL where the on-line kernels were kept that were not within the control of the Herschel Science Centre. In order to fix the directory error, you have to edit the PhotometryConvoloveResolutionKernel.py script and change this line (line #424):

kernelRoot = '
http://www.astro.princeton.edu/~ganiano/Kernels/Ker_2012_May/Kernels_fits_Files/Hi_Resolution/
'

With this line:

kernelRoot = '
https://www.astro.princeton.edu/~ganiano/Kernels/Ker_2012/Kernels_fits_Files/Hi_Resolution/
'

This should resolve both issues.

There are two, well-known sources of potential problems:

First, check that you are using the secure "https" protocol and not "http". If you have the latter, the link will fail.

Second, it is possible that the script is searching to the wrong directory because, unfortunately there were some changes to the URL where the on-line kernels were kept that were not within the control of the Herschel Science Centre. In order to fix the directory error, you have to edit the SpectroscopyConvolveResolutionKernel.py script and change this line:

kernelRoot = '
http://www.astro.princeton.edu/~ganiano/Kernels/Ker_2012_May/Kernels_fits_Files/Hi_Resolution/
'

With this line:

kernelRoot = '
https://www.astro.princeton.edu/~ganiano/Kernels/Ker_2012/Kernels_fits_Files/Hi_Resolution/
'

This should resolve both issues.

 

Normally the reason is that the e-mail address associated with your profile is no longer valid. You will still be able to see answers if you enter Helpdesk but, logically, the replies sent by e-mail will not arrive. You should update your profile by going to http://www.cosmos.esa.int/web/herschel/registration and change the e-mail associated with your name.

If this does not work, contact Helpdesk and we will investigate the problem.



Even though HSpot is no longer used for preparing and submitting Herschel proposals, it still has many interesting and useful functionalities to examine observations taken with Herschel.

We have prepared two short videotutorials that explain on which platforms HSpot will still work and how to install and run a virtualized version of HSpot that is platform independent. The videos also show you some of the features of HSpot and how to exploit them.

The videotutorial on classic HSpot is available here:

https://www.youtube.com/watch?v=MsUZAfmGXcU

And the videoguide to installing and running Virtual HSpot, is here:

https://www.youtube.com/watch?v=910Gj7_IAmg


Even though most Herschel data is now processed to the highest degree that is possible with the automatic processing pipeline and the results cannot be improved significantly, there are still datasets for which human intervention, particularly in post-processing, can improve the result. Many videotutorials are directed at users of Herschel data that benefit from the additional analysis and post-processing that HIPE can offer.

You can find a full list of videotutorials, sorted by instrument, at this URL:

https://www.cosmos.esa.int/web/herschel/legacy-training-materials

If you wish to learn about HIPE and how to install it on your computer, you should check this videotutorial on how to  install HIPE and how to import and to inspect data products:

https://www.youtube.com/watch?v=H9vof8iyqSc

 

The Herschel Helpdesk will continue to function after Herschel’s retirement, although, logically, the available support will be more limited than it has been in Operations and post-Operations, so it may on occasion take a little longer to answer questions, particularly if they require a detailed answer.

Most popular articles 
 
Newest articles