## Deadline for the *Exoplanet Imaging Data Challenge* Phase 2:

31st of May 2024

**EvalAI platform:** The results of the Phase 2 (characterization) of the *Exoplanet Imaging Data Challenge* must be submitted on EvalAI.

The second phase of the *Exoplanet Imaging Data Challenge* is focused on the characterization of point source: it consists in performing *two* tasks, each one will have its own type of submission and metrics.

- For each submission (one submission per algorithm), the participant must provide
**one zip file per task**. - In each zip file,
**one MEF (multi-extension**.*.fits*file) file per data set -
In total, the submission consists of
**two**, each consisting of*.zip*files**eight**.*MEF .fits*files

### Task 1 (astrometry)

For each target star, the participant must provide one MEF `.fits`

file containing:

(i) the estimated position from the star (in other words, the star is located at the coordinates [0,0]), in pixels;

(ii) [optional] the 1-sigma uncertainties on the estimated position;

(iii) [optional] the corresponding posterior distribution used to estimate the position and its uncertainties.

If the posterior distribution (iii) is not provided, we will assume that the posterior follows a normal distribution.

**Note:** One MEF .fits file contains the information of all the injections (2 to 3) of a given data set.

**Task 1: filename:** Please call all your files following this naming convention **astrometry_instID.fits**, with **inst** the instrument in lower case (e.g. `sphere`

or `gpi`

) and **ID** the dataset index (e.g. `1`

, `2`

, `3`

or `4`

).

### Task 2 (spectrophotometry)

For each target star, the participant must provide one MEF `.fits`

file containing:

(i) the estimated contrast wrt the star;

(ii) [optional] the 1-sigma uncertainties on the estimated contrast;

(iii) [optional] the corresponding posterior distribution used to estimate the contrast and its uncertainties.

If the posterior distribution (iii) is not provided, we will assume that the posterior follows a normal distribution.

**Note:** One MEF .fits file contains the information of all the injections (2 to 3) of a given data set.

**Task 2: filename:** Please call all your files following this naming convention **photometry_instID.fits**, with **inst** the instrument in lower case (e.g. `sphere`

or `gpi`

) and **ID** the dataset index (e.g. `1`

, `2`

, `3`

or `4`

).

### Example of submission files

#### Task 1:

For example, for submitting your results of the 1st task (astrometry), you must gather the following eight files in *.zip* format (any name can be used for the *.zip* file - please do not use any subfolder!):

`astrometry_gpi1.fits`

,`astrometry_gpi2.fits`

,`astrometry_gpi3.fits`

,`astrometry_gpi4.fits`

,`astrometry_sphere1.fits`

,`astrometry_sphere2.fits`

,`astrometry_sphere3.fits`

,`astrometry_sphere4.fits`

.

Each file must *at least* contain the 1st extension with the position values in x and y of each candidate, following the order given by the `first_guess_astrometry_instID.fits`

file provided with the data sets (and same numpy array format - e.g. 3 rows (ny) of 2 columns (nx) for the xy coordinates of 3 planets).

#### Task 2:

For example, for submitting your results of the 2nd task (photometry), you must gather the following eight files in *.zip* format (any name can be used for the *.zip* file - please do not use any subfolder!):

`photometry_gpi1.fits`

,`photometry_gpi2.fits`

,`photometry_gpi3.fits`

,`photometry_gpi4.fits`

,`photometry_sphere1.fits`

,`photometry_sphere2.fits`

,`photometry_sphere3.fits`

,`photometry_sphere4.fits`

.

Each file must *at least* contain the 1st dimension with the contrast estimates along each spectral channel, for each candidate, following the order given by the `first_guess_astrometry_instID.fits`

file provided with the data sets (and same numpy array format - e.g. 3 rows (ny) of 39 columns (nx) for the spectra of 3 planets).

**File format:** Every file must be in MEF **.fits** format. In the EIDC2 Github repository, you will find a short tutorial, as a jupyter notebook, to put your results into a MEF .fits file.

**Submission into a .zip file:** All of the 8 MEF .fits files must be submitted within **a single .zip file**, with a flat structure (without subfolder structure, e.g. using the command on Mac > `zip -r -X -j archive_name.zip folder_to_compress`

).
One .zip file must be submitted for each task (one for **astrometry** and one for **spectrophotometry**).

**Note:** Each submission must correspond to the results of applying **a single algorithm to all the datasets**. You can therefore make several submissions, even if only one will show up on the competition webpage leaderboard, all of the submissions will be processed off-line.

**IMPORTANT:** If you really cannot detect the companion, despite the first guess provided, please put `-1`

as an entry (for both estimated value and its uncertainties).

### Evaluation metrics

For the leaderboard display, the EIDC team has decided to use simple metrics: the metrics consist in mean normalized distances between the estimated value submitted by the participant and the ground truth used for injection. The distance is based on the L2 and L1 norms for astrometry and spectrophotometry, respectively.

**Task 1 (astrometry)**

For each injected companion, we compute the L2-norm distance (i.e. the Euclidian distance) between the estimated position value (xy_est) and the corresponding ground-truth value (xy_gt):

`dist_astro = sqrt| | x_est - x_gt |^2 + | y_est - y_gt |^2 |`

We then average all the distances *dist_astro* obtained for each of the 21 exoplanet injections.

**Task 2 (spectrophometry)**

For each injected companion, we compute the L1-norm distance between the estimated contrast value (cont_est) and the ground-truth value (cont_gt), for each wavelength (lambda), and normalize it with the ground truth contrast (to not penalize smaller true contrasts in the averaged score):

`dist_photo = sum| | cont_est_lambda - cont_gt_lambda | / cont_gt_lambda |`

, (sum over all the wavelength)

We then average all the distances *dist_photo* obtained for each of the 21 exoplanet injections.

Further analysis of the results will be conducted off-line by our team. The metric used for the phase 2 of the *Exoplanet Imaging Data Challenge* are published in the proceeding accessible here.

The final comparison results will be published an SPIE Astronomical telescopes + instrumentation conference proceeding in summer 2024.

### Potential error message

**Other error message:** Please contact us if you encounter any issue when submitting your results exoimg.datachallenge@gmail.com.

### EvalAI team:

- Anthony Cioppa (ULiège, Belgium): putting the challenge and EvalAI in place, tests.
- Carles Cantero (ULiège, Belgium): writing the backend for the leaderboard, tests.
- Faustine Cantalloube (LAM, France): coordination, description and tests.