{ "cells": [ { "cell_type": "markdown", "id": "intro", "metadata": {}, "source": [ "# Downloading FITS files and plotting a spectrum\n", "\n", "You have found the observations you want — now get the spectra onto disk and\n", "look at one. This tutorial queries a few NEID **Level-1** frames, downloads\n", "them in parallel, opens one with `astropy.io.fits`, and plots its extracted\n", "flux against wavelength.\n", "\n", "```{note}\n", "This notebook runs against the live NEID server. The published version here\n", "uses synthetic data so it builds without an archive account — no files are\n", "actually fetched, and the spectrum at the end is illustrative.\n", "```\n", "\n", "## Why L1?\n", "\n", "An {term}`L1` frame is a calibrated, wavelength-solved 1D\n", "{term}`extracted spectrum` — the star's flux at each wavelength, pulled out of\n", "the raw 2D detector image, with one spectrum per echelle order. It is the level\n", "you reach for when you want the **spectra themselves** (for custom line or\n", "activity analysis), as opposed to {term}`L2`, which gives you the radial\n", "velocity the pipeline already measured. See {doc}`../concepts/data-levels`.\n", "\n", "Most NEID data is public and downloads without a login. Reaching data still\n", "inside its {term}`proprietary period` requires `neid.login()` with IPAC NEID\n", "credentials — see {doc}`../concepts/downloading-and-proprietary`." ] }, { "cell_type": "code", "execution_count": null, "id": "setup", "metadata": { "tags": [ "remove-cell" ] }, "outputs": [], "source": [ "import os\n", "from pathlib import Path\n", "\n", "from astropy.table import Table\n", "\n", "OFFLINE = bool(os.getenv(\"NEXSCI_RV_OFFLINE_FIXTURES\"))\n", "\n", "if OFFLINE:\n", " from unittest.mock import patch\n", "\n", " _fake_query = Table(\n", " {\n", " \"l1filename\": [\"neidL1_20241101T031200.fits\", \"neidL1_20241101T032500.fits\"],\n", " \"l1filepath\": [\n", " \"/path/handle/neidL1_20241101T031200\",\n", " \"/path/handle/neidL1_20241101T032500\",\n", " ],\n", " \"obsdate\": [\"2024-11-01T03:12:00\", \"2024-11-01T03:25:00\"],\n", " \"object\": [\"HD 189733\", \"HD 189733\"],\n", " }\n", " )\n", " _patcher_q = patch(\"nexsci_rv.tap.TapClient.query\", return_value=_fake_query)\n", " _patcher_q.start()\n", "\n", " _patcher_d = patch(\n", " \"nexsci_rv.base.parallel_download\",\n", " return_value=[\n", " Path(\"data/neidL1_20241101T031200.fits\"),\n", " Path(\"data/neidL1_20241101T032500.fits\"),\n", " ],\n", " )\n", " _patcher_d.start()\n", " print(\"Offline mode: queries and downloads return synthetic data.\")" ] }, { "cell_type": "markdown", "id": "query-md", "metadata": {}, "source": [ "## Query a small batch of L1 frames" ] }, { "cell_type": "code", "execution_count": null, "id": "query", "metadata": {}, "outputs": [], "source": [ "from nexsci_rv import NEIDArchive\n", "\n", "neid = NEIDArchive()\n", "\n", "tbl = neid.query_datetime(\n", " \"l1\",\n", " (\"2024-11-01\", \"2024-11-02\"),\n", " columns=[\"l1filename\", \"l1filepath\", \"obsdate\", \"object\"],\n", " maxrec=2,\n", ")\n", "tbl" ] }, { "cell_type": "markdown", "id": "939ed746", "metadata": {}, "source": [ "We deliberately kept two columns that {term}`download` needs:\n", "\n", "- `l1filename` — the name of each L1 {term}`FITS` file.\n", "- `l1filepath` — the server-side handle download uses to build the fetch URL.\n", "\n", "If you narrow a result with `columns=` and drop these, `download()` cannot\n", "locate the files. (`obsdate` and `object` are here just for orientation.) See\n", "{doc}`../concepts/results-columns` for the full set of\n", "{term}`filename / filepath / filehand` keys." ] }, { "cell_type": "markdown", "id": "download-md", "metadata": {}, "source": [ "## Download in parallel\n", "\n", "`download()` returns the list of successfully downloaded paths. Partial\n", "failures raise `DownloadError` *after* completing the others; the exception\n", "carries a `.failures` list of `(filename, exception)` tuples for inspection.\n", "\n", "- `workers=4` by default (astronomy archives throttle aggressively).\n", "- `resume=True` skips files already present at full size.\n", "- `progress=True` shows a `tqdm` progress bar." ] }, { "cell_type": "code", "execution_count": null, "id": "download", "metadata": {}, "outputs": [], "source": [ "outdir = Path(\"data\")\n", "outdir.mkdir(exist_ok=True)\n", "\n", "paths = neid.download(tbl, outdir=outdir, workers=2, progress=False)\n", "for p in paths:\n", " print(p)" ] }, { "cell_type": "markdown", "id": "open-fits-md", "metadata": {}, "source": [ "## Open one of the FITS files\n", "\n", "A {term}`FITS` file holds the data itself plus header metadata. The header in\n", "the primary HDU records what was observed (target, time, exposure, conditions),\n", "while the {term}`L1` flux and wavelength arrays live in named image extensions —\n", "`hdul.info()` lists them all. NEID L1 stores one row per echelle order, so we\n", "read a single representative order to plot.\n", "\n", "(Skipped in offline mode, since no file was actually downloaded.)" ] }, { "cell_type": "code", "execution_count": null, "id": "open-fits", "metadata": {}, "outputs": [], "source": [ "if OFFLINE:\n", " print(\"(skipped in offline mode — no file was downloaded)\")\n", "else:\n", " from astropy.io import fits\n", "\n", " with fits.open(paths[0]) as hdul:\n", " hdul.info()\n", " hdr = hdul[0].header\n", " for key in (\"OBJECT\", \"DATE-OBS\", \"EXPTIME\", \"OBSMODE\", \"AIRMASS\"):\n", " print(f\"{key:>10s}: {hdr.get(key, '?')}\")\n", "\n", " # NEID L1 stores the extracted spectrum as 2D arrays\n", " # (echelle order x pixel) in named image extensions. Pick one\n", " # representative order to plot. (Adapt the extension/order names if\n", " # your pipeline version differs; \"SCIFLUX\"/\"SCIWAVE\" are the L1\n", " # science arrays.)\n", " order = 60\n", " spec_flux = hdul[\"SCIFLUX\"].data[order]\n", " spec_wavelength = hdul[\"SCIWAVE\"].data[order]" ] }, { "cell_type": "markdown", "id": "3223512d", "metadata": {}, "source": [ "## Plot the extracted spectrum\n", "\n", "Finally, the payoff: flux against wavelength for one echelle order. You should\n", "see the continuum with narrow dark stellar absorption lines dipping below it —\n", "the very features whose Doppler shift the L2 pipeline turns into a\n", "{term}`radial velocity (RV)`. In offline mode we plot a single synthetic line\n", "so the figure renders without real data." ] }, { "cell_type": "code", "execution_count": null, "id": "48f38c4d", "metadata": {}, "outputs": [], "source": [ "import matplotlib.pyplot as plt\n", "import numpy as np\n", "\n", "if OFFLINE:\n", " wave = np.linspace(5000, 5050, 500)\n", " flux = 1 - 0.5 * np.exp(-0.5 * ((wave - 5025) / 0.3) ** 2)\n", "else:\n", " # from the opened L1 FITS HDU (the order extracted in the cell above)\n", " wave, flux = spec_wavelength, spec_flux\n", "\n", "fig, ax = plt.subplots(figsize=(8, 3))\n", "ax.plot(wave, flux, lw=0.8)\n", "ax.set_xlabel(\"wavelength (Å)\")\n", "ax.set_ylabel(\"normalized flux\")\n", "ax.set_title(\"NEID L1 extracted spectrum\")\n", "fig.tight_layout()" ] }, { "cell_type": "markdown", "id": "partial-failure", "metadata": {}, "source": [ "## Handling partial failures\n", "\n", "If any file fails (network blip, 5xx after retries), `download()` raises\n", "`DownloadError` carrying the successful paths in the underlying state and the\n", "failed filenames in `.failures`. Catch it to recover what you can:\n", "\n", "```python\n", "from nexsci_rv import DownloadError\n", "\n", "try:\n", " paths = neid.download(tbl, outdir=outdir)\n", "except DownloadError as exc:\n", " for name, err in exc.failures:\n", " print(f\"failed: {name} ({err})\")\n", "```" ] } ], "metadata": { "kernelspec": { "display_name": "Python 3", "language": "python", "name": "python3" }, "language_info": { "name": "python", "version": "3.12" } }, "nbformat": 4, "nbformat_minor": 5 }