Querying vs downloading, and proprietary data

Getting files from the archive is two steps, and it helps to keep them separate.

Query first, download second

A query gives you an astropy Table describing the matching observations — times, targets, RV values, file names. It does not fetch any data files. To get the actual FITS files, pass that table to download:

table = neid.query_datetime("l2", ("2024-11-01", "2024-11-06"))
paths = neid.download(table, outdir="data/")

download returns a list of pathlib.Path objects pointing at the files it fetched.

Note

download finds each file using the filename / filepath / filehand columns of your table. If you narrowed the result with columns= and dropped those columns, download cannot locate the files. Keep l2filename and l2filepath (or the l0/l1 equivalents for your level) in any selection you plan to download.

Explore small, then pull what you need

While you are still working out the right query, keep results small with maxrec (e.g. maxrec=10) so each round-trip is fast. Once the query returns the rows you want, drop or raise the cap and download only those.

When you need to log in

Some archives hold data under a proprietary period (often about a year), during which only the observing team can reach it; after that it becomes public.

  • Public data needs no login — query and download freely.

  • Proprietary data (still inside its window) requires login with your archive credentials, e.g. neid.login(). The client caches your session, so you log in once.

  • Fully public archives (such as the EXPRES solar data) have no login at all.

If a query or download of proprietary data fails with AuthExpired, your session expired; call login() again. Each archive’s page says whether it has proprietary data.

Very large pulls

A normal query_* call is synchronous and returns immediately, which is fine for bounded queries. For very large or multi-year pulls that would time out, use the async TAP job (AsyncJob) path instead: submit_async(), then .wait() and .result(). The server works in the background while you wait. See TAP and ADQL, just enough.